May 2014 - dovecot - dovecot.org

[Dovecot] Problem with quota calculation...
by Klaus Tachtler 30 May '14

30 May '14

Hello, since I changed the quota backend from maildir (maildir:User quota) to dictionary (dict:User quota::proxy::quota), I have problems with tha calculated size and messages count. --- Input/Output --- # du -hs 23M . # doveadm quota get -u ich(a)tachtler.net Quota name Type Value Limit % User quota STORAGE 46602 1024000 4 User quota MESSAGE 166 - 0 # doveadm quota recalc -u ich(a)tachtler.net # doveadm quota get -u ich(a)tachtler.net Quota name Type Value Limit % User quota STORAGE 23301 1024000 2 User quota MESSAGE 83 - 0 --- Input/Output ---- It seems, that all was counted twice? ----- doveconf -n ----- # dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.17.1.el6.x86_64 x86_64 CentOS release 6.5 (Final) auth_master_user_separator = * auth_mechanisms = plain digest-md5 cram-md5 login auth_verbose = yes dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } hostname = dovecot.tachtler.net listen = * mail_location = maildir:~/Maildir mail_plugins = " quota acl zlib mail_log notify" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace { list = children location = maildir:%%h/Maildir:INDEX=%h/shared/%%u:CONTROL=%h/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db mail_log_fields = uid box msgid size from quota = dict:User quota::proxy::quota quota_grace = 10%% quota_rule = INBOX/Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is over quota quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster(a)tachtler.net protocols = imap lmtp sieve service auth { unix_listener auth-userdb { group = vmail user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_min_avail = 1 service_count = 0 } service lmtp { inet_listener lmtp { address = 192.168.0.80 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = 192.168.0.80 port = 12340 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_cert = </etc/pki/dovecot/certs/CAcert-class3-wildcard_all_in_one.crt ssl_key = </etc/pki/dovecot/private/tachtler.net.key userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = " quota acl zlib mail_log notify sieve" } protocol imap { mail_plugins = " quota acl zlib mail_log notify imap_quota imap_acl imap_zlib" } Thank you for any help, Klaus. -- ------------------------------------------ e-Mail : klaus(a)tachtler.net Homepage: http://www.tachtler.net DokuWiki: http://www.dokuwiki.tachtler.net ------------------------------------------

3 2

[Dovecot] Filed to write auth token secret file
by Chris Vaas 29 May '14

29 May '14

Hey guys, I am getting the following error. It seems not be be severe, since my setup works without any signs of drawbacks. But I'd rather rely on a professional opinion. May 28 23:02:54 example dovecot: auth: Error: open(/var/run/dovecot/auth-token-secret.dat.tmp) failed: Permission denied May 28 23:02:54 example dovecot: auth: Error: Failed to write auth token secret file; returned tokens will be invalid once auth restarts Thanks in advance Chris

2 4

Re: [Dovecot] Corrupted Mail?
by Joseph Tam 29 May '14

29 May '14

> * ^List-Id:.*users.lists.fedoraproject.org > | $DELIVER -m $DESTDIR/MLists/Fedora You removed the '-d' parameter -- it may or may not work without it, but you'll need it in a virtual user setup. I'm not sure whether procmail sets $USER by itself or it gets passed as an environment variable, so if $USER is not working, use $LOGNAME, or better yet, just hardcode the value if this is your own script. Also, the -m parameter is the name of a mailbox, not a pathname, so I think "MLists/Fedora" is what you want. > This is what I found in the procmail log: > lda: Fatal: destination user parameter (-d user) not given Well, this error couldn't be more explicit, can it? Joseph Tam <jtam.home(a)gmail.com>

1 0

[Dovecot] dovecot 2.1.15 hangs while connecting.
by Albert Whale 28 May '14

28 May '14

I am using Dovecot in two Server environments. My Legacy (5 years+ old) servers are running dovecot 1.1.20, and are responding as expected. These servers are using an NFS based storage for the /home directory structure, and are working beautifully. However, I need to replace these inefficient servers, and get more current. That brings me to the problem, where I am implementing Dovecot 2.1.15, which does not complete the authentication cycle. I have converted the 1.1.20 configuration using the documentation available, and I have the following for my running configuration (which is not working): # 2.1.15: /etc/dovecot/dovecot.conf # OS: Linux 3.8.13.4-server-1.mga3 i686 Mageia 3 auth_debug = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no lock_method = dotlock login_greeting = SpamZapper Email ready. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_privileged_group = mail mmap_disable = yes passdb { driver = pam } protocols = imap pop3 service auth { user = root } service imap-login { chroot = login client_limit = 256 user = dovecot } service pop3-login { chroot = login client_limit = 256 user = dovecot } ssl = no userdb { driver = passwd } verbose_proctitle = yes protocol pop3 { mail_max_userip_connections = 50 pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_uidl_format = %08Xv%08Xu } protocol lda { postmaster_address = postmaster(a)example.com } I have captured the Debug output of the dovecot process while attempting to connect: May 27 13:02:10 ns dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011session=IzAYqGT6pgBCz4Xk#011lip=66.207.133.227#011rip=66.207.133.228#011lport=110#011rport=38566#011resp=<hidden> May 27 13:02:10 ns dovecot: auth-worker(551): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth May 27 13:02:10 ns dovecot: auth-worker(551): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so May 27 13:02:10 ns dovecot: auth-worker(551): Debug: pam(aewhale,66.207.133.228): lookup service=dovecot May 27 13:02:10 ns dovecot: auth-worker(551): Debug: pam(aewhale,66.207.133.228): #1/1 style=1 msg=Password: May 27 13:02:10 ns dovecot: auth: Debug: client passdb out: OK#0111#011user=aewhale May 27 13:02:10 ns dovecot: auth: Debug: master in: REQUEST#0112671116289#011453#0111#01174d85bf0d878139a83e682a869786da9 May 27 13:02:10 ns dovecot: auth-worker(551): Debug: passwd(aewhale,66.207.133.228): lookup May 27 13:02:10 ns dovecot: auth: Debug: master userdb out: USER#0112671116289#011aewhale#011system_groups_user=aewhale#011uid=501#011gid=501#011home=/home/aewhale May 27 13:02:10 ns dovecot: pop3-login: Login: user=<aewhale>, method=PLAIN, rip=66.207.133.228, lip=66.207.133.227 May 27 13:02:10 ns dovecot: pop3(aewhale): Debug: Effective uid=501, gid=501, home=/home/aewhale May 27 13:02:10 ns dovecot: pop3(aewhale): Debug: maildir++: root=/home/aewhale/Maildir, index=, control=, inbox=/home/aewhale/Maildir, alt= All processing stops here. This is an NFS Storage Mail Spool, which works on 1.1.20 but not 2.1.15? What other pink elephants have I missed? Thank you for your assistance. -- Albert E. Whale, CEH CHS CISA CISSP *President - Chief Security Officer* http://www.IT-Security-inc.com - IT Security, Inc. Phone: 412-515-3010 | Email: Albert.Whale(a)IT-Security-inc.com Cell: 412-889-6870

1 1

[Dovecot] Corrupted Mail?
by Arthur Dent 28 May '14

28 May '14

Hello All, I have a Dovecot v2.2.12 installation running on a Fedora 20 box which is the mailserver for my family. This means that it is serving only 4 accounts. I use fetchmail->procmail to feed Dovecot and I read my mail on clients such as Evolution on (another) Fedora 20 machine, iPad/iPhone or squirrelmail on the web (running on the same Fedora 20 server). The average mail throughput for the whole family is around a couple of hundred messages per day, but each (and every) day I get between 3 and 10 of the following messages: dovecot: imap(mark): Error: Next message unexpectedly corrupted in mbox file /home/mark/mail/MLists/Fail2Ban at 3407473: 1 Time(s) I have googled, but with little success. What I have found seemed to suggest that removing the .index files and allowing Dovecot to rebuild them will solve this. I did: find /home -type f -name "dovecot.index.*" -exec rm -f {} \; but it has made no difference. Can anyone help me solve this please? Thanks in advance Mark

5 12

[Dovecot] both perosnal and global sieve scripts
by Selcuk Yazar 28 May '14

28 May '14

Hi we have redhat 6 EL and dovecot 2.0.9 personel scripts running with no problem, but now i try to add global scripts for sieve i added parameters sieve_global_path = /var/lib/dovecot/sieve/default.sieve sieve_global_dir = /var/lib/dovecot/sieve/ and create folder with owned vmail and creat script default.sieve require ["fileinto"]; # rule:[Move Spam to Junk Folder] if header :is "X-Spam-Flag" "YES" { fileinto "Spam"; stop; } if header :contains ["Subject"] "**global**" { fileinto "Spam" ; stop ; } keep; after that when try to compile this script it says mbox: INBOX defaulted to /home/vmail/domains//root/inbox fs: root=/home/vmail/domains//root, index=, control=, inbox=/home/vmail/domains//root/inbox i think sievec doesnt recognazi mail_location or anothers things, when i try to sent e-mail, it didn't deliver also i saw root folder after domains folder. any idea our conf is dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-131.6.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.0 (Santiago) ext4 auth_debug = yes auth_debug_passwords = yes auth_default_realm = ........................edu.tr auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot.debug disable_plaintext_auth = no first_valid_uid = 97 info_log_path = /var/log/dovecot.info last_valid_uid = 5000 mail_debug = yes mail_gid = 1001 mail_location = mbox:/home/vmail/domains/%d/%u mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { args = /etc/dovecot/conf.d/dovecot-ldap.conf.ext driver = ldap } plugin { quota = maildir:User quota quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%u.sieve sieve_dir = /home/vmail/domains/sieve/%n/.sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } ssl = no ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/conf.d/dovecot-ldap.conf.ext driver = ldap } protocol lda { debug_log_path = /var/log/dovecot-lda-errors.log info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log mail_plugins = " sieve quota" } protocol imap { mail_plugins = " quota imap_quota" } thanks in advance -- Selçuk YAZAR

1 0

[Dovecot] doveadm fts: mbox/lucene-indexes, and other errors
by Larry Rosenman 27 May '14

27 May '14

I turned on fts_autoindex today, and wanted to rescan all my mailboxes.... Ran doveadm -Dv fts rescan, and garnered: thebighonker.lerctr.org /home/ler/mail $ doveadm -Dv fts rescan doveadm(ler): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(ler): Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so doveadm(ler): Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_lucene_plugin.so doveadm(ler): Debug: Module loaded: /usr/local/lib/dovecot/lib90_stats_plugin.so doveadm(ler): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(ler): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: Undefined symbol "acl_user_module" (this is usually intentional, so just ignore this message) doveadm(ler): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: Undefined symbol "expire_set_lookup" (this is usually intentional, so just ignore this message) doveadm(ler): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: Undefined symbol "quota_user_module" (this is usually intentional, so just ignore this message) doveadm(ler): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so doveadm(ler): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so doveadm(ler): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so doveadm(ler): Debug: Effective uid=1002, gid=1002, home=/home/ler doveadm(ler): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mail/INBOX doveadm(ler): Debug: fs: root=/home/ler/mail, index=, indexpvt=, control=, inbox=/home/ler/mail/INBOX, alt= doveadm(ler): Debug: Namespace archive: type=private, prefix=#ARCHIVE/, sep=/, inbox=no, hidden=no, list=no, subscriptions=yes location=mbox:~/MAILARCHIVE doveadm(ler): Debug: mbox: INBOX defaulted to /home/ler/MAILARCHIVE/inbox doveadm(ler): Debug: fs: root=/home/ler/MAILARCHIVE, index=, indexpvt=, control=, inbox=/home/ler/MAILARCHIVE/inbox, alt= doveadm(ler): Error: Couldn't get mailbox lucene-indexes/dovecot-expunges.log GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_a1.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9t.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9z.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9t_1.del GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/segments.gen GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9u.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_a0.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9x.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9v.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/segments_ke GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9w.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_9y.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_5w.cfs GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: Couldn't get mailbox lucene-indexes/_5w_1.del GUID: Mailbox isn't a valid mbox file doveadm(ler): Error: lucene: Couldn't open mailbox (error in mailbox with GUID=21ba3234186a115376580000bbe98eac): Internal error occurred. Refer to server log for more information. [2014-05-27 14:59:45] doveadm(ler): Error: lucene: Couldn't open mailbox (error in mailbox with GUID=77614b1bb0383a530fb30000bbe98eac): Internal error occurred. Refer to server log for more information. [2014-05-27 14:59:45] doveadm(ler): Error: lucene: Couldn't open mailbox (error in mailbox with GUID=c77d8c12ac6b115348590000bbe98eac): Internal error occurred. Refer to server log for more information. [2014-05-27 14:59:45] doveadm(ler): Error: lucene: Couldn't open mailbox (error in mailbox with GUID=d07d8c12ac6b115348590000bbe98eac): Internal error occurred. Refer to server log for more information. [2014-05-27 14:59:45] doveadm(ler): Error: fts rescan failed thebighonker.lerctr.org /home/ler/mail $ 1) why doesn't Dovecot make the lucene-indexes directory \NoSelect or invisible? 2) how can I find the error(s) for the internal errors? (I looked in my debug.maillog and find NOTHING). Dovecot 2.2.13 FreeBSD 10.0 dovecot -n: thebighonker.lerctr.org /home/ler/mail $ dovecot -n # 2.2.13: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.0-STABLE amd64 auth_default_realm = lerctr.org auth_mechanisms = plain login auth_realms = lerctr.org thebighonker.lerctr.org tbh.lerctr.org auth_username_format = %Ln disable_plaintext_auth = no lda_mailbox_autocreate = yes lmtp_save_to_detail_mailbox = yes login_access_sockets = tcpwrap mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mail/INBOX mail_plugins = fts fts_lucene stats mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace archive { hidden = no inbox = no list = no location = mbox:~/MAILARCHIVE prefix = "#ARCHIVE/" separator = / } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX { auto = create } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. normalize no_snowball sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 5s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } service auth { unix_listener auth-client { mode = 0666 } unix_listener auth-master { mode = 0666 } } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0666 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } verbose_proctitle = yes protocol imap { imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags mail_max_userip_connections = 50 mail_plugins = fts fts_lucene stats imap_stats } thebighonker.lerctr.org /home/ler/mail $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 108 Turvey Cove, Hutto, TX 78634-5688

1 0

[Dovecot] dsync changing source permission to "root" in backup mode
by Peter Mogensen 27 May '14

27 May '14

Hi, We have dsync failing once in a while when running in "backup" mode. What's strange is that the result is that the file permissions on the *source* machine ends up with the wrong permissions (set to uid 0). Even though the dsync manual clearly says: "Backup mails from default mail location to location2 (or vice versa, if -R parameter is given). No changes are ever done to the source location. Any changes done in destination are discarded." Running: 'dsync -R -o mail_home=/users/maildir backup ssh -c arcfour src-host "dsync -o mail_home=/users/maildir"' I know it's running as root, but even then ... it shouldn't modify the source in "backup" mode ?? The error message from dsync when failing is: dsync-remote(root): Error: Cached message size larger than expected (5292 > 5289) dsync-remote(root): Error: Maildir filename has wrong S value, renamed the file from /users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5292,W=5411:2,S to /users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5289:2,S dsync-remote(root): Error: Corrupted index cache file /users/maildir/.Sent/dovecot.index.cache: Broken physical size for mail UID 1040 dsync-remote(root): Error: dsync(dst-host): read(/users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5292,W=5411:2,S) failed: Cached message size larger than expected (5292 > 5289) /Peter

1 1

[Dovecot] Segfault when deselecting virtual folder 2.2.13+ HG TIP
by Colin Knox 27 May '14

27 May '14

Hello, I'm seeing a segfault in the imap process with the current mercurial tip (including changeset 17382 e99cd21e1f92) when selecting a particular virtual mail folder (but no other virtual mailboxes) and then selecting a different mail folder as in the IMAP transcript below. The same segfaults are triggered with IMAP clients. The situation is worse with the 2.2.13 which segfaults when any virtual mailbox is deselected so the recent patch helps partially. Configuration summary, IMAP session in gbd including backtrace and virtual mailbox spec follow. Any ideas what's causing this? Thank you Colin ======= GDB SESSION INCLUDING BACKTRACE ======= colin@xxxx [ ~ ]$ MAIL=maildir:/var/mail/colin gdb /usr/local/libexec/dovecot/imap GNU gdb (GDB) 7.6.1 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/local/libexec/dovecot/imap...done. (gdb) r Starting program: /usr/local/libexec/dovecot/imap warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? process 21455 is executing new program: /usr/local/bin/doveconf warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? process 21455 is executing new program: /usr/local/libexec/dovecot/imap warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY NOTIFY METADATA] Logged in as colin a SELECT PERSONAL.Home * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 23 EXISTS * 0 RECENT * OK [UIDVALIDITY 1261140830] UIDs valid * OK [UIDNEXT 24] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest a OK [READ-WRITE] Select completed (0.001 secs). a SELECT virtual.Conversations * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 5 EXISTS * 0 RECENT * OK [UNSEEN 3] First unseen. * OK [UIDVALIDITY 1400091139] UIDs valid * OK [UIDNEXT 18] Predicted next UID a OK [READ-WRITE] Select completed (0.017 secs). a SELECT PERSONAL.Home Program received signal SIGSEGV, Segmentation fault. 0x00007ffff75f1c6c in mail_search_args_deinit_sub.30037.5970 () from /usr/local/lib/dovecot/libdovecot-storage.so.0 (gdb) bt full #0 0x00007ffff75f1c6c in mail_search_args_deinit_sub.30037.5970 () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #1 0x00007ffff75f1d41 in mail_search_args_deinit () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x00007ffff6ca3adb in virtual_mailbox_close_internal.9208 () from /usr/local/lib/dovecot/lib20_virtual_plugin.so No symbol table info available. #3 0x00007ffff6ca3b39 in virtual_mailbox_close.9256 () from /usr/local/lib/dovecot/lib20_virtual_plugin.so No symbol table info available. #4 0x00007ffff761daac in mailbox_close () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #5 0x00007ffff75ff283 in mailbox_free () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #6 0x0000000000410a2e in close_selected_mailbox.part.1 (client=client@entry=0x6577c0) at cmd-select.c:383 box = 0x0 #7 0x0000000000410b59 in close_selected_mailbox (client=0x6577c0) at cmd-select.c:376 client = 0x6577c0 #8 cmd_select_full (cmd=0x6583e0, readonly=<optimized out>) at cmd-select.c:427 client = 0x6577c0 ctx = 0x6584e0 args = 0x65ccd8 list_args = 0x632038 mailbox = 0x6322a8 "PERSONAL.Home" error = 0x648748 "" __FUNCTION__ = "cmd_select_full" __FUNCTION__ = "cmd_select_full" #9 0x00000000004129ac in command_exec (cmd=0x6583e0) at imap-commands.c:158 hook = 0x63b5c0 ret = <optimized out> #10 0x00000000004178ef in client_command_input (cmd=0x6583e0) at imap-client.c:778 client = 0x6577c0 command = <optimized out> __FUNCTION__ = "client_command_input" __FUNCTION__ = "client_command_input" #11 0x00000000004179a5 in client_command_input (cmd=0x6583e0) at imap-client.c:839 client = 0x6577c0 command = <optimized out> __FUNCTION__ = "client_command_input" __FUNCTION__ = "client_command_input" #12 0x0000000000417c65 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x6577c0) at imap-client.c:877 No locals. #13 client_handle_input (client=client@entry=0x6577c0) at imap-client.c:889 _data_stack_cur_id = 3 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" __FUNCTION__ = "client_handle_input" #14 0x0000000000417e8f in client_input (client=0x6577c0) at imap-client.c:931 cmd = 0x7ffff732abce <io_loop_context_activate+62> output = 0x658200 bytes = 23 __FUNCTION__ = "client_input" __FUNCTION__ = "client_input" #15 0x00007ffff732d3ff in io_loop_call_io () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #16 0x00007ffff731f407 in io_loop_handler_run_internal () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #17 0x00007ffff731f4d9 in io_loop_handler_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #18 0x00007ffff731f558 in io_loop_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x00007ffff731f5c3 in master_service_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x000000000040bfa0 in main (argc=1, argv=0x63a390) at main.c:400 set_roots = {0x4274e0 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0x0, postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x0, failure_callback = 0x0, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out> set_roots = {0x4274e0 <imap_setting_parser_info>, 0x0} ===================== DOVECONF -n ===================== colin@xxxx [ /var/mail/colin/virtual ]$ doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.14.4 x86_64 Linux From Scratch ext4 auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %Ln first_valid_gid = 1000 first_valid_uid = 1000 hostname = mail.theknoxes.co.uk imap_metadata = yes last_valid_gid = 1100 last_valid_uid = 1100 listen = * login_trusted_networks = 127.0.0.1 mail_attribute_dict = file:/var/mail/%n/shared_metadata mail_location = maildir:/var/mail/%n mail_plugins = " fts fts_solr virtual" mail_temp_dir = /tmp/mail mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace { location = virtual:/var/mail/%n/virtual prefix = virtual. separator = . subscriptions = no } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox "Junk E-mail" { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } mailbox virtual/All { special_use = \All } prefix = separator = . } passdb { driver = pam } plugin { fts = solr fts_autoindex = yes fts_decoder = decode2text fts_solr = url=http://localhost:8080/solr/dovecot/ break-imap-search sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap lmtp sieve sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener auth-client { group = exim mode = 0660 } } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 } user = dovecot } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 0 } } service lmtp { unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } ssl = required ssl_cert = </etc/ssl/certs/dovecot.pem ssl_cipher_list = EECDH:EDH:@STRENGTH:HIGH:!MEDIUM:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4:!ADH:!aNULL:!eNULL ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/dovecot.key ssl_prefer_server_ciphers = yes submission_host = localhost:25 userdb { driver = passwd } protocol imap { mail_max_userip_connections = 100 } protocol lmtp { mail_plugins = " fts fts_solr virtual sieve" } =========================== VIRTUAL MAILBOX SPEC =========================== colin@xxxx [ /var/mail/colin/virtual ]$ cat Conversations/dovecot-virtual virtual.All inthread refs x-mailbox Inbox ===========================

2 2

[Dovecot] Odd ownership of the dovecot-uidlist file
by dclist.hook＠hook.net.nz 27 May '14

27 May '14

Hi, We have just recently switched from Courier to Dovecot for both IMAAP and POP access for our shared hosting platform, and while the issues we have been seeing with Courier have gone away a new one has popped up. Unfortunately googling for an answer is difficult as it falls into the ownership and permissions category which people seem to have all the time. But in those cases it always happens, whereas what we are seeing is a random ownership change on the dovecot-uidlist file. Basically what we see is the user is able to login fine for days and then all of a sudden the file has changed ownership to a user under a different domain, what's worse is the other user has not even logged in during that time. Here's a snippet from a case from a month ago which was escalated to us, there are plenty of other examples but its so random we only know its happened when the customer complains or we happen to watch the logs, the current help desk procedure when the customer calls is to just fix the ownership of the file. -- 8<-- Mar 25 11:18:05 brio dovecot: pop3-login: Login: user=<accounts@DOMAIN1>, method=PLAIN, rip=IP, lip=IP, mpid=21739, TLS, session=<mwQLnGH1ZAB2XalI> Mar 25 11:18:08 brio dovecot: pop3(accounts@DOMAIN1): Disconnected: Logged out top=0 (0 b), retr=4 (274142 b), messages=27 (10797760 b), del=0 Mar 25 11:19:24 brio dovecot: pop3(accounts@DOMAIN1): Error: open(/mnt/spool/keepers/t/DOMAIN1/accounts/Maildir/dovecot-uidlist) failed: Permission denied Mar 25 11:19:24 brio dovecot: pop3(accounts@DOAMIN1): Error: open(/mnt/spool/keepers/t/DOMAIN1/accounts/Maildir/dovecot-uidlist) failed: Permission denied -- 8<-- In this case the ownership of the file was: -rw------- 1 jacqui-30747 DOMAIN2 48839 Mar 25 10:31 dovecot-uidlist when it should be: -rw------- 1 accounts-25105 DOMAIN1 48839 Mar 25 10:32 dovecot-uidlist As you can see both the user and the group have changed to a different user as if that user had recreated the file. Its worth noting that the parent directory does not change: drwx------ 8 accounts-25105 DOMAIN1 4096 Mar 25 10:31 . so this user should not even be able to access this file. It would be great if someone can give us some hints where the problem maybe as this has us stumped. Some more detail if it helps: root@brio:~# dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.2 auth_mechanisms = plain login cram-md5 apop auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@& default_process_limit = 3000 disable_plaintext_auth = no lock_method = dotlock mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/home/indexes/%1d/%n@%d mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " quota" maildir_stat_dirs = yes mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/dovecot-sql-cram.conf.ext driver = sql } plugin { quota = maildir:User quota } protocols = " imap pop3" service imap-login { inet_listener imap25 { port = 25143 } } service imap-postlogin { executable = script-login -d /usr/local/bin/postlogin.sh } service imap { executable = imap imap-postlogin } service pop3-login { inet_listener pop325 { port = 25110 } } service pop3 { executable = pop3 imap-postlogin } ssl_cert = </var/www/mailadmin/ssl/cert.pem ssl_crypto_device = dynamic ssl_key = </var/www/mailadmin/ssl/mailx.hosts.net.nz.key userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } userdb { args = /etc/dovecot/dovecot-sql-cram.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_capability = +XHOST_brio imap_logout_format = bytes=%i/%o mail_max_userip_connections = 50 mail_plugins = " quota imap_quota" } protocol pop3 { mail_max_userip_connections = 10 pop3_lock_session = no pop3_logout_format = top=%t (%p b), retr=%r (%b b), messages=%m (%s b), del=%d pop3_save_uidl = yes pop3_uidl_format = %f } root@brio:~# dpkg -l|grep dovecot ii dovecot-core 1:2.1.7-7 amd64 secure mail server that supports mbox, maildir, dbox and mdbox mailboxes ii dovecot-imapd 1:2.1.7-7 amd64 secure IMAP server that supports mbox, maildir, dbox and mdbox mailboxes ii dovecot-mysql 1:2.1.7-7 amd64 MySQL support for Dovecot ii dovecot-pop3d 1:2.1.7-7 amd64 secure POP3 server that supports mbox, maildir, dbox and mdbox mailboxes Mailbox storage is done over various NFS stores, however each user has a specific NFS store Connections are load balanced using LVS and keepalived over approx 10 real servers, persistant connections are setup so that a single user should only hit a single dovecot server (this is different from our courier setup due to the known NFS limitations with dovecot) We also have IMAP indexes stored on the dovecot servers ext3 file system itself rather than NFS. Each server is setup and maintained by puppet, and any one of the servers may see the issue. All servers are x86 with at least 2Gb of RAM User information comes from a MySQL database, and testing the SQL statements only ever return the correct record. Exim is writing to the Maildir with the correct permissions, and after fixing the ownership of the file dovecot can read and change the mailboxes. We do have a postlogin.sh script which runs after authentication to convert the old courier files over to the dovecot ones, this only happens however when the process has not run before (it appears dovecot deletes the files and recreates them, so we use a state file). Adding debugging to the script shows its running as the correct user at the last successful login. It does not run at all after the ownership has changed. There are some users who this is happening regularly, and other users who it has happened to only once. It is also not just limited to POP3, although this is where we commonly see it as dovecot accepts the IMAP connection if the ownership on a sub folders uidlist is different. If you need any further detail, I can provide it. Debugging could be much more difficult because of how random it is, while it can happen daily there have been periods of up to one week before its seen. Any help in finding this would be greatly appreciated, Cheers, Bruce

2 1