April 2014 - dovecot - dovecot.org

[Dovecot] Director NFS backups
by Murray Trainer 10 Apr '14

10 Apr '14

Hi All, What is the recommended method of taking backups of user mailboxes when using the Director setup with NFS backend mail storage? Is there any way to avoid the likely NFS locking issues by backing up the filesystem directly or do I have to do some sort of dsync method. If I have to use dsync can I dsync the mailboxes to compressed copies of the mailboxes without requiring another dovecot server to do the dsync. Thanks for you responses. Murray

1 0

Re: [Dovecot] POP3 sessions
by Joseph Tam 10 Apr '14

10 Apr '14

<doctor(a)doctor.nl2k.ab.ca> writes: > Is their a way in the config file to not lock > POP3 sessions in dovecot for only 1 user? That ought to be the default: http://wiki2.dovecot.org/POP3Server (Section Session locking) Did you set pop3_lock_session=yes somewhere? Locking is good if the client connection is stable: if you disable locking, you could end up with a scrambled mailbox as two or more users access it at the same time. Joseph Tam <jtam.home(a)gmail.com>

1 0

[Dovecot] Error: broken sync positions in index file
by Alejandro Angelico 10 Apr '14

10 Apr '14

Hi everybody, I'm runing dovecot 2.2.12 and a user mailbox got the "broken sync positions in index file" error twice in the last 20 days. this is an extract from the maillog maillog First Error -------------------- Mar 23 16:13:25 andromeda dovecot: pop3-login: Login: user=<myuser(a)mydomain.com>, method=PLAIN, rip=187.237.107.24, lip=207.7.90.83, mpid=3983, session=<5KSEbU310wC77WsY> Mar 23 16:13:27 andromeda dovecot: pop3(myuser(a)mydomain.com): Error: broken sync positions in index file /home/myuser/mail/mydomain.com/myuser/dovecot.index Mar 23 16:13:27 andromeda dovecot: pop3(myuser(a)mydomain.com): Warning: fscking index file /home/myuser/mail/mydomain.com/myuser/dovecot.index Mar 23 16:13:27 andromeda dovecot: pop3(myuser(a)mydomain.com): Error: Fixed index file /home/myuser/mail/mydomain.com/myuser/dovecot.index: log_file_tail_offset 164 -> 144 Mar 23 16:13:27 andromeda dovecot: pop3(myuser(a)mydomain.com): Panic: file mail-transaction-log.c: line 350 (mail_transaction_log_set_mailbox_sync_pos): assertion failed: (file_offset >= log->head->saved_tail_offset) Mar 23 16:13:27 andromeda dovecot: pop3(myuser(a)mydomain.com): Fatal: master: service(pop3): child 3983 killed with signal 6 (core dumps disabled) Mar 23 16:13:27 andromeda dovecot: pop3: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0xb7e59350] -> /usr/lib/dovecot/libdovecot.so.0 [0xb7e593cd] -> /usr/lib/dovecot/libdovecot.so.0 [0xb7e58c64] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_set_mailbox_sync_pos+0x9a) [0xb7f8c1aa] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x9c) [0xb7f82a5c] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7f00f2b] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7f0034a] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7f006ba] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xc1) [0xb7f00851] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x39) [0xb7f33b79] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x3d) [0xb7f365ad] -> dovecot/pop3(client_create+0x2e2) [0x804c5e2] -> dovecot/pop3 [0x804afec] -> dovecot/pop3 [0x804b607] -> /usr/lib/dovecot/libdovecot.so.0 [0xb7e1033a] -> /usr/lib/dovecot/libdovecot.so.0 [0xb7e10622] -> /usr/li maillog Second Error -------------------- Apr 9 17:22:00 andromeda dovecot: imap-login: Login: user=<myuser(a)mydomain.com>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=29811, secured, session=<ZteFh6P2uAB/AAAB> Apr 9 17:22:00 andromeda dovecot: imap(myuser(a)mydomain.com): Error: Transaction log file /home/myuser/mail/mydomain.com/myuser/dovecot.index.log seq 4: log_file_tail_offset update shrank it (284 vs 420 sync_offset=268) Apr 9 17:22:00 andromeda dovecot: imap(myuser(a)mydomain.com): Error: broken sync positions in index file /home/myuser/mail/mydomain.com/myuser/dovecot.index Apr 9 17:22:00 andromeda dovecot: imap(myuser(a)mydomain.com): Warning: fscking index file /home/myuser/mail/mydomain.com/myuser/dovecot.index Apr 9 17:22:00 andromeda dovecot: imap(myuser(a)mydomain.com): Error: Fixed index file /home/myuser/mail/mydomain.com/myuser/dovecot.index: log_file_tail_offset 420 -> 284 Apr 9 17:22:00 andromeda dovecot: imap(myuser(a)mydomain.com): Panic: file mail-transaction-log.c: line 350 (mail_transaction_log_set_mailbox_sync_pos): assertion failed: (file_offset >= log->head->saved_tail_offset) Apr 9 17:22:00 andromeda dovecot: imap(myuser(a)mydomain.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0xb7d9ae4f] -> /usr/lib/dovecot/libdovecot.so.0 [0xb7d9aecd] -> /usr/lib/dovecot/libdovecot.so.0 [0xb7d9a744] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_set_mailbox_sync_pos+0x9a) [0xb7ecf60a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x9c) [0xb7ec5e5c] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7e43f8b] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7e433aa] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7e4371a] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xc1) [0xb7e438b1] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x39) [0xb7e76d19] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x3d) [0xb7e7974d] -> dovecot/imap(cmd_select_full+0x23e) [0x805721e] -> dovecot/imap(cmd_select+0x19) [0x8057bf9] -> dovecot/imap(command_exec+0x33) [0x805c9a3] -> dovecot/imap [0x805b6df] -> dove Apr 9 17:22:00 andromeda dovecot: imap(myuser(a)mydomain.com): Fatal: master: service(imap): child 29811 killed with signal 6 (core dumps disabled) Deleting dovecot.index.* files solved the problem both times, but I'm wondering if something is wrong because it's always the same mailbox. Before deleting dovecot.index I run doveadm with this oputput: root@myserver [/home/myuser/mail/.myuser@mydomain_com]# doveadm index -u myuser INBOX doveadm(myuser): Fatal: User doesn't exist root@myserver [/home/myuser/mail/.myuser@mydomain_com]# doveadm index -u myuser(a)mydomain.com INBOX doveadm(myuser(a)mydomain.com): Error: Transaction log file /home/myuser/mail/mydomain.com/myuser/dovecot.index.log seq 4: log_file_tail_offset update shrank it (284 vs 420 sync_offset=268) doveadm(myuser(a)mydomain.com): Error: broken sync positions in index file /home/myuser/mail/mydomain.com/myuser/dovecot.index doveadm(myuser(a)mydomain.com): Warning: fscking index file /home/myuser/mail/mydomain.com/myuser/dovecot.index doveadm(myuser(a)mydomain.com): Error: Fixed index file /home/myuser/mail/mydomain.com/myuser/dovecot.index: log_file_tail_offset 420 -> 284 doveadm(myuser(a)mydomain.com): Panic: file mail-transaction-log.c: line 350 (mail_transaction_log_set_mailbox_sync_pos): assertion failed: (file_offset >= log->head->saved_tail_offset) doveadm(myuser(a)mydomain.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0xb7d35e4f] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x43) [0xb7d35f63] -> /usr/lib/dovecot/libdovecot.so.0 [0xb7d35744] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_set_mailbox_sync_pos+0x9a) [0xb7e6a60a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x9c) [0xb7e60e5c] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7ddef8b] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7dde3aa] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0xb7dde71a] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xc1) [0xb7dde8b1] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x39) [0xb7e11d19] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x3d) [0xb7e1474d] -> doveadm [0x80595db] -> doveadm [0x80558de] -> doveadm(doveadm_mail_try_run+0x293) [0x8055d03] -> doveadm(main+0x3bd) [0x805eb5d] -> /lib/libc.so.6(__libc_start_main+0xdc) [0xb7b89ebc] -> doveadm [0x8055061] Aborted Is there anything else I should try? or just wait to see if this happens again? the mailbox is not very big, around 38Mb in 84 mails. Thanks! Alex

1 0

[Dovecot] Override maildir for just a few users
by Charles Marcus 09 Apr '14

09 Apr '14

Hi all, Currently, I have mail_location = maildir:~/Maildir set and all passdb and userdb lookups are SQL based (doveconf -n pasted below). However, I want to override the users maildir location - and *only* the maildir location - for just a few users. I was thinking the easiest way would be to add a simple flat file based userdb that is checked before the SQL based one, where I can define a different maildir for these users. So, basically, dovecot checks the flat file first, and if the user isn't found, falls back to SQL. I still want all AUTH to go through SQL... It seems like this should be fairly simple, but after reading the wiki for a few days, I'm just not sure... This is on a production box, so I have to be careful (note to self - MUST set up testing box for things like this)... Is anyone else doing anything like this, and if so would you be willing to share sanitized configs? Thx, -- Best regards, Charles

3 2

[Dovecot] Sieve (randomly?) flagging messages as \\Seen \\Deleted
by Emanuele Balla (aka Skull) 09 Apr '14

09 Apr '14

Hi, I'm trying to debug (without great results, so far) a very weird behaviour in my environment. Sorry if this came out in another form before, but I've not been able to find out anything resembling what I'm experiencing, so far... OS is a debian wheezy and dovecot is the version provided by the OS: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.4 The system uses postfix with virtual users, credentials and maildir locations stored on mysql, local delivery through LMTP and sieve+managesieve available to users (although managesieve only listens on localhost and is only usable through the webmail interface). FWIW, I've been running 10-20 systems just like this one (mostly with earlier dovecot versions and using LDA when LMTP wasn't available yet) for many years, and this is the very first time I'm so puzzled I end up searching for help here :-| Even though the system has been working for months (more or less 1 year now) without big changes, a strange behaviour appeared (or became evident enough to be noticed) in the last few days, apparently involving sieve and some sort of "default/hidden" action triggered by something. Or at least that's what I ended up supposing. The problem in short: Messages I expected to find in certain folders were simply not there, although sieve logs reported them as delivered in those same folders. By manually inspecting the maildir, I actually found the messages, but they were all flagged as \\Seen + \\Deleted and therefore not displayed by the MUA (ThunderBird in my case). I initially supposed this to be a TB issue, 'cause nothing in my sieve script ever flags messages that way, but I could not find anything on my client explaining that. Then I received a notification from one of my users saying he was experiencing exactly the same, except he's using mutt as IMAP client; that restricted the issue to server-side interactions only, but nothing useful could be found in logfiles... This is the "relevant" portion of a "dovecot -n" output: [...] plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +vacation-seconds +editheader sieve_vacation_default_period = 2d sieve_vacation_dont_check_recipient = yes sieve_vacation_max_period = 30d sieve_vacation_min_period = 2s sieve_vacation_use_original_recipient = no } [...] protocol lmtp { mail_plugins = sieve hostname = alternativename.bofhland.org } There is no sieve_(before|after) script in addition to what users create for themselves. The mailboxes are relatively low-traffic; most of the traffic is caused by a single role address translated into multiple-recipients alias going to many of the users (100-300 messages per day). Almost all the users using sieve rely on it to store traffic for that role-address in a dedicated folder, including the mutt user and myself. Not that's all that important, but means that most of the misbehaviours observed (but not all) are observed in these folders. I enabled mail_debug, but there's nothing unusual in there AFAICT: it just says: - using the following location for user's Sieve script: /blabla/.dovecot.sieve - opening script /blabla/.dovecot.sieve - script binary /blabla/.dovecot.svbin successfully loaded - not saving binary /blabla/.dovecot.svbin, because it is already stored - executing script from /blabla/.dovecot.svbin - stored mail into mailbox 'DedicatedFolder' And it does, except the message happens to be flagged as seen and deleted... In order to try restricting what may be causing the issue, I added... removeflag "\\Deleted"; removeflag "\\Seen"; ...to the rule sending messages to the dedicated folder cited above for my mailbox. The problem apparently disappeared for me as long as I kept those in place. This should mean (I suppose) that the decision of flagging messages that way is taken before the users' sieve scripts are applied (sort of "hidden sieve_before"). The problem is hard to replicate as it's apparently not deterministic: the same message is being flagged for one user but not for the other, different messages between the same people and with the same thread are sometimes flagged and sometimes not. Something is surely triggering whatever is causing this, but it's not evident enough for me to point a finger and say "ha ha!" (https://www.youtube.com/watch?v=rX7wtNOkuHo) So, a question for people knowing pigeonhole's internals: does it ever choose to mark messages as described above for any reason? If yes, I may be triggering this behavior with something I recently added/modified, and at least I'll have something to focus my investigation on. :-| Any other suggestion is obviously welcome...

3 6

[Dovecot] sieve pigeonhole vacation filter responding when recipient is not explicitly listed
by Milan Cvetkovic 09 Apr '14

09 Apr '14

Hi dovecot.org, When I set up vacation filter for a mailbox, the vacation response is generated even though none of the addresses in the email are same as the mailbox address itself. I have example email sent to "aliastouser(a)mpathix.com" which gets delivered to "user(a)my.domain.com" by dovecot-lda. The invocation command is: # /usr/lib/dovecot/dovecot-lda \ -d user \ -f john.doe(a)somewhere.com \ -a aliastouser(a)mpathix.com \ -p /tmp/test.email # cat /tmp/test.email Message-ID: <test3> From: John Doe <john(a)somewhere.com> To: aliastouser(a)mpathix.com Subject: test3 vacation response as aliastouser This is the only line in test.email. Here is the vacation filter: if true { vacation :days 1 :addresses ["user(a)my.domain.com"] text: Please ignore this message, I am only testing vacation filter . ; } /var/log/mail.log shows that sieve vacation sends vacation, (and after succeeding attempts, discards due to already sent address) Apr 9 13:37:21 mx01 dovecot: lda(user): sieve: <test3>: sent vacation response to <john(a)somewhere.com> Apr 9 13:37:21 mx01 dovecot: lda(user): sieve: msgid=<test3>: stored mail into mailbox 'INBOX' ... Apr 9 13:40:15 mx01 dovecot: lda(user): sieve: msgid=<test3>: discarded duplicate vacation response to <john(a)somewhere.com> Apr 9 13:40:15 mx01 dovecot: lda(user): sieve: msgid=<test3>: stored mail into mailbox 'INBOX' Attached is the doveconf -n output. Any help is appreciated. Milan.

2 2

[Dovecot] Prevent sieve redirects to external domains
by adarshvs 09 Apr '14

09 Apr '14

Hi, Is there any way to disable sieve forwards to all domains except one. Users should be able to forward their mails to only one specific domain. If not then any idea on how this feature can be implemented ??. -- View this message in context: http://dovecot.2317879.n4.nabble.com/Prevent-sieve-redirects-to-external-do… Sent from the Dovecot mailing list archive at Nabble.com.

2 1

Re: [Dovecot] Heartbleed openssl vulnerability?
by Jake Alley 09 Apr '14

09 Apr '14

It's an interesting issue. In my experience "stale" passwords are rarely used to compromise systems. However, passwords tend to end up on sticky notes and even worse, in email databases regardless. As far as compromised email passwords, they seem to mostly come from infected clients and insecure public logins as far as I can tell. A server can control the later, but not the former. I know of a major accounting software that forces Admin users to change their passwords every few months under certain circumstances. Those passwords always end up in emails to fellow users, so in that case forcing people to change seems to be definitely counterproductive. IMV the moral of the story is that you can't crypt your way into a 100% secure world. You need other forms of checks & reconciliations that are disjoint from purely cryptographic infrastructure. For instance ask Mt. Gox and Bitcoin if they agree in hindsight, and Heartbleed is a very good example of this concept. Thanks, Jake On 4/9/2014 10:27 AM, Reindl Harald wrote: "change passwords from time to time is always clever" is a strawmans argument with no context to the issue, forcing people to change their passwords all the time for no good reasons leads mostly to completly insecured passwords to remember them easier or have them on a sticky on the screen or under the keyboard the word "counterproductive" describes that policies perfectly

1 0

[Dovecot] Migration to Dovecot 2.2.12 - How to trigger full site indexing
by kadafax＠gmail.com 09 Apr '14

09 Apr '14

Hi list, We are in the process of migrating our old (dovecot 1.2.4 based) mail system to the new one: - Centos 6 x86_64 - Dovecot 2.2.12 - Users in an openldap directory - clucene FTS I'm trying to trigger a full indexing for all the user and all their mailboxes (avoiding them to individually trigger it using a search within their MUA). I'm trying to use the "doveadm index" command. First question, is there some sort of 'wildcard' option for the name of the mailbox to index, as I dont know all the mailboxes's names the users have created (beside the regular Sent Inbox Trash and Drafts ? Second question I've tried: # doveadm -D index -A Sent which gave me: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users Can the '-A' option work with ldap backend ? Thanks a lot and have a nice day, kfx --------------------------------------------- # 2.2.12: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.11.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) ext4 auth_cache_size = 10 M auth_debug = yes auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log default_vsz_limit = 2 G hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_debug = yes mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = fts fts_lucene quota stats mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_autoindex_max_recent = 1000 fts_lucene = whitespace_chars=@ quota = maildir:User quota quota_rule = *:storage=20GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 30 secs stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmaster(a)mydomain.com protocols = imap pop3 sieve sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 5 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert = </etc/pki/tls/certs/mail.mydomain.com.pem ssl_key = </etc/pki/tls/private/mail.mydomain.com.key userdb { args = uid=12347 gid=12347 home=/var/vmail/%Ln/home allow_all_users=yes driver = static } verbose_proctitle = yes protocol lda { mail_plugins = fts fts_lucene quota stats sieve virtual } protocol imap { mail_plugins = fts fts_lucene quota stats imap_quota virtual imap_stats }

2 2

[Dovecot] POP3 sessions
by The Doctor 09 Apr '14

09 Apr '14

Is their a way in the config file to not lock POP3 sessions in dovecot for only 1 user? -- Member - Liberal International This is doctor(a)nl2k.ab.ca Ici doctor(a)nl2k.ab.ca God,Queen and country!Never Satan President Republic!Beware AntiChrist rising! http://www.fullyfollow.me/rootnl2k Look at Psalms 14 and 53 on Atheism C'est temps pour le Quebec de voter aujourd'hui.

2 1