January 2014 - dovecot

[Dovecot] Dsync Panic
by Gilles Chauvin 14 Feb '14

14 Feb '14

Hi, Using dsync to backup my users' mailboxes (dovecot 2.2.10 on both sides), I sometimes encounter the following issue. Below is the output of: dsync -Dvf -u "user" -R backup ssh root(a)server.domain.tld dsync -u "user" doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(user): Debug: Effective uid=495, gid=495, home=/srv/mail/u/user doveadm(user): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=sdbox:/srv/mail/u/user doveadm(user): Debug: fs: root=/srv/mail/u/user, index=, indexpvt=, control=, inbox=, alt= doveadm(user): Debug: Namespace : Using permissions from /srv/mail/u/user: mode=0700 gid=default dsync-local(user): Debug: brain M: in state=master_recv_handshake dsync-local(user): Debug: brain M: out state=send_mailbox_tree changed=1 dsync-local(user): Debug: brain M: in state=send_mailbox_tree dsync-local(user): Debug: brain M: out state=send_mailbox_tree_deletes changed=1 dsync-local(user): Debug: brain M: in state=send_mailbox_tree_deletes dsync-local(user): Debug: brain M: out state=recv_mailbox_tree changed=1 dsync-local(user): Debug: brain M: in state=recv_mailbox_tree dsync-local(user): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(user): Debug: brain M: in state=recv_mailbox_tree dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes changed=1 dsync-local(user): Debug: brain M: in state=recv_mailbox_tree_deletes dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes changed=0 dsync-remote(user): Panic: file dsync-brain-mailbox-tree-sync.c: line 111 (dsync_brain_mailbox_tree_sync_change): assertion failed: (brain->no_backup_overwrite) dsync-remote(user): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x6889a) [0x7f1e89ee889a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f1e89ee89a2] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f1e89ea21cf] -> dsyn(dsync_brain_mailbox_tree_sync_change+0x3c6) [0x42e046] -> dsyn(dsync_brain_recv_mailbox_tree_deletes+0x12a) [0x42d30a] -> dsyn(dsync_brain_run+0x2e6) [0x42ac96] -> dsyn() [0x42b5f0] -> dsyn() [0x43d740] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f1e89ef8336] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f1e89ef93a7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f1e89ef82a8] -> dsyn() [0x427fd1] -> dsyn() [0x411c97] -> dsyn(doveadm_mail_try_run+0x238) [0x4120a8] -> dsyn(main+0x3d1) [0x41aaa1] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f1e89b0ad1d] -> dsyn() [0x411419] dsync-local(user): Debug: brain M: in state=recv_mailbox_tree_deletes dsync-local(user): Error: read(server.domain.tld) failed: EOF dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes changed=0 dsync-local(user): Error: Remote command returned error 255 Regards, Gilles

1 1

[Dovecot] Problem to setup replication
by Daniel Thielemann 14 Feb '14

14 Feb '14

Hi, i tried to setup replication since some days but I'am stuck now. My conf: # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.3.1.el6.x86_64 x86_64 CentOS release 6.5 (Final) dsync_remote_cmd = ssh -l%{login} %{host} /usr/bin/doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace} log_path = /var/log/dovecot mail_gid = 5000 mail_home = /var/vmail/%d/%n mail_location = maildir:~/Maildir mail_plugins = " notify replication" mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = remote:vmail@server1.domain.dom replication_full_sync_interval = 1 hours sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve_deprecated { port = 2000 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol lda { auth_socket_path = /var/run/dovecot/auth-master log_path = /var/log/protocol-lda.log mail_plugins = sieve postmaster_address = postmaster(a)domain.dom } ----------------- sql-conf: driver = mysql connect = host=127.0.0.1 dbname=xxx user=mailuser password=xxx default_pass_scheme = SSHA512 password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; user_query = SELECT email as user FROM virtual_users WHERE email ='%u'; iterate_query = SELECT email as user FROM virtual_users WHERE email='%u'; ------------------ The config on the other node is similar, except the mail_replica settings. The weird thing is, that my logs are completly empty. I see only the dovecot startup messages and nothing else. I don't understand why the replication never starts by itself or why there are no errors or anything else in the log. I already read the dovecot replication howto in the official documentation, but IMO I've setup everything already like in the docs. ----------------- I've already tried to replicate my mailboxes using dsync, but if I run the dsync command I get only this output: bash-4.1$ ssh vmail(a)server1.domain.dom /usr/bin/doveadm dsync-server -u john(a)domain.dom VERSION dsync 3 2 Hhostname sync_ns_prefix sync_box sync_box_guid sync_type debug sync_visible_namespaces exclude_mailboxes send_mail_requests backup_send backup_recv lock_timeout no_mail_sync no_backup_overwrite purge_remote Smailbox_guid last_uidvalidity last_common_uid last_common_modseq last_common_pvt_modseq changes_during_sync Nname existence mailbox_guid uid_validity uid_next last_renamed_or_created subscribed last_subscription_change Dhierarchy_sep mailboxes dirs unsubscribes Bmailbox_guid uid_validity uid_next messages_count first_recent_uid highest_modseq highest_pvt_modseq mailbox_lost cache_fields have_guids have_save_guids have_only_guid128 Atype key value stream deleted last_change modseq Ctype uid guid hdr_hash modseq pvt_modseq save_timestamp add_flags remove_flags final_flags keywords_reset keyword_changes Rguid uid Mguid uid pop3_uidl pop3_order received_date stream cname decision last_used . Hserver1.domain.dom dsync-remote(john(a)domain.dom): Error: dsync(local): I/O has stalled, no activity for 600 seconds dsync-remote(john(a)domain.dom): Error: Timeout during state=slave_recv_handshake Any help would be appreciated, because I've really no other ideas anymore. Thanks in advance PS: Excuse my english, I'am german :D

1 1

[Dovecot] Sieve is not getting the propper RCPT from the LMTP daemon
by klondike 14 Feb '14

14 Feb '14

Hi! Since the setup I use is a bit strange let's go for it first: * The system uses postfix as MTA and dovecot as LDA after alias translation by local via LMTP * The dovecot version is 2.2.5 and the pigeonhole version is 0.4.1 * The SMTP server at gentoo.org is configured to send me a backup of my emails to klondike+gentoo (at) xiscosoft.es using sendmail * The email is received by the server and then (hopefully filtered). * The email comes with two Delivered-To: headers, the first one has the correct address, the second one has the gentoo address. The interesting part of the filter is this: if envelope :detail "to" "gentoo" { addflag "\\Seen"; fileinto "gentoo"; stop; } But this doesn't seem to work, I enable debugging with the following lines: require "vnd.dovecot.debug"; require "envelope"; require "variables"; if envelope :matches "to" "*" { debug_log "envelope to `${1}'"; } if envelope :matches "from" "*" { debug_log "envelope from `${1}'"; } The relevant lines for the test e-mail I sent are these: sieve: info: started log at Dec 24 13:37:23. main script: line 9: info: DEBUG: envelope to `klondike (at) gentoo.org'. main script: line 10: info: DEBUG: envelope from `klondike (at) gentoo.org'. info: msgid=<52B97FF7.6050109(a)gentoo.org>: stored mail into mailbox 'INBOX'. The thing is that the to envelope should be instead klondike+gentoo (at) xiscosoft.es as show by the lmtp logs: Dec 24 13:37:23 localhost dovecot: lmtp(26249): Connect from local Dec 24 13:37:23 localhost dovecot: lmtp(26249, klondike): Cd6YHwOAuVKJZgAA0J78UA: sieve: msgid=<52B97FF7.6050109(a)gentoo.org>: stored mail into mailbox 'INBOX' Dec 24 13:37:23 localhost dovecot: lmtp(26249): Disconnect from local: Successful quit Dec 24 13:37:23 localhost postfix/lmtp[26248]: 657D9101E5E: to=<klondike+gentoo (at) xiscosoft.es>, relay=xiscosoft.net[private/dovecot-lmtp], delay=0.29, delays=0.05/0.03/0.03/0.18, dsn=2.0.0, status=sent (250 2.0.0 <klondike+gentoo (at) xiscosoft.es> Cd6YHwOAuVKJZgAA0J78UA Saved) I really have no idea of why sieve is behaving like this as the evelope address is sent properly by postfix's lmtp according to the logs. Any ideas? klondike PS: please do CC me when answering this e-mail as I'm not subscribed to the list yet

4 4

[Dovecot] RFC 5267 Contexts for IMAP4 PARTIAL results
by Anders Thøgersen 11 Feb '14

11 Feb '14

Hello, I am looking at PARTIAL responses using a dovecot version 2.2.5 and am running into an inconsistency with the RFC which I would like to ask about. The following section http://tools.ietf.org/html/rfc5267#section-4.4 lists client requests and server responses which I have tried running against a folder, specifying the ranges that fit the contents of the folder I have selected. The Last of the listed client requests returns a response that is not as I would expect: C: A04 UID SEARCH RETURN (PARTIAL 24000:24500) UNDELETED UNKEYWORD $Junk should return S: * ESEARCH (TAG "A04") UID PARTIAL (24000:24500 NIL) // No results are present, this is beyond the end of the results. S: A04 OK Completed. according to the RFC, but instead of NIL in the PARTIAL result I see the full range of UIDs that are in the folder, as in: C: A02 UID SEARCH RETURN (PARTIAL 23500:24000) UNDELETED S: * ESEARCH (TAG "A02") UID PARTIAL (23500:24000 1:9) S: A02 OK Search completed (0.000 secs). The folder that is selected contains 9 emails with the UIDs 1-9. Should the server return NIL Instead of returning 1:9? Thanks! Anders

2 2

[Dovecot] Dsync Errors
by Asai 07 Feb '14

07 Feb '14

Greetings, I've seen something like this in another thread, about a month ago, but we're running into this error pretty frequently when we run dsync backup. Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x39ccc5f1c0] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x35) [0x39ccc5f2a5] -> /usr/lib64/dovecot/libdovecot.so.0 [0x39ccc5ebb3] -> dsync(dsync_brain_mailbox_tree_sync_change+0x41c) [0x42cc0c] -> dsync(dsync_brain_recv_mailbox_tree_deletes+0x10a) [0x42be7a] -> dsync(dsync_brain_run+0x4fc) [0x429cdc] -> dsync [0x42885b] -> dsync [0x4119ff] -> dsync(doveadm_mail_try_run+0x269) [0x411e09] -> dsync(main+0x358) [0x41a2c8] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3f4281d994] -> dsync [0x4111c9] We're running Dovecot 2.2.4 and the dsync command is this: dsync -u <username> backup maildir:/mnt/backups/<period>/<domain>/<username> When this happens to a particular users account I delete the backup and let it rebuild, which works for awhile, but then it happens again, and it seems to happen to particular users. Can you point me in the right direction to start troubleshooting this? Thanks. -- --Asai

3 6

[Dovecot] Auth socket can't listen using ssl
by Anthony Bourguignon 07 Feb '14

07 Feb '14

Hi, I'm using Dovecot 2.2.9 (debian package on testing). It seems that it's not possible to open an auth socket using ssl. I'm using this configuration : service auth { [...] inet_listener authxmpp-client { address = [ips] port = 5220 } inet_listener genericauth-client { ssl = yes address = [ips] port = 5221 } [...] } Both ports are running fine. But the 5221, which should listen in ssl (because of the "ssl = yes" as written in the documentation here : http://wiki2.dovecot.org/Services ) is using plaintext. If I telnet directly to it, the content is delivered on an unsecured socket. Is there a way to make ssl works on this kind of socket ? Thanks a lot

4 7

[Dovecot] mime support for pigeonhole
by Tim Dickson 06 Feb '14

06 Feb '14

is there any chance of rfc5703 (mime support/extension) being added to pigeonhole? I am trying to filter mail with .exe and .zip attachments, to no success. I have tried *header :contains "x-attached" [".zip", ".exe"] *but emails i'm getting don't have x-attached. a sample snippet is below. Subject: testa1 Content-Type: multipart/mixed; boundary="------------020304040002050407030900" This is a multi-part message in MIME format. --------------020304040002050407030900 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit this has a compressed file added a1 --------------020304040002050407030900 Content-Type: application/x-zip-compressed; name="pinglog.zip" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="pinglog.zip" I have tried *header :contains ["Content-Type","name","filename"] ["application/x-zip-compressed",".zip",".exe"]* but that doesn't work either. I can't use body because that doesn't include the mime headers, so the only way I can see is to use mime (aka rfc5703) I am using dovecot 2.1.8 with pigeonhole 0.3.3 (and am about to use dovecot 2.2.9 with pigeonhole 0.4.2 on another server) If I'm just missing the point, or there is already support, please point me in the right direction.

6 7

[Dovecot] imap2json
by Kai Hendry 05 Feb '14

05 Feb '14

Hi there, I've started work on a side project that I've been long wanting to do; better HTML archives of email. https://github.com/kaihendry/imap2json I'm just designing the JSON schema atm, and I thought it would be good to get your feedback (Esp Timo) before I make a horrible mistake. https://github.com/kaihendry/imap2json/blob/master/JSON-design.mdwn Is it a good idea to base the id of a converation by the sha1sum of the first message for example? Any other tips or words of encouragement would be great. Kind regards,

1 1

[Dovecot] dsync backup; compressed to uncompressed
by Michael Smith (DF) 04 Feb '14

04 Feb '14

I need to backup some of the mailboxes on our system. We are currently using mdbox with zlib compression plugin. The backups need to be maildir without compression. How can I accomplish this using dsync? The following just causes each msg file in maildir to be compressed as well. dsync -f -u user1 backup maildir:/var/tmp/user1 While this just causes the dsync to fail while spewing a bunch of errors because it can no longer read the compressed mdbox files. dsync -f -u user1 -o mail_plugins= backup maildir:/var/tmp/user1 # doveconf -n # 2.2.4: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_debug = yes auth_debug_passwords = yes auth_default_realm = domain1.net auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot-debug.log disable_plaintext_auth = no dotlock_use_excl = no lda_mailbox_autocreate = yes lock_method = dotlock mail_access_groups = mail mail_debug = yes mail_fsync = always mail_location = mdbox:~/mail:INDEX=~/index mail_plugins = quota zlib mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_read_locks = dotlock mbox_write_locks = dotlock mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 20 M mmap_disable = yes namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Drafts autosubscribe4 = Sent autosubscribe5 = INBOX mail_max_userip_connections = 10 managesieve_max_line_length = 65536 quota = dict:User quota::file:%h/dovecot.quota quota2_rule = Trash:storage=+10%% quota3_rule = Junk:storage=+20%% quota_rule = *:storage=100M:messages=100000 recipient_delimiter = + sieve_before = /var/opt/mail/global.sieve/ zlib_save = gz zlib_save_level = 9 } pop3_reuse_xuidl = yes protocols = imap pop3 sieve lmtp service auth { inet_listener auth { port = 113 } unix_listener auth-userdb { user = nobody } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 service_count = 1 } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } ssl = no ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota zlib sieve } protocol imap { mail_plugins = quota zlib imap_quota } protocol sieve { mail_max_userip_connections = 10 mail_plugins = quota zlib managesieve_max_line_length = 65536 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date } protocol lmtp { mail_plugins = quota zlib sieve }

2 2

[Dovecot] mail_log_events, but who exactly triggered events?
by Arkadiusz Miśkiewicz 04 Feb '14

04 Feb '14

Hi. mail_log_events is nice addition but how to log who exactly triggered particular event? For example 5 users from 5 IP addresses uses single imap user/mailbox. One of them deletes email and I'm logging delete related events. The only logged thing is: dovecot: imap(user): delete: box=INBOX, uid=673287, msgid=<some@thing>, size=1230 which tells me nothing about who triggered it actually (note all 5 users were logged in at deletion time) How to solve this problem? Thanks, -- Arkadiusz Miśkiewicz, arekm / maven.pl

3 6