- dovecot - dovecot.org

RFC 3501 violation in FETCH BODY responses
by Guilhem Moulin 18 Dec '16

18 Dec '16

Hi there, Quoting RFC 3501 sec. 7.4.2 “FETCH Response” (data item BODYSTRUCTURE): “A body type of type MESSAGE and subtype RFC822 contains, immediately after the basic fields, the envelope structure, body structure, and size in text lines of the encapsulated message.” According the ABNF (RFC 3501 sec. 9) the envelope structure is that of the ENVELOPE FETCH data item, and the env-{from,sender,reply-to,to,cc, bcc} fields are non-space-separated address lists: body-type-msg = media-message SP body-fields SP envelope SP body SP body-fld-lines envelope = "(" env-from SP … SP env-to SP … ")" env-from = "(" 1*address ")" / nil env-to = "(" 1*address ")" / nil While this is indeed the case for ‘FETCH … (ENVELOPE)’, for ‘FETCH … (BODY)’ dovecot 2.2.25 adds a space between addresses of an address list of the envelope structure of an encapsulated MESSAGE/RFC822 message. See the attached patch to ‘src/lib-imap/test-imap-bodystructure.c’, which currently (2.2.25) fails as follows test-imap-bodystructure.c:122: Assert failed: strcmp(str_c(str), testmsg_body) == 0 test-imap-bodystructure.c:129: Assert failed: strcmp(str_c(str), testmsg_bodystructure) == 0 imap bodystructure parser ............................................ : FAILED because the ‘env-to’ field of the envelope structure of the encapsulated MESSAGE/RFC822 message is printed as ((NIL NIL "sub-to1" "domain.org") (NIL NIL "sub-to2" "domain.org")) while it should be ((NIL NIL "sub-to1" "domain.org")(NIL NIL "sub-to2" "domain.org")) After a quick look at the source, this seems to be due to src/lib-imap/imap-bodystructure.c:imap_write_list, which always separates list items with spaces. In the case of an envelope, only the top-level list should be space-separated. Indeed, not adding a space between items of type IMAP_ARG_LIST in the recursive call makes the test pass again. Cheers, -- Guilhem.

2 1

Error while migrating mailbox with 'doveadm sync'
by Zhang Huangbin 18 Dec '16

18 Dec '16

Dear all, I successfully migrated hundreds of mailboxes with Dovecot 2.2.24 on OpenBSD 6.0 (amd64. Old server is OpenBSD 5.1, i386, Dovecot-2.0.17), but got error while migrating 3 mailboxes: # doveadm -o mail_fsync=never sync -1 -R -s '' -u user(a)domain.com imapc: dsync(user(a)domain.com): Panic: file mailbox-alias-plugin.c: line 77 (mailbox_symlink_exists): assertion failed: (ret > 0) Abort trap Same error while running ‘doveadm … backup …'. What does this error actually mean? How can i fix it? - Here’s full log running doveadm with ‘-D’ option: http://pastebin.com/Um5fbuNM - ‘dovecot -n’ on old server: http://pastebin.com/4E6xzFKU - ‘dovecot -n’ on new server: http://pastebin.com/4uHwQnMN I checked release notes of Dovecot 2.2.25/26/27, there’re 3 fixes related to 'doveadm sync’, but not sure whether it helps fix this issue. 2.2.26: + doveadm sync/backup: Added -I <max size> parameter to skip too large mails. + doveadm sync/backup: Fixed -t parameter and added -e for "end date”. 2.2.27: - doveadm sync -1: Fixed handling mailbox GUID conflicts.

1 1

Antispam plugin: insufficent error messages
by Volker Wysk 18 Dec '16

18 Dec '16

Hi! "Dovecot always logs a detailed error message if something goes wrong. If it doesn't, it's considered a bug and will be fixed." (http:// wiki2.dovecot.org/Logging) I'm trying to set up a spam filter with dovecot-antispam and dspam as backend. When I move a spam message from INBOX to Spam, I get this in syslog: Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(Spam): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(Spam): 0 Nov 28 21:15:58 desktop imap: antispam: mail copy: from trash: 0, to trash: 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(Spam): 1 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 There are no log entries about how dspam is called, or what goes on. Indeed, dspam doesn't get called at all. I know, because I've examined the antispam- plugin source code. There would be a log message if dspam was called. It's hard to study the source code, because there are hardly any comments. I have been able to trace the problem to function signature_extract_to_list() in signature.c. There, -1 is returned: int signature_extract_to_list(const struct signature_config *cfg, struct mailbox_transaction_context *t, struct mail *mail, struct siglist **list, enum classification wanted) { const char *const *signatures; struct siglist *item; signatures = get_mail_headers(mail, cfg->signature_hdr); if (!signatures || !signatures[0]) { if (!cfg->signature_nosig_ignore) { mail_storage_set_error(t->box->storage, ME(NOTPOSSIBLE) "antispam signature not found"); return -1; /* <-- HERE */ } else { return 0; } } while (signatures[1]) signatures++; item = i_new(struct siglist, 1); item->next = *list; item->wanted = wanted; item->sig = i_strdup(signatures[0]); *list = item; return 0; } So, what's happening is "antispam signature not found". My question: What does this mean? What's going on? My "dovecot -n" output is attached. Bye Volker

4 7

Managing Sieve on macOS 10.12 Sierra with Server 5.2?
by Gerben Wierda 17 Dec '16

17 Dec '16

I don’t know if this is considered OT, but dovecot has support for sieve. I had a rule set running under Mac OS X 10.8.5 Server 2.2.5 which I did maintain by hand on the server because a while before Mac OS X Server had dropped support for Squirrelmail. Now I wonder if there is a decent way to get sieve running again on macOS 10.12 Sierra with Server 5.2 G

1 0

Existing shared mailbox ... does not exist
by Ruga 17 Dec '16

17 Dec '16

Hello, Dovecot 2.2.26.0 Thunderbird 45.4.0 ESR Scenario: We have a shared mailbox with e-mails in its subdirectories. There are no e-mails in the shared mailbox itself. Example: /Shared <----------------------------------- empty /Shared/call_for_papers <-------------- empty /Shared/call_for_papers/AAMAS <-- populated ...more of the same... Problem: When selecting /Shared, Thunderbird chirps this message: "The current operation on 'Shared' did not succeed. The mail server for account @example.com responded: Mailbox doesn't exist: Shared/Shared (0.000 + 0.000 secs)."

1 3

Fwd: Re: making a plugin encrypt index data
by Aki Tuomi 16 Dec '16

16 Dec '16

> ---------- Original Message ---------- > From: Aki Tuomi <aki.tuomi(a)dovecot.fi> > To: micah anderson <micah(a)riseup.net> > Date: December 16, 2016 at 11:25 PM > Subject: Re: making a plugin encrypt index data > > When we released our encryption plugin, mail-crypt, it's capabilities include fs-crypt. This can be used to encrypt things like FTS indexes and attachments, and with suitable mail storage, such as obox, you can also encrypt indexes. > > To extend this support to dbox or maildir, does require rather involved changes in dovecot core, which currently has no support for fs-api in index handling. This might happen on v2.3 or v2.4, depending. > > I somehow suspect that the work estimate would exceed your budget. But it is going to happen, it's just matter of time. Can't give you any timeline though, since we have not decided on one yet. > > Aki > > > On December 16, 2016 at 9:53 PM micah anderson <micah(a)riseup.net> wrote: > > > > > > > > Hi Aki, > > > > Do you have any idea approximately when this would be planned for? > > > > We are also interested potentially paying for the ability to encrypt our > > indexes, as this is a major concern for us. We don't have a lot of money > > as a non-profit, but if there is a possibility of contract work, we > > would be interested to know what it would cost to do it. > > > > thanks, > > micah > > > > Aki Tuomi <aki.tuomi(a)dovecot.fi> writes: > > > > >> On December 16, 2016 at 6:48 PM micah anderson <micah(a)riseup.net> wrote: > > >> > > >> > > >> > > >> Hello, > > >> > > >> I'm encrypting mail on disk using a plugin[0], but the index files are > > >> not encrypted (specifically the dovecot.index.cache can be read). > > >> > > >> I want to do is encrypt index on disk, so I'm looking for how a plugin > > >> can achieve that by hooking into the right locations. Is that easily > > >> possible in a plugin? > > >> > > >> I can turn off those indexes by passing INDEX=MEMORY, but that isn't > > >> possible if I use sdbox/mdbox. > > >> > > >> thanks for any suggestions! > > >> micah > > >> > > >> 0. https://0xacab.org/riseuplabs/tofu-scrambler > > > > > > Hi! > > > > > > At the moment it's not possible without making new storage class. We are planning to add support for fs drivers for indexes at some point. > > > > > > Aki Tuomi > > > Dovecot Oy

1 0

service(indexer-worker): child 18977 killed with signal 11 (core dumped)
by Matteo Biagini 16 Dec '16

16 Dec '16

Hello to everybody, first of all: thanks a lot for your hard work and for this nice piece of software! After many search about the following SIGSEGV, I decide to post the backtrace and I hope to have avoided duplicated thread. The problem arise after I enable clucene fts. I tryed the clucene ports version (I am using OpenBSD) and a fixed version from github (Blue-Rocket/clucene::feature/pod) which contains FreeBSD fixes and others memory leak patches, but the problem persists. I built either dovecot-* and clucene-core with -g and the following logs are taken from that version. Maybe, I can do more debugging but I don't know how; some ideas? thanks a lot == LOG == Dec 16 12:30:34 mail02 dovecot: indexer-worker(user@hostname): Indexed 1 messages in Trash Dec 16 12:31:16 mail02 dovecot: imap-login: Login: user=<user@hostname>, method=PLAIN, rip=10.244.3.101, lip=10.244.3.200, mpid=90234, TLS, session=<68Ny6MRDml8K9ANl> Dec 16 12:31:16 mail02 dovecot: imap-login: Login: user=<user@hostname>, method=PLAIN, rip=10.244.3.101, lip=10.244.3.200, mpid=59810, TLS, session=<c7xz6MRDzV4K9ANl> Dec 16 12:31:16 mail02 dovecot: imap-login: Login: user=<user@hostname>, method=PLAIN, rip=10.244.3.101, lip=10.244.3.200, mpid=66756, TLS, session=<UtNz6MRDiJwK9ANl> Dec 16 12:31:16 mail02 dovecot: imap-login: Login: user=<user@hostname>, method=PLAIN, rip=10.244.3.101, lip=10.244.3.200, mpid=81959, TLS, session=<gJZ06MRD9HQK9ANl> Dec 16 12:31:16 mail02 dovecot: imap-login: Login: user=<user@hostname>, method=PLAIN, rip=10.244.3.101, lip=10.244.3.200, mpid=14099, TLS, session=<BL116MRD3DgK9ANl> Dec 16 12:31:16 mail02 dovecot: indexer-worker(user@hostname): Indexed 1 messages in INBOX Dec 16 12:32:18 mail02 dovecot: indexer-worker: Fatal: master: service(indexer-worker): child 18977 killed with signal 11 (core dumped) == GDB == (gdb) bt #0 strlen () at /usr/src/lib/libc/arch/amd64/string/strlen.S:152 #1 0x0000093434d7c120 in __vfprintf (fp=0x7f7fffff96c0, fmt0=0x9343cef79ae "Timeout leak: %p (%s:%u)", ap=Variable "ap" is not available. ) at /usr/src/lib/libc/stdio/vfprintf.c:885 #2 0x0000093434d9b068 in *_libc_vsnprintf (str=0x933ab0940b8 "Timeout leak: 0x9342f652300", n=120, fmt=0x9343cef79ae "Timeout leak: %p (%s:%u)", ap=0x7f7fffff9870) at /usr/src/lib/libc/stdio/vsnprintf.c:61 #3 0x000009343cdd4aa5 in str_vprintfa (str=0x933ab094078, fmt=0x9343cef79ae "Timeout leak: %p (%s:%u)", args=0x7f7fffff9870) at str.c:129 #4 0x000009343cdaacd1 in internal_handler (ctx=0x9343d22fab0, format=0x9343cef79ae "Timeout leak: %p (%s:%u)", args=0x7f7fffff9870) at failures.c:602 #5 0x000009343cdab839 in i_internal_error_handler (ctx=Variable "ctx" is not available. ) at failures.c:677 #6 0x000009343cdab3f4 in i_warning (format=0x9343cef79ae "Timeout leak: %p (%s:%u)") at failures.c:325 #7 0x000009343cdc107b in io_loop_destroy (_ioloop=Variable "_ioloop" is not available. ) at ioloop.c:741 #8 0x000009343cd485bd in master_service_deinit (_service=Variable "_service" is not available. ) at master-service.c:876 #9 0x0000093193802642 in main (argc=1, argv=0x7f7fffff9b28) at indexer-worker.c:83 (gdb) bt full #0 strlen () at /usr/src/lib/libc/arch/amd64/string/strlen.S:152 No locals. #1 0x0000093434d7c120 in __vfprintf (fp=0x7f7fffff96c0, fmt0=0x9343cef79ae "Timeout leak: %p (%s:%u)", ap=Variable "ap" is not available. ) at /usr/src/lib/libc/stdio/vfprintf.c:885 mbs = { __mbstate8 = "{\t\000\000\000\000\000\000\000\000\000\000r3\t\000\000\000\000\000\000\000\000\000\0004\t\000\000\001\000\000\000\000\000\000\000223\177\177\000\000\001\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\t216U\035\rP", '\0' <repeats 47 times>, __mbstateL = 10118011532464} mbseqlen = Variable "mbseqlen" is not available. == DOVECONF == # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: OpenBSD 6.0 amd64 auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN base_dir = /var/run/dovecot dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } first_valid_uid = 2000 last_valid_uid = 2000 listen = * [::] login_greeting = Welcome mail_gid = 2000 mail_location = mdbox:%Lh/Maildir/:INDEX=%Lh/Maildir/ mail_plugins = quota mailbox_alias acl zlib fts fts_lucene trash mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mdbox_rotate_size = 60 M namespace { inbox = yes location = mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = no special_use = \Archive } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox "Junk E-mail" { auto = no special_use = \Junk } mailbox SPAM { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 30 days special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = mdbox:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%Ld/%%Ln prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users driver = passwd-file master = yes } plugin { acl = vfile acl_shared_dict = proxy::acl antispam_backend = MAILTRAIN antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/local/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = Spam;Junk antispam_spam_pattern = spam;Spam;junk;Junk antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_trash_pattern = trash;Trash;Deleted * antispam_trash_pattern_ignorecase = TRASH auth_socket_path = /var/run/dovecot/auth-master fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = SPAM mailbox_alias_new4 = Spam mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Junk mailbox_alias_old4 = Junk quota = dict:user::proxy::quotadict quota_grace = 10%% quota_rule = *:storage=1G quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u sieve = %Lh/sieve/dovecot.sieve sieve_before = /var/vmail/sieve/dovecot.sieve sieve_dir = %Lh/sieve sieve_global_dir = /var/vmail/sieve sieve_max_redirects = 30 trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = lz4 zlib_save_level = 6 } protocols = pop3 imap sieve lmtp service auth { inet_listener { address = 127.0.0.1 port = 12345 } unix_listener /var/spool/postfix/private/dovecot-auth { group = _postfix mode = 0666 user = _postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_limit = 500 service_count = 1 } service lmtp { executable = lmtp -L inet_listener lmtp { address = 127.0.0.1 port = 24 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = _postfix mode = 0600 user = _postfix } user = vmail } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 1 } service quota-status { client_limit = 2 executable = quota-status -p postfix inet_listener { address = 127.0.0.1 port = 12340 } } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } ssl = required ssl_cert = </etc/ssl/dovecot.crt ssl_cipher_list = EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH ssl_dh_parameters_length = 2048 ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 syslog_facility = local5 userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } protocol lda { auth_socket_path = /var/run/dovecot/auth-master lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_plugins = quota mailbox_alias acl zlib fts fts_lucene trash sieve notify push_notification postmaster_address = root } protocol lmtp { lmtp_save_to_detail_mailbox = yes log_path = syslog mail_plugins = quota sieve notify push_notification postmaster_address = postmaster recipient_delimiter = + } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep imap_idle_notify_interval = 29 mins imap_max_line_length = 4 M mail_max_userip_connections = 30 mail_plugins = quota mailbox_alias acl zlib fts fts_lucene trash imap_quota imap_acl imap_zlib antispam } protocol pop3 { mail_max_userip_connections = 30 mail_plugins = quota mailbox_alias acl zlib fts fts_lucene trash pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv }

2 1

making a plugin encrypt index data
by micah anderson 16 Dec '16

16 Dec '16

Hello, I'm encrypting mail on disk using a plugin[0], but the index files are not encrypted (specifically the dovecot.index.cache can be read). I want to do is encrypt index on disk, so I'm looking for how a plugin can achieve that by hooking into the right locations. Is that easily possible in a plugin? I can turn off those indexes by passing INDEX=MEMORY, but that isn't possible if I use sdbox/mdbox. thanks for any suggestions! micah 0. https://0xacab.org/riseuplabs/tofu-scrambler

2 1

v2.2.27 Debian packages on xi.dovecot.fi
by Markus Schönhaber 16 Dec '16

16 Dec '16

Hi, at the moment I see only packages named *2.2.26* on xi.dovecot.fi. Can we expect packages for v2.2.27 to appear there? -- Regards mks

2 2

public folder subscriptions sync issue with ldap user/group in dovecot-acl
by Mike Fröhner 16 Dec '16

16 Dec '16

Hello people, I am having an issue with 'doveadm sync'. I am currently trying to have two dovecots behind an haproxy (works fine). Therefore I configured these two dovecot server (imap-1/imap-2) to sync throught dsync. This works just partly. The sync of the maiboxes is fine, but the sync of the subscriptions file just works partly. It works for private folder subscription, but not completly for public folder subscription. I found two issues, if I am using LDAP (user/groups) in dovecot ACLs. 1. I would like to subscribe 2 public folder (public/test/test1 and public/test/test2). My user (ldaptestuser) is an ldap user and this user is member of the ldap group (ldaptestgroup) which does have all dovecot-acl rights on these folders. imap-1 # cat /opt/mail/_public/publictest/.test*/dovecot-acl group=ldaptestgroup akxeilprwts group=ldaptestgroup akxeilprwts I am now connecting with my mail client to imap-1 (throught haproxy) and the subscription to this folder works. The file which is written looks like: imap-1 # cat /opt/mail/ldaptestuser/Mails/subscriptions Sent publictest/test/test1 publictest/test/test2 Now I am awaiting the synch to imap-2, but the file which it written looks like: imap-2 # cat /opt/mail/ldaptestuser/Mails/subscriptions Sent If I modify the dovecot-acl for .test1 to imap-1 # cat /opt/mail/_public/publictest/.test1/dovecot-acl group=ldaptestgroup akxeilprwts user=ldaptestuser akxeilprwts and execute the subscription again - the synced file looks like: imap-2 # cat /opt/mail/ldaptestuser/Mails/subscriptions Sent publictest/test/test1 The subscription of public folder test2 will also been synced, if I add my ldaptestuser to the acl file for this folder. 2. Another issue is to unsubscribe a public folder. If I unsubscribe folder test1, it is written to subscriptions file on the imap where I am connected, but it is NOT synced even if my user and group are configured at the dovecot-acl file. If I then unsubscribe a not public folder (like Sent), the former unsubscribed folder test1 is (faulty) subscribed again. But both imap do have the same subscriptions for my ldaptestuser user. I do have the behavior with dovecot-2.2.26 and dovecot-2.2.27 on CentOS-7 (selinux disabled). If you need more information like the dovecot -n or some other stuff give me a short notice. Mike;

2 4