- dovecot - dovecot.org

[Dovecot] Assertion failed with imapc after upgrading Dovecot from 2.1.7 to 2.2.9
by Sylvain 29 Jan '14

29 Jan '14

Hi ! I have an old Courier IMAP and in front of it, I have put a proxy cache with Dovecot/imapc. I use Debian Wheezy (stable) which package Dovecot in version 2.1.7. I have tested the upgrade to Debian Jessie (testing) which package Dovecot in version 2.2.9 but an assertion is thrown : dovecot: imap(xxx): Panic: file imapc-list.c: line 499 (imapc_list_delete_unused_indexes): assertion failed: (strncmp(vname, fs_list->ns->prefix, fs_list->ns->prefix_len) == 0) I have checked source code and have seen that if *imapc_list_prefix* is not set, assertion will not be "walked". It's works but special inbox aren't detected correctly in email clients. If I understand the meaning of *vname* variable, it is because our Courier IMAP send us "INBOX" which is the value of my *imapc_list_prefix* and thus, assertion is thrown. Here some details of my tests : Courier IMAP : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2005 Double Precision, Inc. See COPYING for distribution information. a login xxx xxx a OK LOGIN Ok. a list "" "*" * LIST (\HasNoChildren) "." "INBOX.Drafts" * LIST (\HasNoChildren) "." "INBOX.Trash" * LIST (\HasNoChildren) "." "INBOX.test" * LIST (\HasNoChildren) "." "INBOX.Sent" * LIST (\HasNoChildren) "." "INBOX.Junk" * LIST (\Unmarked \HasChildren) "." "INBOX" a OK LIST completed Dovecot version 2.1.7 : * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready. a login xxx xxx a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in a list "" "*" * LIST (\HasChildren) "." "INBOX" * LIST (\HasNoChildren \Drafts) "." "INBOX.Drafts" * LIST (\HasNoChildren \Trash) "." "INBOX.Trash" * LIST (\HasNoChildren) "." "INBOX.test" * LIST (\HasNoChildren \Sent) "." "INBOX.Sent" * LIST (\HasNoChildren \Junk) "." "INBOX.Junk" a OK List completed. Dovecot version 2.2.9 : * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. a login xxx xxx a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in a list "" "*" Connection closed by foreign host. And the dovecot configuration relative to the inbox : imapc_list_prefix = INBOX namespace inbox { inbox = yes separator = . prefix = INBOX. } Any help will be welcome :) Sylvain

1 1

[Dovecot] Shared mailbox ACL
by Alex Ferrara 29 Jan '14

29 Jan '14

Hi everyone, I am having some trouble with shared folders in trying to replicate how we use them with a Cyrus backend. The auth database is Samba4 active directory, so I am using an LDAP lookup to authenticate and forcing the UID and GID to be all the same. I have a post-login script that sets the group ACL, and this seems to be working fine. /mnt/mail is an NFS mount to a FreeNAS machine, and there is only one Dovecot server connecting to that NFS share. I have been able to get the inbox of the shared mailbox to appear in Thunderbird, but I would like to allow all subfolders to have the same ACLs. Is there a way to do this without having an ACL entry for each folder in the dovecot-acl file? I also cannot seem to create folders under the shared inbox. Thanks. Below is the output of /mnt/mail/acl/shared-mailboxes shared/shared-boxes/group/accounting/accounting 1 shared/shared-boxes/group/team1/team1 1 shared/shared-boxes/group/team2/team2 1 Below is the output of dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-22-pve x86_64 Debian 7.3 nfs auth_mechanisms = plain login auth_username_format = %n mail_debug = yes mail_location = maildir:/mnt/mail/mailboxes/%n/Maildir:INDEX=/var/local/dovecot-indexes/%n mail_plugins = acl mail_shared_explicit_inbox = no managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = yes location = maildir:/mnt/mail/mailboxes/%%n/Maildir:INDEX=/var/local/dovecot-shared/%%n prefix = ZGroup/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_defaults_from_inbox = yes acl_shared_dict = file:/mnt/mail/acl/shared-mailboxes sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap lmtp sieve" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-postlogin { executable = script-login /usr/local/bin/postlogin.py user = vmail } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota sieve } protocol lda { mail_plugins = quota sieve } protocol imap { mail_plugins = acl imap_acl }

2 4

Re: [Dovecot] Architecture for large Dovecot cluster
by Joseph Tam 29 Jan '14

29 Jan '14

>> Just a guess, but I expect the difference is because NetApp de-dupes >> by checksumming blocks and mark whole blocks as duplicates if they >> have the same checksum. > > True, the start of the message is always at byte 0, but because of > different header length per user for the same message (different mail > address with different lengths) the body will never start at the same > byte. Oh yes, that's right. I confused maildir format with sendmail queue files that separates header and body. There is still some similarlity for mass mailouts to the same mail domain: they will have almost identical headers +/- message IDs and a few bytes here and there, but as you say, SiS is the way to go for deduping bulky message attachments. Joseph Tam <jtam.home(a)gmail.com>

1 0

[Dovecot] recipient_delimiter -- feature request
by Andrew Ray 29 Jan '14

29 Jan '14

Hi, It would be nice if the recipient_delimiter configuration option supported multiple recipient delimiters as Postfix now does. e.g. now if I set: recipient_delimiter = +- in Postfix, then test-extension(a)sample.com and test+extension(a)sample.com are both recognized. But if I set: recipient_delimiter = +- in Dovecot, neither of the above are recognized and instead test+-extension(a)sample.com is valid. Well, presently this feature isn't that important to me, but I imagine it could be a source of confusion for people who are migrating from say qmail to postfix (as I am doing) and notice Postfix's cool ability to support both types of delimiters and assume Dovecot will as well (as I did). Thanks, Andrew Ray -- Andrew Ray +1 404-418-5443 +421 (0) 917 832 253 http://guns.freedomlives.net/

2 2

[Dovecot] post-login scripting failing with "denied access to user" v2.1.7
by Frank 29 Jan '14

29 Jan '14

I am trying to have a default sieve rule for new mailboxes that can be edited by users using the round cube sieve plugin. It would actually be better executed when a new user is created but my postfixadmin runs on a separate machine so that's a bit tricky. The "sieve-default =" setting won't quite achieve what I want as I want users to be able to edit the default rule (to adjust SPAM level) Mail works fine before I add my post-login script to 10-master.conf ---------------------- service imap { # tell imap to do post-login lookup using a socket called "imap-postlogin" executable = imap imap-postlogin } service imap-postlogin { # all post-login scripts are executed via script-login binary executable = script-login /etc/dovecot/custom_sieve/postlogin.sh # the script process runs as the user specified here (v2.0.14+): user = mail # also tried $default_internal_user and root # this UNIX socket listener must use the same name as given to imap executable unix_listener imap-postlogin { } } After adding this setting, login fails and gives the error below. It certainly looks like a permission issue, but I have checked my script is executable and owned by vmail:vmail. The bash script executes fine as the mail user. The other possibility is that dovecot is denying access to the $HOME variable, but that seems unlikely. Nevertheless, I am stuck! Log shows: ------------------ dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) dovecot: auth-worker(17522): mysql(database1): Connected to database postfixadmin dovecot: imap-login: Login: user=<test2(a)fqdn.net>, method=PLAIN, rip=10.0.3.120, lip=10.0.1.180, mpid=17524, TLS, session=<+blqxxxxxxxxxM> dovecot: imap(test2(a)fqdn.net): Post-login script denied access to user test2(a)fqdn.net BASH script: /etc/dovecot/custom_sieve/postlogin.sh -------------------------------------------------- #!/bin/sh if [ ! -f $HOME/dovecot.sieve ] # if file doesn't exist then mkdir -p $HOME/sieve cp /etc/dovecot/custom_sieve/default.sieve $HOME/sieve/managesieve.sieve ln -s $HOME/sieve/managesieve.sieve $HOME/dovecot.sieve fi DEFAULT SIEVE RULE: /etc/dovecot/custom_sieve/default.sieve ----------------------------------- require ["fileinto"]; # rule:[SPAM] if header :contains "X-Spam-Level" "**********" { fileinto "Junk"; } Config ======= # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.3 ext4 dict { sqldomainquota = mysql:/etc/dovecot/dovecot-sql-domain.conf sqluserquota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf } mail_location = maildir:/home/vmail/%d/%n/Maildir:INDEX=/home/vmail/%d/%n/Maildir/indexes mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User Quota::proxy::sqluserquota quota_rule2 = Trash:storage=+100M sieve = ~/../.dovecot.sieve sieve_dir = ~/../sieve } protocols = " imap sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-postlogin { executable = script-login /var/spool/mail/postlogin.sh user = vmail } service imap { executable = imap imap-postlogin } ssl_cert = </etc/ssl/mailserver/mailserver.pem ssl_key = </etc/ssl/mailserver/private/mailserver_dcp.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " quota sieve" postmaster_address = hidden(a)fqdn.net } protocol imap { mail_plugins = " quota imap_quota" }

2 2

Re: [Dovecot] Architecture for large Dovecot cluster
by Joseph Tam 28 Jan '14

28 Jan '14

Sven Hartge <sven(a)svenhartge.de> wrote: > Interesting datapoint: NetApp Deduplication did only recover about 1% of > storage space with mdbox-based mail storage, while on an maildir-based > mail storage, the rate was about 15%. (This was tested with a copy of > real user data, so is accurate for my workload.) Just a guess, but I expect the difference is because NetApp de-dupes by checksumming blocks and mark whole blocks as duplicates if they have the same checksum. The message body has the same block offset in maildir (i.e. the start of a message is at byte 0), whereas mdbox might align message body anywhere in a block, so you might have 512 different block configurations for the same message. I don't know whether message alignment would be a worthwhile optimization for mdbox. Joseph Tam <jtam.home(a)gmail.com>

2 1

Re: [Dovecot] Architecture for large Dovecot cluster
by Stan Hoeppner 28 Jan '14

28 Jan '14

This went to me only so bringing back on list. On 1/24/2014 11:09 AM, Tom Johnson wrote: > Is anybody using the Object Storage plugin for large-scale > installations? I've not used it. > We're considering it, but are thinking of an in-house > S3 storage system (riak, or ceph, or ?) Looking to support perhaps > 300k users. I was thinking that if we use a bank of dovecot servers > (with director) with ssds as cache, we might be able to consolidate > all the storage on something like a riak cluster, which would make > scaling simple and inexpensive - certainly much less than a NetApp > solution. Everything costs less than a NetApp...except an EMC. > If anyone has any first-hand experience (or even > off-the-top-of-their-head thoughts), I'd love to hear them) Distributed filesystems give you the advantage of a single filesystem namespace with massive amounts of storage, fairly easy addition of storage space, and distributed replication to allow failure of a storage node without service interruption. Replication mitigates node failure, but not disk failure, so you still need RAID in each node. So you have RAID6 in a node and filesystem block mirroring amongst nodes. Thus storage utilization is -worse- than direct attach, CFS on SAN, or NFS head attached RAID10 and far worse than RAID6 in these 3 setups. And if using large SSD cache you'd surely use RAID6 with DAS, CFS, or NFS. You'd need half as many disk drives vs DFS. Each DFS expansion, assuming the typical model, entails the cost of a server, RAID HBA (unless using md) and disks, not strictly buying disks as with DAS, CFS/SAN, or NFS filer. Then you also need more switch ports, more power connections, greater UPS capacity due to all the CPUs, RAM, etc in the nodes. And you'll have a higher electric bill. So while a distributed filesystem storage architecture may seem less expensive it may not be. And just as one can build a DIY DFS cluster, one can also build a DIY NFS cluster instead of buying a NetApp, saving significant cash on the front end box and on disks since you'd need half as many vs a distributed filesystem architecture, though failure of one node may not be quite as graceful as with a NetApp losing a controller board. -- Stan

3 4

[Dovecot] Replication - what am I doing wrong?
by IT geek 31 27 Jan '14

27 Jan '14

Hi, I'm having issues getting replication working, and I'm not also not having much luck finding help on it. So I'm wondering... am I not asking the right questions? Or am I not providing enough information... or do we just not know how this aspect of the technology works? If it's the former, then any pointers on what questions to ask or information to provide would be very helpful. If the community genuinely doesn't know how this works then that's okay too - I will look at another solution (if one exists) and try to move on. But if I can get this working then that would be amazing as I really like Dovecot and would be gutted to move away from it. -Mark

2 2

[Dovecot] managesieve - point me in the right direction?
by erik 27 Jan '14

27 Jan '14

Hi, I am using Dovecot 2.1.16. I'm trying to get Roundcube to talk to the managesieve plugin. Postfix/Dovecot/Roundcube all exist on the same server. I'd like to get managesieve working. I've gone through a few guides, such as: https://wiki.archlinux.org/index.php/dovecot#Sieve But so far roundcube won't connect to the managesieve plugin. ("Cant reach server"). I see that port 4190 is listening and I can telnet to that port: Connected to localhost. Escape character is '^]'. "IMPLEMENTATION" "Dovecot Pigeonhole" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave" "NOTIFY" "mailto" "SASL" "PLAIN LOGIN" "STARTTLS" "VERSION" "1.0" OK "Dovecot ready." I'm not sure what else to check. Any help appreciated!

2 1

[Dovecot] Panic: file index-mailbox-check.c: line 75 (index_mailbox_check_add): assertion failed: (box->notify_min_interval > 0)
by Kevin Holly 27 Jan '14

27 Jan '14

Hey guys, i have a weird problem i'm trying to fix since 5 months now, but i really couldn't find any explanation, why this happens and how it can be fixed. My client is Thunderbird, my dovecot version is 2.1.7-7 from the official Debian repo. If you want to see my config i'll grab all the fils together and send it to you. This is the error i'm always getting when my client tries to get new mails, which i want to get rid off: http://sprunge.us/NZhf Jan 25 11:49:30 old dovecot: auth-worker(15979): Debug: sql(root(a)hallowe.lt,81.217.212.85): query: SELECT password FROM mailbox WHERE username = 'root(a)hallowe.lt' Jan 25 11:49:30 old dovecot: auth: Debug: client out: OK#0111#011user=root(a)hallowe.lt Jan 25 11:49:30 old dovecot: auth-worker(15979): Debug: sql(root(a)hallowe.lt,81.217.212.85): SELECT CONCAT('maildir:/var/vmail/',maildir) AS mail, 1000 AS uid, 1000 AS gid FROM mailbox WHERE username = 'root(a)hallowe.lt' Jan 25 11:49:30 old dovecot: auth: Debug: master out: USER#0113983671297#011root@hallowe.lt#011mail=maildir:/var/vmail/hallowe.lt/root/#011uid=1000#011gid=1000 Jan 25 11:49:30 old dovecot: imap-login: Login: user=<root(a)hallowe.lt>, method=PLAIN, rip=81.217.212.85, lip=10.111.56.107, mpid=17606, TLS, session=<rQGqOsnwuABR2dRV> Jan 25 11:49:30 old dovecot: imap(root(a)hallowe.lt): Debug: Effective uid=1000, gid=1000, home= Jan 25 11:49:30 old dovecot: imap(root(a)hallowe.lt): Debug: quota: No quota setting - plugin disabled Jan 25 11:49:30 old dovecot: imap(root(a)hallowe.lt): Debug: maildir++: root=/var/vmail/hallowe.lt/root, index=, control=, inbox=/var/vmail/hallowe.lt/root, alt= Jan 25 11:49:30 old dovecot: imap(root(a)hallowe.lt): Panic: file index-mailbox-check.c: line 75 (index_mailbox_check_add): assertion failed: (box->notify_min_interval > 0) Jan 25 11:49:30 old dovecot: imap(root(a)hallowe.lt): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4414a) [0x7f858118714a] -> /usr/lib/dovecot/libdovecot.so.0(+0x4418e) [0x7f858118718e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f858115e82a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x99027) [0x7f858146b027] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x4ccc9) [0x7f858141ecc9] -> dovecot/imap(cmd_idle+0x66) [0x7f8581907c96] -> dovecot/imap(command_exec+0x3c) [0x7f858190dedc] -> dovecot/imap(+0x11f2e) [0x7f858190cf2e] -> dovecot/imap(+0x11fda) [0x7f858190cfda] -> dovecot/imap(client_handle_input+0x11d) [0x7f858190d23d] -> dovecot/imap(client_input+0x65) [0x7f858190db35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f8581194016] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f8581194cc7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f8581193a28] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8581180483] -> dovecot/imap(main+0x297) [0x7f85819053e7] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f8580dd7ead] -> dovecot/imap(+0xa561) [0x7f8581905561] Jan 25 11:49:30 old dovecot: imap(root(a)hallowe.lt): Fatal: master: service(imap): child 17606 killed with signal 6 (core dumps disabled) I asked for this in the irc already but got no answer. I would be really happy if someone can help me with this. Thank you very much! Best regards, Kevin Holly - root(a)hallowe.lt - http://hallowe.lt/

2 3