- dovecot - dovecot.org

Too many references: cannot splice
by Sebastian Köhler 02 Dec '16

02 Dec '16

Hello, multiple times per day one of our Dovecot servers has the problem that it suddenly refuses to accept new connections and then floods the logs with these messages: Dec 2 12:43:06 alfa3201 dovecot: pop3-login: Error: fd_send(pop3, 18) failed: Too many references: cannot splice Dec 2 12:43:07 alfa3201 dovecot: pop3-login: Error: fd_send(pop3, 18) failed: Too many references: cannot splice Dec 2 12:43:07 alfa3201 dovecot: imap-login: Error: fd_send(imap, 16) failed: Too many references: cannot splice Dec 2 12:43:07 alfa3201 dovecot: imap-login: Error: fd_send(imap, 16) failed: Too many references: cannot splice We have to either do a restart of the server or a doveadm kick on all users to make it reachable again. On my search for a solution I found a similar report from March[1] however without a fix. [1] http://dovecot.org/pipermail/dovecot/2016-March/103514.html dovecot -n # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.17-040417-generic x86_64 Debian 8.6 auth_mechanisms = plain login debug_log_path = /var/log/dovecot-debug.log default_client_limit = 8003 default_process_limit = 2000 default_vsz_limit = 512 M disable_plaintext_auth = no listen = * login_greeting = [CENSORED] - ready. mail_location = maildir:~/Maildir:CONTROL=/opt/dovecot/control/%n-control:INDEX=/opt/dovecot/index/%n mail_plugins = " stats quota" namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . } passdb { args = dovecot-%s driver = pam } plugin { quota = fs:%n(soft quota) sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 30 secs stats_session_min_time = 15 mins stats_track_cmds = no stats_user_min_time = 24 hours } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imaps { ssl = yes } } service imap-postlogin { executable = script-login /etc/dovecot/post-login/expire.sh } service imap { executable = imap imap-postlogin } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3-postlogin { executable = script-login /etc/dovecot/post-login/expire.sh } service pop3 { executable = pop3 pop3-postlogin } service quota-status { executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota { group = postfix mode = 0660 user = postfix } } service stats { fifo_listener stats-mail { mode = 0666 user = dovecot } } ssl_ca = </etc/ssl.crt/ca-admin.crt ssl_cert = </etc/ssl.crt/admin.crt ssl_key = </etc/ssl.key/admin.key ssl_protocols = !SSLv2 !SSLv3 userdb { driver = passwd } protocol imap { imap_capability = +XLIST +NAMESPACE METADATA QUOTA imap_idle_notify_interval = 1 mins mail_max_userip_connections = 500 mail_plugins = " stats quota imap_quota imap_stats" } protocol pop3 { mail_max_userip_connections = 500 mail_plugins = " stats quota" pop3_client_workarounds = outlook-no-nuls oe-ns-eoh }

2 2

Defining INDEX target to other location than maildir seems to have no effect.
by Julian Sternberg 02 Dec '16

02 Dec '16

Dovecot Version 2.2.13 Linux Distribution: Debian Jessie CPU Architecture: x64 Filesystem: GlusterFS/NFS, XFS for Base System/Index Files. Two Dovecot/Postfix nodes accessing same GlusterFS/NFS Maildir. Regardless what i choose in mail_location (:INDEX=MEMORY or :INDEX=/var/indexes/%d/%n) all Mailbox index files will still get created within the users mail_location maildir: ~/Maildir My mail_location Maildir directory is shared on GlusterFS mount so i need to get index files away from this share due locking mechanism and faster caching. The weird is, if i set INDEX to /var/indexes, the index files getting created sometimes but then not updated frequently and exist parallel on Maildir which are mostly newer than on /var/indexes. If you delete the dovecot.index* files from Maildir, they will get recreated immediatly on Imap access but not on the alternative set INDEX location, they are getting recreated within Maildir again. Here is doveconf -n Output: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 auth_mechanisms = plain login cram-md5 disable_plaintext_auth = no first_valid_uid = 2000 hostname = censored.hostname.com last_valid_uid = 2000 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * lock_method = dotlock mail_fsync = always mail_gid = 2000 mail_home = /storage/vmail/%d/%n mail_location = maildir:~/Maildir:LAYOUT=fs:INDEX=MEMORY mail_nfs_storage = yes mail_privileged_group = vmail mail_temp_dir = /var/tmp mail_uid = 2000 maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = no special_use = \Archive } mailbox "Deleted Items" { auto = no special_use = \Trash } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = create special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox virtual/All { auto = no special_use = \All } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_rule3 = Sent:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /storage/vmail/%d/%n/sieve/dovecot.sieve sieve_before = /storage/vmail/sieve/dovecot.sieve sieve_dir = /storage/vmail/%d/%n/sieve sieve_global = /storage/vmail/sieve } postmaster_address = postmaster(a)censored.hostname.com protocols = " imap lmtp sieve pop3" quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } service_count = 0 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_ca = </etc/ssl/censored.hostname.com/ssl-bundle-mail.crt ssl_cert = </etc/ssl/censored.hostname.com/censored_hostname.com.crt ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_key = </etc/ssl/censored.hostname.com/censored.hostname.com.key ssl_protocols = !SSLv2 !SSLv3 ssl_require_crl = no userdb { args = uid=2000 gid=2000 home=/storage/vmail/%d/%n allow_all_users=yes driver = static } protocol lmtp { mail_plugins = " quota sieve" } protocol lda { mail_plugins = " sieve quota" } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = " quota imap_quota" } protocol sieve { mail_max_userip_connections = 10 } protocol pop3 { mail_plugins = " quota" pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv }

3 3

maildir-size-fix.pl - The file command can wrongly detect gzipped mails as being Minix filesystems
by Rarylson Freitas 02 Dec '16

02 Dec '16

Hi, Some days ago, I have used the maildir-size-fix.pl script (cited in this mail thread: http://www.dovecot.org/list/dovecot/2013-January/087942.html, in available here: http://dovecot.org/tools/maildir-size-fix.pl) This is a very good script and helped me a lot! However, some compressed mails where misdetected as non-compressed mail. So I discovered that some versions of the file command (some old versions) can wrongly detect gzipped files as being Minix filesystems. Actually, the error was occurring in about 1 gzipped mail for each 60000 or 100000 mails. I reported this file command bug here: https://bugs.launchpad.net/ubuntu/+source/file/+bug/1646233. But I think it's worth to make a small improvement in maildir-size-fix.pl, making it more error prone. What do you, guys, think about this? This is the patch I'm proposing: ``` --- maildir-size-fix.pl.orig 2016-12-01 14:41:30.044689101 -0200 +++ maildir-size-fix.pl 2016-12-01 14:49:42.971197715 -0200 @@ -37,7 +37,10 @@ return $stat[7]; } - my $type=`file "$path"`; + # Detecting all possible types (keep going) since some versions of the file + # command can wrong detect gzip files as Minix filesystems. + # See: https://bugs.launchpad.net/ubuntu/+source/file/+bug/1646233 + my $type=`file -k "$path"`; my $program = ""; if ($type =~ /gzip/) { $program = "gunzip"; ``` -- *Rarylson Freitas*

2 1

Implementing secondary quota w/ "Archive" namespace
by Fred Turner 01 Dec '16

01 Dec '16

Hello folks— My first post, so please be gentle… :-) I have a client email server using SSDs for primary user mailboxes, but since the number of users keeps growing and they all seem to be very reluctant to delete anything, I’ve implemented an “Archive” namespace that stores its mailboxes on a larger HD RAID. The idea is that, as the users approach their quota, they move messages to the Archive mailboxes to alleviate space in their primary Inbox namespace. This secondary storage part is working well, but I’m having trouble w/ getting the quotas to work right. Here are the basics of the setup: Mac Pro Server 2012 Mac OS X Server 10.6.8 Dovecot 1.1.20apple0.5 Here is how I’ve configured my namespaces (during testing): namespace private { separator = / prefix = inbox = yes } namespace private { separator = / prefix = testArchive/ location = maildir:/Shared Items/MailArchive/%u subscriptions = yes } My quota research has led me to try this: quota = maildir:User quota:ns= quota2 = maildir:ns=testArchive/ quota2_rule = *:storage=20G The first line is already in the default config, with the exception of the added “:ns=“ at the end. The 2nd line in the examples I saw had a middle component w/ the quota name, but when I tried that, like so: quota2 = maildir:Archive quota:ns=testArchive/ my server fails and shows this in the logs: > Fatal: IMAP(*): Quota root test backend maildir: Unknown parameter: ns=testArchive/ Any idea why it doesn’t like that? Also, do I need to add a quota_rule for the primary quota? It does not have one normally in the Mac OS X Server config… Thus far in my testing, I’ve been able to get the 2 quotas to show up in Roundcube and Mac Mail.app. It’s a little messy…the first shows up as “User quota”, the 2nd as “ns=testArchive/“, presumably because I cannot leave the description field in there. Unfortunately, both quotas show the same amount of space in use. If I drop the primary quota to a mere 4MB for testing, and if I have 5.2MB of messages in a testArchive folder, the space used for “User quota” shows as 5.2MB (>100%), as does the “ns=testArchive/“ quota (which is 20GB). In actuality, the Inbox namespace is really only using a few KB— the 5.2MB is in the testArchive namespace. This means that I cannot move messages between either set of namespaces, and new messages are not delivered. So, the quota trouble here is negating the whole point of having the Archive namespace... Is there a way to get Dovecot to “see” the 2 quotas as unique/discrete? It seems like I’m close to accomplishing what I want, but just can’t quite get it to cooperate. And that “Unknown parameter” error is bewildering. Any ideas? Thx, Fred P.S. I can add my Dovecot config to the thread upon request…didn’t want to make this initial message even longer.

4 16

AW: User unknown in local recipient table
by Kalle Erlacher 01 Dec '16

01 Dec '16

Hallo Liste, hallo Ralf, danke schon mal. Ja die Whitespaces sind beim Rauslöschen entstanden. Ich verstand das so, dass das nun durch die address_verify_map erledigt würde. Muss ich dann entsprechend einfach auf die verweisen? Ich versuchte gerade die address_verify_map = btree:$data_directory/verify_cache auf local_recipient_maps = btree:$data_directory/verify_cache verweisen zu lassen. Leider ohne Erfolg. Bisher ist verify_cache nämlich leider noch unbefüllt. Entsprechend bekomme ich folgende Fehler: Dec 1 16:16:29 test-mail postfix/smtpd[5550]: warning: btree:/var/lib/postfix/verify_cache is unavailable. open database /var/lib/postfix/verify_cache.db: No such file or directory Dec 1 16:16:29 test-mail postfix/smtpd[5550]: warning: btree:/var/lib/postfix/verify_cache lookup error for "juergen @domain.org" Dec 1 16:16:29 test-mail postfix/smtpd[5550]: NOQUEUE: reject: RCPT from localhost[127.0.0.1]: 451 4.3.0 <juergen.pittorf(a)doag.org>: Temporary lookup failure; from=<kalle(a)domain.org> to=<juergen(a)domain.org> proto=ESMTP helo=<test-mail.domain.org> Auch ein plumpes touch auf /var/lib/postfix/verify_cache.db bringt danach den selben Fehler. Liebe Grüße Kalle -----Ursprüngliche Nachricht----- Von: Dovecot [mailto:dovecot-bounces@listen.jpberlin.de] Im Auftrag von Ralf Hildebrandt Gesendet: Donnerstag, 1. Dezember 2016 15:38 An: dovecot(a)listen.jpberlin.de Betreff: Re: User unknown in local recipient table * Kalle Erlacher <kalle.erlacher(a)doag.org>: > Dec 1 14:38:44 test-mail postfix/smtpd[3366]: NOQUEUE: reject: RCPT > from > localhost[127.0.0.1]: 550 5.1.1 <juergen(a)domain.org>: Recipient > address > rejected: User unknown in local recipient table; > from=<kalle(a)domain.org> to=<juergen @domain.org> proto=ESMTP > helo=<test-mail.domain.org> ... local recipient table = domain ist in mydestination > mydestination = test-mail.domain.org mail. domain.org domain.org > localhost Ich hoffe das Leerzeichen zwischen "mail." und "domain.org" ist nicht wirklich da Explizit ist da nix gesetzt, laut "postconf -n", ergo defaults für local_recipient_maps: local_recipient_maps = proxy:unix:passwd.byname $alias_maps Aber die Domain domain.org soll doch sicher im dovecot landen, oder? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 3

Re: dovecot Digest, Vol 163, Issue 34
by J. Nick Koston 01 Dec '16

01 Dec '16

Aki, Multiple local_names would be ideal to accommodate certificates that have multiple names. The way I’m reading the code it looks like its having to pay for the memory for every name on the certificate because a unique CTX is being created for each name even if they are all on a single certificate. This would be a big memory win for anyone using a certificate with multiple names on it. Thanks -Nick cPanel Inc >> > > Dear Aki et al., > > How straightforward would it be to implement the following or similar syntax: > > local_name foo.tld www.foo.tld mail.foo.tld bar.tld { > ... > } > > ?? > > Thank you! > > -FG > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > dovecot mailing list > dovecot(a)dovecot.org > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot > > ------------------------------ > > End of dovecot Digest, Vol 163, Issue 34 > ****************************************

2 3

use IMAPSIEVE to update database with last_read date
by KT Walrus 30 Nov '16

30 Nov '16

Just noticed the Dovecot support IMAPSIEVE extension… Could I use this extension to update an external database with the date that the user last read the message? My app sends certain “notification” messages to the user’s dovecot mail address. The user reads the messages in their dovecot mailboxes only using IMAP. I want to update my app’s database to record this read time for all “notification” messages sent by the app. Seems to me I could write a short sieve script to send a “notify” message to an app specific address that my app “watches” and updates the appropriate database record with the last_read time. Is this workable for production deploy? Or, is there a better way for a sender to be notified when the recipient actually reads the message? The sender will be my app and the recipient is a dovecot mailbox accessed by IMAP. Kevin

2 3

Sieve script won't compile. Compiler output isn't helpful.
by Steven Mainor 30 Nov '16

30 Nov '16

Hello! I hope you will forgive my ignorance but I have a problem I have been trying to solve for a week and I'm not sure where else to turn. I'm trying to write a sieve script to sort all of my mail. I was writing this script to replace the one I am currently using but I can't seem to get it to compile. I don't know a lot about sieve or really scripting of any kind and I just can't figure out what I'm doing wrong. The compiler just puts out a bunch of output that doesn't make sense. like "unexpected character" when I know the character works there because I am already running a script with some of the same lines. I'm attaching the script with some names and addresses changed and the sieve output to this email. I would greatly appreciate any help anyone could offer.

5 8

Quota Clone not tracking unlimited quota
by Prakash Autade 30 Nov '16

30 Nov '16

We are configuring 'Count quota' with 'Quota Clone Plugin' Everything is working perfectly when we set limit in quota_rule(quota_rule = *:bytes=500M). In case of unlimited quota(quota_rule = *:bytes=0) - Count quota is updating values * - Quota Clone plugin is not updating values in redis.* We have configured dovecot as mail_plugins = $mail_plugins quota quota_clone mail_log notify mailbox_list_index = yes plugin { quota = count:User quota quota_vsizes = yes quota_rule = *:bytes=0 quota_clone_dict = redis:host=127.0.0.1:port=6379 } -- Sincerely, Prakash P. Autade.

1 0

Dovecot seems to open extra namespaces
by Thorsten Hater 30 Nov '16

30 Nov '16

Dear all, I am slightly confused by the following log entries ...: imap(...): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir ...: imap(...): Debug: maildir++: root=~/Maildir, index=, indexpvt=, control=, inbox=~/Maildir, alt= ...: imap(...): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none ...: imap(...): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= It seems that Dovecot opens two different namespaces, despite only one being configured in 10-mail.conf/15-mailboxes.conf. Is this normal and intended? Further, what is the purpose of the line starting with 'none'? Best regards, Thorsten Dovecot version: 2.2.26.0 on Debian Jessie

3 4