- dovecot - dovecot.org

Re: New feature: HTTP API
by aki.tuomi＠dovecot.fi 24 May '16

24 May '16

> On April 5, 2016 at 9:27 AM Tom Sommer <mail(a)tomsommer.dk> wrote: > > > On 2016-04-04 20:22, aki.tuomi(a)dovecot.fi wrote: > > > Since the API is still a little beta, it will not return anything in > > case > > doveadm does not like the arguments (that is, calls i_fatal). Also, > > success is > > often indicated with 200 OK along with [["doveadmResponse",[],"c01"]]. > > I don't see the "doveadm proxy kick" command in the list? Not all commands are included, unfortunately. Also, the list only shows commands that are available with doveadm as per configuration. Aki

4 7

Error: Filename keeps changing
by Ward Romp 24 May '16

24 May '16

Dear all, Two days ago, my ldap server (for authentication) crashed and I had to set up a new one. Everything works fine except the mailbox for one user (let's name him debugger). When I try to access his mail whatever way, dovecot complains about "Error: maildir_file_do(........): Filename keeps changing" For example, a command "doveadm copy -u debugger(a)boontje.net Archive mailbox Netspar" returns doveadm(debugger(a)boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/14230…: Filename keeps changing doveadm(debugger(a)boontje.net): Error: Copying message UID 1 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] doveadm(debugger(a)boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/14472…: Filename keeps changing doveadm(debugger(a)boontje.net): Error: Copying message UID 2 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] doveadm(debugger(a)boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/14472…: Filename keeps changing doveadm(debugger(a)boontje.net): Error: Copying message UID 3 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] doveadm(debugger(a)boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/14498…: Filename keeps changing doveadm(debugger(a)boontje.net): Error: Copying message UID 4 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] It does this on ALL mailboxes of this user and only this user. Other users give no problems. Accessing the mails via a mail client (Roundcube, Thunderbird, etc...) give the same error. Dovecot runs on a virtual machine and the mailboxes are stored via a CIFS-share on a NAS. I rebooted both machines, checked the filesystem (no errors), CIFS-share, directories and mail files are readable and writeable by dovecot user (vmail on the mail server). The mails are intact, cat <filename> shows the full mail. I'm baffled, any suggestions? Kind regards, Ward Romp The configuration etc.... ================================================ # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 4.2.0-36-generic x86_64 Ubuntu 14.04.4 LTS cifs auth_mechanisms = plain login auth_verbose = yes listen = * mail_home = /var/mail/vmail/%d/%n mail_location = maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/conf.d/ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_after = /var/mail/vmail/sieve-after sieve_before = /var/mail/vmail/sieve-before sieve_dir = ~/sieve } protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/private/mail.boontje.net-chain.crt ssl_cipher_list = ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AES:RSA+3DES:!ADH:!AECDH:!MD5:!DSS ssl_client_ca_dir = /etc/ssl/certs ssl_key = </etc/ssl/private/mail.boontje.net.key userdb { args = uid=5000 gid=5000 home=/var/mail/vmail/%d/%n allow_all_users=yes driver = static } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 10 } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = webmaster(a)boontje.net quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r

1 0

Growing number of dovecot processes
by John Oliver 24 May '16

24 May '16

OS is CentOS 6, dovecot-2.0.9-19 Just started using this server as a replacement for an old Debian server running an old dovecot. The number of dovecot processes just grows and grows until there are 100 concurrent login processes (we have maybe 40 or 50 users), and then nobody can log in any more. A reload resets, but the number of stuck processes climbs very quickly. We're authenticating against an OpenLDAP. Unfortunately, this system is not connected to the Internet so I cannot paste logs or output of commands. I've been Googling, but aren't hitting on the right phrase to describe this issue to uncover possible answers :-( If anyone can get me pointed in the right direction, I'd appreciate it. -- *********************************************************************** * John Oliver http://www.john-oliver.net/ * * * ***********************************************************************

2 1

No implicit INBOX with mailbox_list_index = yes?
by Karsten Heiken 23 May '16

23 May '16

Hello all, I am running a Dovecot Cluster (2.2.13 and 2.2.23 show the same behaviour) using dsync replication (as described here: http://wiki.dovecot.org/Replication) In addition to that I allow users to share mailboxes using IMAP ACLs. My problem is that I need to set "mail_shared_explicit_inbox = yes" if I want my users to see Inboxes of other users. If I leave mail_shared_explicit_inbox = no, then the user cannot see any Inboxes of other users: > * LIST (\Subscribed \UnMarked) "/" Sent > * LIST (\Subscribed \UnMarked) "/" Drafts > * LIST (\Subscribed) "/" INBOX When I set explicit_inbox = yes, then I can see the shared mailboxes: > * LIST (\Subscribed \UnMarked) "/" Sent > * LIST (\Subscribed \UnMarked) "/" Drafts > * LIST (\NonExistent) "/" shared > * LIST (\NonExistent) "/" shared/bob(a)luis.uni-hannover.de > * LIST (\Subscribed) "/" shared/bob(a)luis.uni-hannover.de/INBOX > * LIST (\Subscribed) "/" INBOX After some digging I found out that if I set mailbox_list_index = NO, then everything works even without explicit_inbox: > * LIST (\Subscribed) "/" Sent > * LIST (\Subscribed) "/" Drafts > * LIST (\NonExistent) "/" shared > * LIST (\Subscribed) "/" shared/bob(a)luis.uni-hannover.de > * LIST (\Subscribed) "/" INBOX Since we are migrating from Cyrus to Dovecot, our users are used to the behaviour of explicit_inbox = no, so I would like to keep it that way. I would simply leave mailbox_list_index disabled, but the comment states it is required for NOTIFY, which in turn is required by the rolling replication -- am I corrent? > # Mailbox list indexes can be used to optimize IMAP STATUS commands. They are > # also required for IMAP NOTIFY extension to be enabled. > # mailbox_list_index = yes What am I doing wrong or don't understand? Thanks, Karsten -----8<----- # doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-44-pve x86_64 Debian 8.1 default_vsz_limit = 512 M dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } lmtp_rcpt_check_quota = yes mail_gid = 7777 mail_location = maildir:%h/Maildir mail_plugins = " zlib quota acl notify" mail_uid = 7777 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEXPVT=~/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox 30dTrash { auto = subscribe special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=5 acl_shared_dict = proxy::acl quota = maildir:Postfach-Limit quota_grace = 2%% quota_rule = *:storage=8G quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_max_redirects = 25 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster(a)luis.uni-hannover.de protocols = " imap lmtp sieve pop3 sieve" replication_dsync_parameters = -d -l 30 -U -n inbox service auth { unix_listener auth-userdb { group = vmail user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 24245 } } service imap-login { process_min_avail = 16 service_count = 0 } service imap { executable = imap postlogin process_limit = 30000 } service lmtp { inet_listener lmtp { port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } service pop3-login { process_min_avail = 4 service_count = 0 } service pop3 { process_limit = 2500 } service postlogin { executable = script-login -d rawlog /usr/local/bin/dovecot-postlogin.sh } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } ssl_cert = </etc/dovecot/private/fullchain.pem ssl_key = </etc/dovecot/private/privkey.pem syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = home=/var/vmail/%Ld/%Ln quota_rule=*:bytes=8589934592 driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = " zlib quota acl notify sieve quota" } protocol lda { mail_plugins = " zlib quota acl notify sieve" } protocol imap { mail_max_userip_connections = 50 mail_plugins = " zlib quota acl notify imap_quota imap_acl" } protocol pop3 { mail_max_userip_connections = 10 mail_plugins = " zlib quota acl notify" } -- Karsten Heiken Leibniz Universität IT Services Kommunikationssysteme E-Mail, XMPP, Kalender Schloßwender Str. 5 D-30159 Hannover

1 0

solr does not index files in the mailbox
by Andreas Meyer 20 May '16

20 May '16

Hello! I startet solr and configured dovecot to use it. Dovecot just indexes two files dovecot.index.log and dovecot.mailbox.log but nothing in the cur directory. ./post -c myfiles /home/a.meyer/Postfach java -classpath /home/andreas/solr-5.5.1/dist/solr-core-5.5.1.jar -Dauto=yes -Dc=myfiles -Ddata=files -Drecursive=yes org.apache.solr.util.SimplePostTool /home/a.meyer/Postfach SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/myfiles/update... Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log Entering recursive mode, max depth=999, delay=0s Indexing directory /home/a.meyer/Postfach (2 files, depth=0) POSTing file dovecot.index.log (text/plain) to [base]/extract POSTing file dovecot.mailbox.log (text/plain) to [base]/extract Indexing directory /home/a.meyer/Postfach/cur (0 files, depth=1) Indexing directory /home/a.meyer/Postfach/new (0 files, depth=1) Indexing directory /home/a.meyer/Postfach/tmp (0 files, depth=1) 2 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/myfiles/update... Time spent: 0:00:00.751 If I give the cur folder as startingpoint ./post -c myfiles /home/a.meyer/Postfach/cur java -classpath /home/andreas/solr-5.5.1/dist/solr-core-5.5.1.jar -Dauto=yes -Dc=myfiles -Ddata=files -Drecursive=yes org.apache.solr.util.SimplePostTool /home/a.meyer/Postfach/cur SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/myfiles/update... Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log Entering recursive mode, max depth=999, delay=0s Indexing directory /home/a.meyer/Postfach/cur (0 files, depth=0) 0 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/myfiles/update... Time spent: 0:00:00.102 nothing is indexed although the directory contains 4 mails. What am I doing wrong? Greetings Andreas

1 0

Dovecot-FTS:
by Peer Heinlein 20 May '16

20 May '16

We're running Dovecot with SOLR Lucene 5.5 and we have several errors like badMessage: 414 for HttpChannelOverHttp@5216da8e{r=0,c=false,a=IDLE,uri=-} 20.5.2016, 14:10:16 and URI is too large >8192 in our solr-logfile. I don't know if this is a problem in SOLR or in Dovecot, so I just bring it to your attention. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

2 1

dovecot-sieve: require "mailbox" fails (Unbuntu 16.04)
by Karim 'Kasi Mir' Senoucci 20 May '16

20 May '16

Hello everyone, I'm moving some old procmail recipes to sieve, but having problems with it. The problem lies with the mailbox extension, which I'm trying to use as proposed in an example in the Dovecot wiki: > require ["variables","date","fileinto","mailbox"]; > > # Extract date info > if currentdate :matches "year" "*" { set "year" "${1}"; } > if currentdate :matches "month" "*" { set "month" "${1}"; } > > # Archive Dovecot mailing list items by year and month. > # Create folder when it does not exist. > if header :is "list-id" "dovecot.dovecot.org" { > fileinto :create "INBOX.Lists.${year}.${month}.dovecot"; > } I'm trying a similar rule (different condition, different mail folder), but cannot seen to get it to work. I always get the errors: > .DOVECOT: LINE 16: ERROR: UNEXPECTED '}' FOUND AT (THE PRESUMED) END OF FILE. > .DOVECOT: ERROR: PARSE FAILED. I'm using the standard dovecot packages from Ubuntu 16.04: > II DOVECOT-CORE 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - CORE FILES > II DOVECOT-IMAPD 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - IMAP DAEMON > II DOVECOT-MANAGESIEVED 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - MANAGE SIEVE SERVER > II DOVECOT-SIEVE 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - SIEVE FILTERS SUPPORT I was under the impression that these sieve extensions, including "mailbox", are builtin into dovecot/pigeonhole, or at least packaged into dovecot-sieve. Is that not the case? How can I get the example working under Ubuntu linux 16.04? Do I need to add or reconfigure something by hand? Thanks in advance for any help on the subject. Greetings Kasi Mir

1 0

Re: BUG?: Dsync over piped process (DSYNC_RUN_TYPE_CMD): EPIPE if X (done) is received in another write after F (finish)
by J. Nick Koston 20 May '16

20 May '16

Hi Timo, The below solved the problem for me as far as I can tell I was getting a EPIPE about 1:30 dsync runs over the socket. I have received 0 in 10000 runs with this change. https://github.com/dovecot/core/pull/10 <https://github.com/dovecot/core/pull/10> You know the code much better then I do so you might have a better way? Also I have a commit to address the constant warnings when the prefix is set to INBOX. with the quota_quota plugin here: https://github.com/dovecot/core/pull/7 <https://github.com/dovecot/core/pull/7> Thanks -Nick Timo Sirainen tss at iki.fi <mailto:dovecot%40dovecot.org?Subject=Re%3A%20BUG%3F%3A%20Dsync%20over%20piped%20process%20%28DSYNC_RUN_TYPE_CMD%29%3A%20EPIPE%20if%20X%0A%09%28done%29%20is%20received%20in%20another%20write%20after%20F%20%28finish%29&In-Reply-To=%3CA5A2C866-9A65-439B-B232-231202B94525%40iki.fi%3E>Thu May 19 23:15:18 UTC 2016 > > On 19 May 2016, at 03:54, J. Nick Koston <nick at cpanel.net <http://dovecot.org/cgi-bin/mailman/listinfo/dovecot>> wrote: > > > > Hi, > > > > It appears that dsync closes the file handle once it gets the F (finish) response even if it hasn’t received the X (done) response. > > If the X (done) response gets delivered in another write this will generate EPIPE in the connected process. > > > > This is much easier to replicate over SSL, however the trace is provided without SSL since its easier to see whats going on. > > Do you see this as causing some problem? I think this is intentional. The F(inish) tells the final response whether everything was OK or not, so all the work is done after that. The 'X' is described in code as: > > /* notify remote that we're closing. this is mainly to avoid > "read() failed: EOF" errors on failing dsyncs */ > o_stream_nsend_str(ibc->output, > t_strdup_printf("%c\n", items[ITEM_DONE].chr)); > > and on the receiving side: > > if (line[0] == items[ITEM_DONE].chr) { > /* remote cleanly closed the connection, possibly because of > some failure (which it should have logged). we don't want to > log any stream errors anyway after this. */ > dsync_ibc_stream_stop(ibc); > return DSYNC_IBC_RECV_RET_TRYAGAIN; > } > > So I think it only matters in error handling.

1 0

BUG?: Dsync over piped process (DSYNC_RUN_TYPE_CMD): EPIPE if X (done) is received in another write after F (finish)
by J. Nick Koston 19 May '16

19 May '16

Hi, It appears that dsync closes the file handle once it gets the F (finish) response even if it hasn’t received the X (done) response. If the X (done) response gets delivered in another write this will generate EPIPE in the connected process. This is much easier to replicate over SSL, however the trace is provided without SSL since its easier to see whats going on. Thanks J. Nick Koston # dovecot --version 2.2.24 (a82c823) dovecot-2.2.24-1.x86_64 F\n is in packet 1 X\n is in packet 2 STRACE of an the unexpected EPIPE: [pid 34753] read(4, "F\n", 524288) = 2 [pid 34753] write(1, "F\n", 2) = 2 [pid 34752] <... epoll_wait resumed> {{EPOLLIN, {u32=42323904, u64=42323904}}}, 3, 3350) = 1 [pid 34753] select(8, [0 4], NULL, NULL, {21600, 0} <unfinished ...> [pid 34752] read(10, "F\n", 1361) = 2 [pid 34752] write(9, "X\n", 2 <unfinished ...> [pid 34753] <... select resumed> ) = 1 (in [0], left {21599, 999925}) [pid 34752] <... write resumed> ) = 2 [pid 34753] read(0, <unfinished ...> [pid 34752] epoll_ctl(6, EPOLL_CTL_DEL, 10, {0, {u32=42323904, u64=42323904}} <unfinished ...> [pid 34753] <... read resumed> "X\n", 524288) = 2 [pid 34752] <... epoll_ctl resumed> ) = 0 [pid 34753] write(4, "X\n", 2 <unfinished ...> [pid 34752] close(9) = 0 [pid 34753] <... write resumed> ) = 2 [pid 34752] close(10 <unfinished ...> [pid 34753] select(8, [0 4], NULL, NULL, {21600, 0} <unfinished ...> [pid 34752] <... close resumed> ) = 0 [pid 34753] <... select resumed> ) = 2 (in [0 4], left {21599, 999998}) [pid 34752] epoll_wait(6, <unfinished ...> [pid 34753] read(4, "X\n", 524288) = 2 [pid 34753] write(1, "X\n", 2) = -1 EPIPE (Broken pipe) STRACE where its in one packet: F\nX\n is in a single packet [pid 34833] select(8, [0 4], NULL, NULL, {21600, 0}) = 1 (in [4], left {21599, 999966}) [pid 34833] read(4, "F\nX\n", 524288) = 4 [pid 34833] write(1, "F\nX\n", 4) = 4 [pid 34833] select(8, [0 4], NULL, NULL, {21600, 0} <unfinished ...> [pid 34832] <... epoll_wait resumed> {{EPOLLIN, {u32=38678464, u64=38678464}}}, 3, 3889) = 1 [pid 34833] <... select resumed> ) = 1 (in [4], left {21599, 999572}) [pid 34832] read(10, "F\nX\n", 1361) = 4

2 1

mailbox.auto ignores dovecot-uidlist.lock
by Tom Sommer 19 May '16

19 May '16

I'm trying to lock down a maildir from modifications using dovecot-uidlist.lock, but when a user with mailbox.auto = create logs in, then the folder is created regardless of dovecot-uidlist.lock existing or not. Is there no way to prevent dovecot from touching anything in a maildir folder? -- Tom

3 5