- dovecot - dovecot.org

[Dovecot] dovecot-uidlist: Duplicate file entry at line
by Luca Palazzo 07 Sep '10

07 Sep '10

Hi All, we are experiencing another strange problem after upgrading to 2.0.1 from 1.2. Many people's client aree-retrieving mail from server due to some problem with dovecot-uidlist file. Looking at log file seem that all file content is sometime duplicated and appended to the and of same file. It look as another process was accesing and trying to modify uidlist table. Our setup has a load balance (dovecot director) and three backend server. All of them use 2.0.1. MTA is a cluster of postfix that deliver mail to load balancer by LMTP. How can I collect some more information? or, better, solve it? Regards Luca PS: Backend configuration is: # 2.0.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-686-bigmem i686 Debian 5.0.5 auth_master_user_separator = * auth_verbose = yes disable_plaintext_auth = no mail_location = maildir:~/Maildir mail_plugins = $mail_plugins notify mail_log quota managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid from size quota = maildir:User quota quota_warning = storage=95%% quota-warning.sh 95 %u quota_warning2 = storage=80%% quota-warning.sh 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /etc/dovecot/sieve/global/ sieve_global_path = /etc/dovecot/sieve/default.sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot mode = 0600 user = dovecot } } service director { inet_listener { port = 8100 } } service lmtp { inet_listener lmtp { address = 10.194.1.31 127.0.0.1 ::1 port = 24 } } service managesieve-login { inet_listener sieve { address = 10.194.1.31 127.0.0.1 ::1 port = 4190 } process_min_avail = 0 service_count = 1 } service quota-warning { executable = script /virtual/shared/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = $mail_plugins quota sieve } protocol lda { mail_plugins = $mail_plugins quota } protocol imap { mail_plugins = $mail_plugins quota imap_quota } protocol pop3 { mail_plugins = $mail_plugins quota }

4 19

[Dovecot] Dovecot director - silly questions
by Guy 07 Sep '10

07 Sep '10

Hi, I'm currently using Dovecot 1.1.18 with storage over NFS and there's been the occasional corrupt index. I'm going to be installing a new set of servers soon and would like to start using Dovecot 2.0. I've had a look at the bit there is on the new Dovecot director, but I'm not totally sure how it's intended to be used. Should I install Dovecot on the software load balancers (IMAP/POP3 and other services balanced by haproxy) and configure the director on there? I'm also using maildrop at the moment but would like to switch to Dovecot deliver at some stage. Since mail is delivered to a random mail server (they both have access to the NFS shares), won't I run into the same index problem? Should I look at per user transports before then or can director also be used to effect Dovecot deliver? Thanks Guy -- Don't just do something...sit there!

2 1

[Dovecot] cant see public mailbox
by spamvoll＠googlemail.com 07 Sep '10

07 Sep '10

hi.. im running dovecot 2.0.1 sharing mailboxes works like a charm, but now im trying to create a public undetected-SPAM folder. thunderbird shows the public namespace but no mailboxes. ive created the public folder and did a mkdir for the folder.. after that doesnt work ive created cur,new,tmp in .undetected-SPAM /home/MAILBOXES/public# ls -al drwx------ 3 vmail vmail 74 7. Sep 14:23 . drwxr-xr-x 57 vmail vmail 4096 7. Sep 14:23 .. -rw------- 1 vmail vmail 0 7. Sep 14:23 dovecot-acl-list -rwx------ 1 vmail vmail 0 7. Sep 13:19 dovecot-shared drwx------ 5 vmail vmail 36 7. Sep 14:17 .undetected-SPAM doveconf -n mail_location = maildir:~/mail namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:%%h/mail:INDEX=~/mail/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace { list = children location = maildir:/home/MAILBOXES/public:INDEX=~/mail/public prefix = public/ separator = / subscriptions = no type = public } any ideas ? Hans

1 1

[Dovecot] v1.2.14 released
by Timo Sirainen 07 Sep '10

07 Sep '10

http://dovecot.org/releases/1.2/dovecot-1.2.14.tar.gz http://dovecot.org/releases/1.2/dovecot-1.2.14.tar.gz.sig + virtual mailboxes: Added support for IDLE notifications. - master: Don't crash on config reload when using dict processes. - IMAP: QRESYNC parameters for SELECT weren't handled correctly.

2 1

[Dovecot] Plugin execution order and Sieve
by Charly 07 Sep '10

07 Sep '10

Hello, I use dovecot 2.0.1 and pigeonhole Sieve (version of August), I try to create a deliver plugin (lda or lmtp), but if I declare, in addition to my plugin, sieve, sieve is called first, and my plugin is never called. I tried to change the order of declaration of plugins, but nothing changes. If I remove Sieve, my plugin is properly called. protocol lmtp { #mail_plugins = $mail_plugins myplugin sieve ## sieve called first, my plugin is never called #mail_plugins = $mail_plugins sieve myplugin ## sieve called first, my plugin is never called mail_plugins = $mail_plugins myplugin ## my plugin is called } I try to patch Pigeonhole Sieve to call other plugin at the end of execution with : --- dovecot-2-0-pigeonhole-d51650c8af85/src/plugins/lda-sieve/lda-sieve-plugin.c.next_deliver 2010-08-23 19:26:12.000000000 +0200 +++ dovecot-2-0-pigeonhole-d51650c8af85/src/plugins/lda-sieve/lda-sieve-plugin.c 2010-09-02 11:48:22.000000000 +0200 @@ -775,7 +775,8 @@ } T_END; - return ret; + //return ret; + return next_deliver_mail(mdctx,storage_r); } /* With this patch, my plugin is called, but, always AFTER Sieve... How can I call my plugin before Sieve and do I need to patch Pigeonhole Sieve ? Thank you, Charly.

2 2

[Dovecot] LAYOUT=maildir++ under mbox?
by William Blunn 06 Sep '10

06 Sep '10

Whilst documenting LAYOUT=maildir++ under dbox, that got me thinking: Can we specify :LAYOUT=maildir++ with mbox? If I have it right, this should then remove the problem of not being able to have messages and mail subfolders in the same mail folder. So for example if we had mail location specified as: mail_location = mbox:~/mail:LAYOUT=maildir++ we would have files: ~/mail/inbox - file storing messages in INBOX ~/mail/.foo - file storing messages in mail folder "foo" ~/mail/.foo.bar - file storing messages in mail folder "foo/bar" and I think all of Dovecot's indexes would go under ~/mail/.imap Bill

3 8

[Dovecot] extended search
by Daniel-Constantin Mierla 06 Sep '10

06 Sep '10

Hello, is it safe to use arg_new_human_date() from src/lib-storage/mail-search-register-human.c instead of arg_new_date() inside src/lib-storage/mail-search-register-imap.c? The benefit is ability to use interval based search rules, e.g., SINCE 2days. I run it for couple of days and looks fine, used for virtual folders. This kind of search I missed for usage on laptop - I have pretty huge inboxes, thunderbird was over 20% CPU all the time. I switched on using dovecot locally and thunderbird connected to it, my common searches (before: thunderbird's "save as folder" searches) are virtual folders. Everything works great, cpu usage is less than 1% (I have to say that I also moved many filtering rules (for sorting incoming messages to folders) from TB to be done by imapfilter, but the reason was mainly sorting flexibility and rescanning folders). If all is safe, would it be considered for addition in the future version? The patch is trivial, I can leave with patching every release, but I believe is a nice thing to have. Afaik, should not break IMAP SEARCH specs, being an extended version. An alternative would be to add 'X-' commands (e.g., X-SINCE 2days). Thanks, Daniel -- Daniel-Constantin Mierla http://www.asipto.com

2 4

[Dovecot] Problems with zlib
by Andre 06 Sep '10

06 Sep '10

Sometimes I see lines like these in my log: 20100906 11:34:32 imap(username(a)domain.com): Panic: file istream-zlib.c: line 173 (i_stream_zlib_read): assertion failed: (zstream->high_pos == 0) 20100906 11:34:32 imap(username(a)domain.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0x7f328d2244a2] -> /usr/lib/dovecot/libdovecot.so.0 [0x7f328d22450a] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f328d2248b3] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so [0x7f328be444fc] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x6b) [0x7f328d22a10b] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x4d) [0x7f328d22a3fd] -> /usr/lib/dovecot/libdovecot.so.0 [0x7f328d21476e] -> /usr/lib/dovecot/libdovecot.so.0 [0x7f328d214cbe] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_next_block+0x29) [0x7f328d2148c9] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_body+0x23) [0x7f328d214a43] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x7f328d4cb72f] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x7f328d4cb84a] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_special+0x3dc) [0x7f328d4cc99c] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x7f328d4dd86d] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID] [0x41256a] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID](imap_fetch_more+0x16c) [0x412b0c] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID](cmd_fetch+0x3fa) [0x40acea] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID] [0x41031c] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID] [0x4103cd] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID](client_handle_input+0x45) [0x410545] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID](client_input+0x62) [0x410f72] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xcb) [0x7f328d22f3ab] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x18) [0x7f328d22e4c8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f328d21dd63] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID](main+0x281) [0x419201] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x7f328caa81a6] -> dovecot/imap [username(a)domain.com 192.168.4.253 UID] [0x408bd9] Here is the backtrace: [New process 11474] #0 0x00007f328cabbed5 in raise () from /lib/libc.so.6 (gdb) bt #0 0x00007f328cabbed5 in raise () from /lib/libc.so.6 #1 0x00007f328cabd3f3 in abort () from /lib/libc.so.6 #2 0x00007f328d2244b5 in ?? () from /usr/lib/dovecot/libdovecot.so.0 #3 0x00007f328d22450a in ?? () from /usr/lib/dovecot/libdovecot.so.0 #4 0x00007f328d2248b3 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 #5 0x00007f328be444fc in ?? () from /usr/lib/dovecot/modules/lib20_zlib_plugin.so #6 0x00007f328d22a10b in i_stream_read () from /usr/lib/dovecot/libdovecot.so.0 #7 0x00007f328d22a3fd in i_stream_read_data () from /usr/lib/dovecot/libdovecot.so.0 #8 0x00007f328d21476e in ?? () from /usr/lib/dovecot/libdovecot.so.0 #9 0x00007f328d214cbe in ?? () from /usr/lib/dovecot/libdovecot.so.0 #10 0x00007f328d2148c9 in message_parser_parse_next_block () from /usr/lib/dovecot/libdovecot.so.0 #11 0x00007f328d214a43 in message_parser_parse_body () from /usr/lib/dovecot/libdovecot.so.0 #12 0x00007f328d4cb72f in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 #13 0x00007f328d4cb84a in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 #14 0x00007f328d4cc99c in index_mail_get_special () from /usr/lib/dovecot/libdovecot-storage.so.0 #15 0x00007f328d4dd86d in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 #16 0x000000000041256a in ?? () #17 0x0000000000412b0c in imap_fetch_more () #18 0x000000000040acea in cmd_fetch () #19 0x000000000041031c in ?? () #20 0x00000000004103cd in ?? () #21 0x0000000000410545 in client_handle_input () #22 0x0000000000410f72 in client_input () #23 0x00007f328d22f3ab in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 #24 0x00007f328d22e4c8 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 #25 0x00007f328d21dd63 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 #26 0x0000000000419201 in main () here is the relevant plugin section: plugin { zlib_save = gz zlib_save_level = 6 } What can I do to investigate deeper? TIA Andre

2 1

Re: [Dovecot] dovecot Digest, Vol 89, Issue 27
by Alan Brown 06 Sep '10

06 Sep '10

> > Timo Sirainen <tss(a)iki.fi> Wrote: > > > > > On Mon, 2010-09-06 at 14:26 +0100, Alan Brown wrote: > > > > Is there any way of enforcing a Maildir per-folder message limit in > > > > Dovecot? > > > > > No. > > > > Would you consider it as a feature request? > What if you just create a script that automatically archives older mails to <box>/archive/yyyy-mm mailboxes when number of messages in a mailbox increases too high for too long? Or something like that. I'm looking at doing that anyway, however it would be much easier to train the users if the server says "folder full"

1 0

Re: [Dovecot] dovecot Digest, Vol 89, Issue 25
by Alan Brown 06 Sep '10

06 Sep '10

Timo Sirainen <tss(a)iki.fi> Wrote: > On Mon, 2010-09-06 at 14:26 +0100, Alan Brown wrote: > > Is there any way of enforcing a Maildir per-folder message limit in > > Dovecot? > No. Would you consider it as a feature request? > > We're finding major performance hits once a threshold of files/directory > > is exceeded. This applies across all common filesystems with the > > threshold differing depending on the FS. GFS2 pretty much _STOPS_ for 10 > > minutes when users put 65k+ messages in one folder... > mdbox would probably work nicely. :) Yes, it probably would, however I have several layers of resistance to get past in order to implement that (ie, up to 2 years to get approval to switch to dbox) > > I'm currently running 1.1.20 but looking to update (RHEL5 cluster > > installation) > With v1.2+ you could also try maildir_very_dirty_syncs=yes That won't help much for initial directory opens, which is where most of the pain is. Past that point, the index of ~/Maildir/.folder/cur is mostly cached until you hit about 16,000 messages(files) on GFS2 and some people are insisting on 32k+ files in one folder with the odd few trying 65k or more (I'm surprised their clients don't break). For info sake: we have Dovecot running on a pair of dedicated dual E5530 (quad-core X86 1300MHZ FSB cpu) servers (RH cluster+GFS2) with 24Gb RAM apiece and ~2TB of 8Gb/s san-attached storage available. There are only ~250 accounts but there's around 600Gb in the maildir folder areas and another 300Gb in a separate dedicated inbox area - and I believe there's another 1TB of stuff in client local folders we're trying to pull back to the servers in order to simplify backups. Even with this kind of horsepower available, over-large folders can render the setup completely unresponsive for minutes at a time. I know this is a case of "work smarter, not harder", but I'm facing major delays until we're able to change the on-disk storage format. Alan

2 1