- dovecot - dovecot.org

[Dovecot] Strange Dovecot 2.0.20 auth chokes and cores
by Alan Brown 02 Jun '12

02 Jun '12

> From: "Konrad ."<len7hir(a)gmail.com> > When we upgrade our kernels from 2.6.32.2 to 3.2.16 something strange > has happened. > On high traffic dovecot/auth looks like not responding. > > We found a lot of this lines at the log: [snip] > We compile Dovecot with poll instead of epoll (--with-ioloop=poll) and > this works for us. > > > Is any problem with epoll on 3.2.x kernels? Yes - and it's been discussed here. Some "bright spark" rewrote the kernel epoll code to prevent DoS attacks caused by "excessive forking". That "bright spark" clearly doesn't use imap or webserver software. Redhat backported this change to kernels released after February 2012 and are about to release an emergency kernel release which removes it. A number of $LARGE companies were "not happy" As you've posted, compiling without epoll fixes the problem, but that doesn't help anyone using distro or repo packages.

3 2

[Dovecot] interesting stats pattern
by Cor Bosman 02 Jun '12

02 Jun '12

Hey all, im experimenting with dovecot stats service, and graphing the result. My initial results are kind of interesting. Check out this graph showing connected sessions and users: http://grab.by/dReu At first I thought maybe one of our 35 imap servers was having issues sending data, but all individual servers show this patters. Here is a bunch of individual servers: http://grab.by/dReC Anyone have any idea what could cause such a pattern? Maybe dovecot does some cleaning up of idle sessions at specific intervals? Or maybe our loadbalancers do, or the imapdirectors. regards, Cor

5 7

[Dovecot] auth trouble
by Glenn English 02 Jun '12

02 Jun '12

Debian Lenny, Dovecot v 1.0.15. I'm getting a lot of what I think is a local socket asking dovecot:auth to verify username/passwords: > May 31 09:00:54 server dovecot-auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=admin rhost= Note the empty 'rhost='. That's why I think it's on the server. I see others that look like bots: > May 30 23:08:43 server dovecot-auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=admin rhost=200.119.139.22 And I know how to promote the latter to a firewall. But with no rhost, I'm stumped... I've read books, googled, read docs, and asked for help on other mailing lists, and I've learned a lot. And I no longer think it really has much to do with Dovecot, other than the login attempt going through it to get to PAM. But has anyone here seen this before? Is my current theory correct? What did you do to make it go away? (I suspect that upgrading to Debian Squeeze might get rid of it, but I'm afraid that if I don't figure out what's going on, it might just come back.) -- Glenn English hand-wrapped from my Apple Mail

1 1

[Dovecot] Inconsistent search results and crash on force-resync
by Joe Beaubien 01 Jun '12

01 Jun '12

Hi, I am seeing inconsistencies in search results (finding 2 emails when only 1 exists, finding the email when it has been moved to another folder, etc). I figured I should run force-resync to fix any issues. I ran the following: doveadm -v force-resync -u <user> <mailbox> and I got some worrysome logs. - I should mention that I have been seeing some crashes of fts-lucene in my logs. I sent a traceback of this on the mailing list 1-2 days ago under the subject "[Dovecot] fts_lucene crashing". - I should also mention that all the problems I am having are only in 1 email account. This email account contains folders of over 100k emails. Do I need to tweak dovecot somehow for this? Up until now all I did was change vsz_limit to 1024 MB for "service imap". Here are the logs: Jun 1 11:15:01 XXXXX dovecot: imap(form): Error: Recent flags state corrupted for mailbox INBOX2 Jun 1 11:15:01 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 XXXXX dovecot: imap(form): Error: Recent flags state corrupted for mailbox contrat Jun 1 11:15:01 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/contrat/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/rep_Immigation soi-m&AOo-me/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/Templates/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: Recent flags state corrupted for mailbox rep_eval_positive Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/rep_eval_positive/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: Recent flags state corrupted for mailbox Sent Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/Sent/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: Recent flags state corrupted for mailbox form_positif Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/Archives/contrat/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:03 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/Archives/form_indetermine/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:03 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/Archives/form_indetermine/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:15 XXXXX dovecot: imap(form): Error: Recent flags state corrupted for mailbox form_indetermine Jun 1 11:15:15 XXXXX dovecot: imap(form): Error: /data/emails/form/mailboxes/form_indetermine/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:25 XXXXX dovecot: indexer-worker: Error: indexer-worker: /home/jd/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:210: std::string lucene::index::DocumentsWriter::closeDocStore(): Assertion `numDocsInStore*8 == directory->fileLength( (docStoreSegment + "." + IndexFileNames::FIELDS_INDEX_EXTENSION).c_str() )' failed. Jun 1 11:15:25 XXXXX dovecot: indexer: Error: Indexer worker disconnected, discarding 28 requests for form Jun 1 11:15:25 XXXXX dovecot: indexer-worker(form): Fatal: master: service(indexer-worker): child 9909 killed with signal 6 (core not dumped) I have 3 questions: 1) When the log says "/mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is now inconsistent" should I be worried, or this will fix itself? 2) Should I expect to see "Error: Recent flags state corrupted for mailbox Sent"??? I ran the force-resync 3 times and I still see this message. 3) Any idea why clucene is crashing? Regards, -Joe

1 0

[Dovecot] inet_listener imaps { port = 0 } question
by henrixd 01 Jun '12

01 Jun '12

Hi Why commenting out "inet_listener imaps {}" won't stop dovecot to listen port 993? I think this would be expected behavior. Just curious, finally got it working with port = 0. :) - Henri

2 1

Re: [Dovecot] High level of pop3 popping causing server to become unresponsive
by Root Kev 30 May '12

30 May '12

Missed CCing on last reply. See below.. Also, would having Nagios checking the number of messages in a mailbox cause issues with the popping of messages? And/Or would having a user accessing the mailbox from two different applications cause issues? Ie. from Outlook and mobile device? I am under the impression that a lock file is generated which should deal with the issue of contention, deleting mail, etc. Thanks for any information. Kevin Sorry for the delay in responding, long weekend in Canada... > > When trying to SSH into the server, the server prompts for user name then > password, then just hangs. Same if there is already a console connection > over, when trying to SU to root, it just hangs after entering the password. > > Our passwords are in the shadow file, and because this is a server > dedicated for this task, there is only the default linux users, and maybe 8 > other user accounts in the shadow file. > > There shouldn't be high IO as all this box does is postfix, pop3ad and > dovecot. This box is seeing less then 3000 emails a day, and only has 5 > mailboxes on it. > > Thanks for the continued suggestions... > > Kevin > > > On Sat, May 19, 2012 at 3:33 PM, Timo Sirainen <tss(a)iki.fi> wrote: > >> On Fri, 2012-05-18 at 09:21 -0400, Root Kev wrote: >> > During the last time that the load went up, it became unable to login / >> su >> > to root for the entire period that dovecot was running, we had to kill >> > dovecot and go back to Popa3d until the mailq was cleared up. We are >> > running CentOS 5.6 server. Based on TOP running at the time the CPU >> usage >> > was running under 10%. Once Dovecot was killed, we were then able to >> log >> > in /su again. >> >> Like Kelsey said, a very high disk IO might explain this, although >> normally the login should still eventually succeed. Another thing I'm >> wondering is if some process limit reached. How does the login/su fail, >> does it just hang or immediately fail with some error? >> >> > We were under the impression that checking to shadow directly should be >> the >> > fastest and least amount of overhead, is any of the other ways to >> connect >> > have less load on authentication to PAM? >> >> Your passwords really are in /etc/shadow file, not LDAP/something else? >> I don't think the problem is with authentication. Reading /etc/shadow is >> pretty fast (unless maybe if it's a huge file) and it anyway can't block >> login/su from working. >> >> >> >

5 8

Re: [Dovecot] Dovecot 2.1 mbox + maildir
by Alan Brown 30 May '12

30 May '12

> From: Timo Sirainen<tss(a)iki.fi> > On Mon, 2012-05-28 at 14:50 +0100, Alan Brown wrote: >> > What syntax is needed to make this work? >> > >> > The 2.0 wiki recomendations don't work - I can see the inboxes or the >> > folders but not both at once and there are lots of error messages about >> > prefix clashes if I simply use the existing 2.0.20 conf file on 2.1.6 > Are you saying that it works in v2.0 but not in v2.1? yes. > Then something's > wrong. Show your doveconf -n output and what error messages you see. Attached to this email (dovecot 2.0 and 2.1 -n outputs) In both cases, we use the same local config file, included from dovecot.conf (I've included both of these too) The error message is: May 30 17:00:31 msslat dovecot: imap(foobar): Error: user foobar: Initialization failed: namespace configuration error: Duplicate namespace prefix: "" May 30 17:00:31 msslat dovecot: imap(foobar): Error: Invalid user settings. Refer to server log for more information. May 30 17:00:31 msslat dovecot: master: Warning: Killed with signal 15 (by pid=18489 uid=0 code=kill) Attempting to tweak MAIL.conf to fix this error simply resulted in various states of "broken", so any help untangling this will be greatly appreciated.

2 1

[Dovecot] Strange Dovecot 2.0.20 auth chokes and cores
by Konrad . 30 May '12

30 May '12

When we upgrade our kernels from 2.6.32.2 to 3.2.16 something strange has happened. On high traffic dovecot/auth looks like not responding. We found a lot of this lines at the log: dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable (...) and clients stop authorizing Some other errors follow in the wake of: dovecot: pop3: Error: Raw backtrace: /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x373ca) [0x7768a3ca] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x3743b) [0x7768a43b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7766048b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x4593a) [0x7769893a] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_add+0xaf) [0x7769757f] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(master_service_init_finish+0x19a) [0x77683c2a] -> dovecot/pop3(main+0xfc) [0x804a90c] -> /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x774c04d3] -> dovecot/pop3() [0x804aba9] dovecot: pop3: Error: Raw backtrace: /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x373ca) [0x7768a3ca] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x3743b) [0x7768a43b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7766048b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x4593a) [0x7769893a] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_add+0xaf) [0x7769757f] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(master_service_init_finish+0x19a) [0x77683c2a] -> dovecot/pop3(main+0xfc) [0x804a90c] -> /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x774c04d3] -> dovecot/pop3() [0x804aba9] dovecot: master: Error: service(pop3): child 18756 killed with signal 6 (core dumped) dovecot: master: Error: service(pop3): child 18756 killed with signal 6 (core dumped) dovecot: master: Error: service(pop3): command startup failed, throttling dovecot: master: Error: service(pop3): command startup failed, throttling dovecot: pop3-login: Error: Raw backtrace: /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x373ca) [0x776b73ca] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x3743b) [0x776b743b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7768d48b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x4593a) [0x776c593a] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_add+0xaf) [0x776c457f] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(master_service_init_finish+0x19a) [0x776b0c2a] -> /opt/dovecot2/lib/dovecot/libdovecot-login.so.0(main+0x143) [0x77705383] -> /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x774ed4d3] -> dovecot/pop3-login() [0x8049471] dovecot: pop3-login: Error: Raw backtrace: /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x373ca) [0x776fd3ca] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x3743b) [0x776fd43b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x776d348b] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x4593a) [0x7770b93a] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(io_add+0xaf) [0x7770a57f] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(master_service_init_finish+0x19a) [0x776f6c2a] -> /opt/dovecot2/lib/dovecot/libdovecot-login.so.0(main+0x143) [0x7774b383] -> /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x775334d3] -> dovecot/pop3-login() [0x8049471] And example stack trace (from pop3, pop3-login throws almost the same): #0 0x776f6424 in __kernel_vsyscall () No symbol table info available. #1 0x7744d1ef in __GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 resultvar = <optimized out> resultvar = <optimized out> pid = 2002518004 selftid = 25476 #2 0x77450835 in __GI_abort () at abort.c:91 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x9bce4a8, sa_sigaction = 0x9bce4a8}, sa_mask = {__val = {163409408, 2002781570, 163374248, 603, 163374280, 604, 163374280, 2001703379, 0, 2002790760, 2140717252, 163374280, 0, 2003786736, 2002596704, 0, 2002618953, 2003087348, 2140717196, 0, 163409408, 2001286473, 163374248, 10, 2000834616, 2002534400, 604, 2003087348, 604, 2002791863, 4294967295, 10}}, sa_flags = 2140717316, sa_restorer = 0x7764bd84 <pid.3786>} sigs = {__val = {32, 0 <repeats 31 times>}} #3 0x77603390 in default_fatal_finish (type=<optimized out>, status=<optimized out>) at failures.c:187 backtrace = 0x9bce098 "/opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x3837a) [0x7760337a] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(+0x383eb) [0x776033eb] -> /opt/dovecot2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x775d877a"... #4 0x776033eb in i_internal_fatal_handler (ctx=0x7f98c174, format=0x9bd6db0 "(25476) epoll_ctl(%s, %d) failed: %m", args=0x7f98c194 "\\\323bw\006") at failures.c:688 status = 0 #5 0x775d877a in i_panic (format=0x7762d364 "epoll_ctl(%s, %d) failed: %m") at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = 0x7f98c194 "\\\323bw\006" f = <optimized out> #6 0x77611d8a in io_loop_handle_add (io=0x9bd6d78) at ioloop-epoll.c:119 ---Type <return> to continue, or q <return> to quit--- ctx = 0x9bd64d8 list = <optimized out> event = {events = 24, data = {ptr = 0x9bd6da0, fd = 163409312, u32 = 163409312, u64 = 163409312}} op = 1 first = true #7 0x776109af in io_add (fd=6, condition=IO_ERROR, callback=0x775fd030 <master_status_error>, context=0x9bd6300) at ioloop.c:54 io = <optimized out> __FUNCTION__ = "io_add" #8 0x775fca9a in master_service_init_finish (service=0x9bd6300) at master-service.c:381 sigint_flags = <optimized out> st = {st_dev = 8, __pad1 = 0, __st_ino = 1286567453, st_mode = 4480, st_nlink = 1, st_uid = 0, st_gid = 0, st_rdev = 0, __pad2 = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1338389522, tv_nsec = 269436805}, st_mtim = {tv_sec = 1338389522, tv_nsec = 269436805}, st_ctim = { tv_sec = 1338389522, tv_nsec = 269436805}, st_ino = 1286567453} value = <optimized out> count = 1 __FUNCTION__ = "master_service_init_finish" #9 0x0804a954 in main (argc=1, argv=0x7f98c414) at main.c:232 set_roots = {0x804ee40, 0x0} service_flags = <optimized out> storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT postlogin_socket_path = 0x0 username = 0x0 c = <optimized out> We do some investigation. Restart helped for a moment, but problem was returning. When we back to older kernel - everything back to normal. We compile Dovecot with poll instead of epoll (--with-ioloop=poll) and this works for us. Is any problem with epoll on 3.2.x kernels? Or maybe this is Dovecot problem? Maybe this is not connected with epoll, but epoll is interferes with this. Problem on Dovecot 2.0.11 and 2.0.20. -- Len7hir

2 1

[Dovecot] i can't make DOTS before mailbox names
by John Goodner 30 May '12

30 May '12

hello i have a real stumper of an issue here and need some help! (config posted below) ok, so i'm building out a new mail system with v1.2.15 of dovecot on debian6 this new system is being designed to seamlessly migrate from an older dovecot mail system (v1.1.3). i have everything almost perfect but i've hit a wall which I need help on. so the existing mailboxes on the old server are as such: cur dovecot.index dovecot.index.cache dovecot.index.log dovecot-uidlist dovecot-uidvalidity dovecot-uidvalidity.4fbffa1a .INBOX.Drafts .INBOX.Sent .INBOX.Trash new subscriptions tmp .Trash NOTICE that it puts the . in before INBOX now on the new mailserver I get this: cur dovecot.index.cache dovecot.index.log dovecot-uidlist dovecot-uidvalidity dovecot-uidvalidity.4fc03e57 Drafts INBOX new Sent subscriptions tmp Trash Notice how it's no longer using DOTS before the names. so what i need to do is make the new mail server DUPLICATE EXACTLY the folder layouts of the old server. i've tried fussing with Namespace and the MailDir and especially the PREFIX option which is seems to ignore a '.' to start a filename. i also fussed with the autocreate hoping i could find something there to adjust. but no good. no matter what i do I can't get it to make or read .Trash .Sent .Drafts! please help me! any advice would be appreciated. i'm also willing to hire someone to finish up the config for me. shouldn't take more than a couple hours. thanks! jg here's my config: root@mailserver1:/# dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.5 nfs log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_location: maildir:/var/vmail/%d/%n@%d:INBOX=/var/vmail/%d/%n@%d:LAYOUT=fs mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 namespace: type: private separator: . prefix: .INBOX. inbox: yes list: yes subscriptions: yes lda: log_path: /var/log/dovecot-deliver.log auth_socket_path: /var/run/dovecot/auth-master postmaster_address: postmaster(a)m.minla.net mail_plugins: sieve global_script_path: /home/vmail/globalsieverc auth default: passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=1010 gid=1010 home=/home/vmail allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail

2 1

[Dovecot] Dovecot-lda - 100% cpu
by For＠ll 30 May '12

30 May '12

Hello, On my MTA server where I have installed postfix+mysql+dovecot(2.0.19), when server receives e.g 1000 e-mails and postfix start 10 threads dovecot-lda then my load is growing up. Disk for e-mails is connected through with iSCSI protocol. Any idea why this is happening?

1 0