[From nobody Mon Jul 2 17:26:43 2007 Return-Path: <mailman-bounces@syksy.dovecot.org> X-Original-To: cras@irccrew.org Delivered-To: cras@irccrew.org Received: from ainavaan.iki.fi (ainavaan.iki.fi [212.16.98.51]) by shodan.irccrew.org (Postfix) with ESMTP id 825C5445C for <cras@irccrew.org>; Mon, 2 Jul 2007 12:35:24 +0300 (EEST) Received: from dovecot.org (dovecot.org [82.118.211.50]) by ainavaan.iki.fi (8.13.8/8.13.8) with ESMTP id l629ZSP7009204 for <tss@iki.fi>; Mon, 2 Jul 2007 12:35:28 +0300 (EEST) Received: from syksy.dovecot.org (localhost.localdomain [127.0.0.1]) by dovecot.org (Postfix) with ESMTP id 4C5E1FA892C; Mon, 2 Jul 2007 12:35:24 +0300 (EEST) X-Original-To: dovecot-owner@dovecot.org Delivered-To: dovecot-owner@dovecot.org Received: from syksy.dovecot.org (localhost.localdomain [127.0.0.1]) by dovecot.org (Postfix) with ESMTP id 44699FA8965 for <dovecot-owner@dovecot.org>; Mon, 2 Jul 2007 12:35:23 +0300 (EEST) Subject: Bounce action notification From: mailman@dovecot.org To: dovecot-owner@dovecot.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0513239137==" Message-ID: <mailman.88.1183368922.7411.dovecot@dovecot.org> Date: Mon, 02 Jul 2007 12:35:22 +0300 Precedence: bulk X-BeenThere: dovecot@dovecot.org X-Mailman-Version: 2.1.9 List-Id: Dovecot Mailing List <dovecot.dovecot.org> X-List-Administrivia: yes Sender: mailman-bounces@syksy.dovecot.org Errors-To: mailman-bounces@syksy.dovecot.org --===============0513239137== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a Mailman mailing list bounce action notice: List: dovecot Member: tss@iki.fi Action: Subscription disabled. Reason: Excessive or fatal bounces. =20 The triggering bounce notice is attached below. Questions? Contact the Mailman site administrator at mailman@dovecot.org. --===============0513239137== Content-Type: message/rfc822 MIME-Version: 1.0 Return-Path: <postmaster@dovecot.org> X-Original-To: dovecot-bounces@dovecot.org Delivered-To: dovecot-bounces@dovecot.org Received: by dovecot.org (Postfix, from userid 505) id 7AEFDFA892C; Mon, 2 Jul 2007 12:24:23 +0300 (EEST) Received: from postmaster?dovecot.org (unknown [81.3.115.182]) by dovecot.org (Postfix) with ESMTP id 636FFFA892C for <dovecot-bounces@dovecot.org>; Mon, 2 Jul 2007 09:24:21 +0000 (UTC) From: "postmaster@dovecot.org" <postmaster@dovecot.org> Subject: Failure Delivery To: dovecot-bounces@dovecot.org Date: Mon, 2 Jul 2007 10:24:10 +0000 Message-Id: <20070702092421.636FFFA892C@dovecot.org> Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Unable to deliver message to the following address(es) dovecot@dovecot.org Remote host said: 554 delivery error: This user doesn't have an account --- Original message follows. On Wed, 27 Jun 2007 23:15:32 +0300 Timo Sirainen <tss@iki.fi> wrote: > On Thu, 2007-06-21 at 16:49 +0900, Christian Balzer wrote: > > > You could try > > > http://dovecot.org/patches/debug/mempool-accounting.diff and send > > > USR1 signal to dovecot-auth after a while. It logs how much memory > > > is used by all existing memory pools. Each auth request has its own > > > pool, so if it's really leaking them it's probably logging a lot of > > > lines. If not, then the leak is elsewhere. > > >=20 > > I grabbed the Debian package source on a test machine (not gonna chance > > anything on the production servers), applied the patch, did add > > --enable-debug to the debian/rules file (and got the #define DEBUG=20 > > in config.h), created the binary packages, installed, configured, > > started them, tested a few logins and... nothing gets logged=20 > > in mail.* if I send a USR1 to dovecot-auth. Anything I'm missing? >=20 > Bug, fixed: http://hg.dovecot.org/dovecot-1.0/rev/a098e94cd318 >=20 Thanks, that fixed the silence of the auth-sheep. This is the output after start-up: --- Jul 2 13:59:54 engtest03 dovecot: auth(default): pool auth request handler= : 104 / 4080 bytes Jul 2 13:59:54 engtest03 last message repeated 19 times Jul 2 13:59:54 engtest03 dovecot: auth(default): pool passwd_file: 56 / 10= 224 bytes Jul 2 13:59:54 engtest03 dovecot: auth(default): pool Environment: 224 / 2= 032 bytes Jul 2 13:59:54 engtest03 dovecot: auth(default): pool ldap_connection: 576= / 1008 bytes Jul 2 13:59:54 engtest03 dovecot: auth(default): pool auth: 1520 / 2032 by= tes --- Used memory of dovecot-auth after 1 login was 3148KB(RSS). This is after a good trashing with rabid (from the postal package), with just 2 users though, using POP3 logins: --- Jul 2 14:12:30 engtest03 dovecot: auth(default): pool auth request handler= : 104 / 4080 bytes Jul 2 14:12:30 engtest03 last message repeated 128 times Jul 2 14:12:30 engtest03 dovecot: auth(default): pool passwd_file: 56 / 10= 224 bytes Jul 2 14:12:30 engtest03 dovecot: auth(default): pool Environment: 224 / 2= 032 bytes Jul 2 14:12:30 engtest03 dovecot: auth(default): pool ldap_connection: 576= / 1008 bytes Jul 2 14:12:30 engtest03 dovecot: auth(default): pool auth: 1520 / 2032 by= tes --- Note that the amount of auth request handler pools have grown to 128.=20 After another short round of rabid the handler pools grew to 137 and the size of dovecot-auth to 5100KB. The number of handler pools never fell, nor did the memory footprint, obviously. :-p At about 800k logins/day/node here it's obvious now why dovecot-auth explodes after less than a week with max size of 512MB.=20 > > But no matter, it is clearly leaking just as bad as 0.99 and I venture > > that his is the largest installation with LDAP as authentication > > backend. I wonder if this leak would be avoided by having LDAP lookups > > performed by worker processes as with SQL.=20 >=20 > Then you'd only have multiple leaking worker processes. > Yes, I realize that. But I presume since these are designed to die off and be recreated on the fly the repercussions would be much better. ;) Of course now it looks like this is not LDAP related after all. > > > The same as 0.99. You could also kill -HUP dovecot when dovecot-auth > > > is nearing the limit. That makes it a bit nicer, although not > > > perfectly safe either (should fix this some day..). > > > > > If that leak can't be found I would very much appreciate a solution > > that at least avoids failed and/or delayed logins. >=20 > That would require that login processes don't fail logins if connection > to dovecot-auth drops, but instead wait until they can connect back to > it and try again. And maybe another alternative would be to just > disconnect the client instead of giving login failure. >=20 Anything that fixes this one way or the other would be nice. ^_^ Oh and HUP'ing the master is not an option here, I guess the system load triggers a race condition in dovecot because several times when doing so I got this: --- Jun 22 15:08:58 mb11 dovecot: listen(143) failed: Interrupted system call --- Which results in a killed off dovecot, including all active sessions. The self terminating dovecot-auth is not nice, but at least more predictable and does recover by itself: --- Jun 30 19:03:27 mb12 dovecot: auth(default): pool_system_malloc(): Out of m= emory Jun 30 19:03:27 mb12 dovecot: child 11110 (auth) returned error 83 (Out of = memory) Jun 30 19:03:28 mb12 dovecot: pop3-login: Can't connect to auth server at d= efault: Resource temporarily unavailable Jun 30 19:03:28 mb12 last message repeated 11 times --- Of course the 12 users that tried to log in at this time are probably not amused or at least confused. Regards, Christian --=20 Christian Balzer Network/Systems Engineer NOC chibi@gol.com Global OnLine Japan/Fusion Network Services http://www.gol.com/ --===============0513239137==-- ]