- dovecot - dovecot.org

[Dovecot] OT: Counting emails in IMAP folders
by Nigel Allen 17 Nov '08

17 Nov '08

Greetings I have a customer who has mail server running dovecot with an IMAP account (Customers) which contains hundreds of folders (one per customer). Looks like this: Customers Fred Smith and Co Joe Blogss Cust-1 Cust-n They have a requirement to report monthly on the total number of emails added to each customers' folder during the month. Basically I need to be able to run some kind of command line query that will give me each of the sub-folder names and the number of emails added to the folder for that month (or other user-specified time period). Any ideas please? TIA Nigel.

3 3

Re: [Dovecot] antispam plugin claims "antispam signature not found"
by Guillaume HILT 15 Nov '08

15 Nov '08

I have the same problem and i'm wondering how should i use this patch ? I'm not familiar with the patch command. Thanks. PS: Sorry for the empty mail Johannes Berg a écrit > On Tue, 2008-11-11 at 11:42 +0100, Johannes Berg wrote: > > >> Not sure, I personally think that moving them silently makes errors >> harder to diagnose. I'll add an option since then you can set this >> option (antispam_missing_signature = [error|move]) to move after you've >> tested and verified that it works just fine. >> > > Try this patch please, I have only compiled it so far. (there might be > rejects in the signature-log.c file unless you update to the current git > tree but you don't care about that) > > diff --git a/antispam.7 b/antispam.7 > index d07c1d0..e26db69 100644 > --- a/antispam.7 > +++ b/antispam.7 > @@ -108,6 +108,13 @@ plugin { > # mail signature (used with any backend requiring a signature) > antispam_signature = X-DSPAM-Signature > > + # action to take on mails without signature > + # (used with any backend requiring a signature) > + # (we recommend only setting this to 'move' after verifying that the > + # whole setup is working) > + # antispam_signature_missing = move # move silently without training > + antispam_signature_missing = error > + > # semicolon-separated list of Trash folders (default unset i.e. none) > # antispam_trash = > # antispam_trash = trash;Trash;Deleted Items > diff --git a/signature-log.c b/signature-log.c > index ab8cd6d..5b7b306 100644 > --- a/signature-log.c > +++ b/signature-log.c > @@ -99,7 +99,12 @@ int backend_handle_mail(struct mailbox_transaction_context *t, > return -1; > } > > - signature = signature_extract(t, mail); > + ret = signature_extract(t, mail, &signature); > + if (ret) > + return ret; > + > + if (!signature) > + return 0; > > switch (wanted) { > case CLASS_SPAM: > diff --git a/signature.c b/signature.c > index 7f9d83b..7ad29d6 100644 > --- a/signature.c > +++ b/signature.c > @@ -5,6 +5,7 @@ > #include "mail-storage-private.h" > > const char *signature_hdr = "X-DSPAM-Signature"; > +static int signature_nosig_ignore = 0; > > void signature_init(void) > { > @@ -12,6 +13,16 @@ void signature_init(void) > if (tmp) > signature_hdr = tmp; > debug("signature header line is \"%s\"\n", signature_hdr); > + > + tmp = get_setting("SIGNATURE_MISSING"); > + if (!tmp) > + tmp = "error"; > + if (strcmp(tmp, "move") == 0) { > + signature_nosig_ignore = 1; > + debug("will silently move mails with missing signature\n"); > + } else if (strcmp(tmp, "error") != 0) { > + debug("invalid signature_missing setting '%s', ignoring\n", tmp); > + } > } > > int signature_extract_to_list(struct mailbox_transaction_context *t, > @@ -23,10 +34,14 @@ int signature_extract_to_list(struct mailbox_transaction_context *t, > > signatures = get_mail_headers(mail, signature_hdr); > if (!signatures || !signatures[0]) { > - mail_storage_set_error(t->box->storage, > - ME(NOTPOSSIBLE) > - "antispam signature not found"); > - return -1; > + if (!signature_nosig_ignore) { > + mail_storage_set_error(t->box->storage, > + ME(NOTPOSSIBLE) > + "antispam signature not found"); > + return -1; > + } else { > + return 0; > + } > } > > while (signatures[1]) > @@ -42,23 +57,30 @@ int signature_extract_to_list(struct mailbox_transaction_context *t, > return 0; > } > > -const char *signature_extract(struct mailbox_transaction_context *t, > - struct mail *mail) > +int signature_extract(struct mailbox_transaction_context *t, > + struct mail *mail, const char **signature) > { > const char *const *signatures; > > signatures = get_mail_headers(mail, signature_hdr); > if (!signatures || !signatures[0]) { > - mail_storage_set_error(t->box->storage, > - ME(NOTPOSSIBLE) > - "antispam signature not found"); > - return NULL; > + if (!signature_nosig_ignore) { > + mail_storage_set_error(t->box->storage, > + ME(NOTPOSSIBLE) > + "antispam signature not found"); > + return -1; > + } else { > + *signature = NULL; > + return 0; > + } > } > > while (signatures[1]) > signatures++; > > - return signatures[0]; > + *signature = signatures[0]; > + > + return 0; > } > > void signature_list_free(struct siglist **list) > diff --git a/signature.h b/signature.h > index e25d1b5..1384924 100644 > --- a/signature.h > +++ b/signature.h > @@ -16,8 +16,8 @@ void signature_init(void); > int signature_extract_to_list(struct mailbox_transaction_context *t, > struct mail *mail, struct siglist **list, > enum classification wanted); > -const char *signature_extract(struct mailbox_transaction_context *t, > - struct mail *mail); > +int signature_extract(struct mailbox_transaction_context *t, > + struct mail *mail, const char **signature); > void signature_list_free(struct siglist **list); > > extern const char *signature_hdr; > > This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

2 4

Re: [Dovecot] [dspam-users] mysql fetch row error
by Marko Weber 15 Nov '08

15 Nov '08

Hi Steve, MTA = Postfix I have amavis - spamassassin - dspam installed. i think spamassassin scnas after dspam, cuase in the mail headers dspam comes at first. I use Dovecot as Imap Server OS = Gentoo Linux i put the config of dspam here = http://rafb.net/p/VCX8XO97.html if u need more confs just tell me , i can paste then marko Steve schrieb: > Hallo Marko > > This is not normal. DSPAM does not add the encoding. It must be something else you are using/doing to get that tagged with the encoding in front of every DSPAM tag. And I don't see the DSPAM signature in your header. This is not normal. If you have the other DSPAM header lines, then you should have the result as well. > > What MTA are you using? > How is the integration between MTA and DSPAM? > Do you have other filters running before and after DSPAM? > What IMAP server are you using? > What OS are you using? > Could you post your DSPAM config? > > > (Grüsse aus Zürich) > > // Steve > > -------- Original-Nachricht -------- > >> Datum: Sat, 15 Nov 2008 10:37:38 +0100 >> Von: Marko Weber <weber(a)zackbummfertig.de> >> An: "dspam-users(a)lists.nuclearelephant.com" <dspam-users(a)lists.nuclearelephant.com> >> Betreff: Re: [dspam-users] mysql fetch row error >> > > >> Hello Steve this is the "full" Header of a spam mail where the training >> also fails. >> withe the mysql fetch error............. >> >> marko >> >> >> Return-Path: <RoxieastronomerMyrick(a)fightingbobfest.org> >> X-Original-To: weber(a)salonwebserver.de >> Delivered-To: weber(a)salonwebserver.de >> Received: from localhost (localhost [127.0.0.1]) >> by salonwebserver.de (Postfix) with ESMTP id 02C8613BC09B >> for <weber(a)salonwebserver.de>; Fri, 14 Nov 2008 21:38:09 +0100 >> (CET) >> X-DSPAM-Result: =?iso-8859-1?Q?Spam=0D?= >> X-DSPAM-Confidence: =?iso-8859-1?Q?0.7177=0D?= >> X-DSPAM-Probability: =?iso-8859-1?Q?1.0000=0D?= >> ?iso-8859-1?Q?491de1b0254959695846858=0D?= >> X-DSPAM-Factors: 15, >> X-Virus-Scanned: amavisd-new at salonwebserver.de >> X-Spam-Flag: NO >> X-Spam-Score: 2.415 >> X-Spam-Level: ** >> X-Spam-Status: No, score=2.415 tagged_above=-1000 required=2.5 >> tests=[DNS_FROM_SECURITYSAGE=0.127, RCVD_IN_BL_SPAMCOP_NET=2.188, >> RDNS_DYNAMIC=0.1] >> Received: from salonwebserver.de ([127.0.0.1]) >> by localhost (kraftwerk1.salonwebserver.de [127.0.0.1]) >> (amavisd-new, port 10024) >> with ESMTP id nOQQilMNvp-o for <weber(a)salonwebserver.de>; >> Fri, 14 Nov 2008 21:38:07 +0100 (CET) >> Received: from zackbummfertig.de >> (static.243.42.46.78.clients.your-server.de [78.46.42.243]) >> (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) >> (No client certificate requested) >> by salonwebserver.de (Postfix) with ESMTPS id E86BD13BC078 >> for <weber(a)salonwebserver.de>; Fri, 14 Nov 2008 21:38:06 +0100 >> (CET) >> Received: from localhost (localhost [127.0.0.1]) >> by zackbummfertig.de (Postfix) with ESMTP id BC4BDECC067 >> for <weber(a)salonwebserver.de>; Fri, 14 Nov 2008 21:37:46 +0100 >> (CET) >> X-Virus-Scanned: amavisd-new at zackbummfertig.de >> Received: from zackbummfertig.de ([127.0.0.1]) >> by localhost (weberweb.zackbummfertig.de [127.0.0.1]) >> (amavisd-new, port 10024) >> with ESMTP id xr6ayW0lyDGC for <weber(a)salonwebserver.de>; >> Fri, 14 Nov 2008 21:37:43 +0100 (CET) >> Received: from rachel22b12579 (unknown [78.146.113.186]) >> by zackbummfertig.de (Postfix) with SMTP id 3E83CECC040 >> for <tbmarcom(a)acidbunker.de>; Fri, 14 Nov 2008 21:37:43 +0100 >> (CET) >> Date: Fri, 14 Nov 2008 20:38:01 +0000 >> From: Chrystal Huddleston <RoxieastronomerMyrick(a)fightingbobfest.org> >> Subject: gambling slot machines >> To: tbmarcom(a)acidbunker.de >> Message-id: <7686026460.20081111151378(a)fightingbobfest.org> >> Organization: fightingbobfest.org >> MIME-version: 1.0 >> Content-type: text/plain; charset=us-ascii >> Content-transfer-encoding: 7bit >> X-Priority: 3 (Normal) >> >> Steve schrieb: >> >>> Can you post the full header of the message in question? >>> >>> >>> -------- Original-Nachricht -------- >>> >>> >>>> Datum: Fri, 14 Nov 2008 23:18:39 +0100 >>>> Von: Marko Weber <weber(a)zackbummfertig.de> >>>> An: "dspam-users(a)lists.nuclearelephant.com" >>>> >> <dspam-users(a)lists.nuclearelephant.com> >> >>>> Betreff: Re: [dspam-users] mysql fetch row error >>>> >>>> >>> >>> >>>> Hi Steve, >>>> this sed thing dont worked for me...... sorry >>>> how i can get dspam working without this mysql fetch error ? >>>> u have any hints ? >>>> >>>> marko >>>> >>>> >>>> Steve schrieb: >>>> >>>> >>>>> -------- Original-Nachricht -------- >>>>> >>>>> >>>>> >>>>>> Datum: Fri, 14 Nov 2008 18:11:55 +0100 >>>>>> Von: Marko Weber <weber(a)zackbummfertig.de> >>>>>> An: "dspam-users(a)lists.nuclearelephant.com" >>>>>> >>>>>> >>>> <dspam-users(a)lists.nuclearelephant.com> >>>> >>>> >>>>>> Betreff: [dspam-users] mysql fetch row error >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>>> Hello to all in the Mailinglist, >>>>>> >>>>>> i installed dspam on our server and it works so far that incoming >>>>>> >> mails >> >>>>>> are tagged by dspam. >>>>>> i installed "dovecot-antispam" for retraining. >>>>>> when in try to retrain a mail i get this in the dspam.debug log = >>>>>> >>>>>> 9056: [11/14/2008 15:23:01] mysql_fetch_row() failed in >>>>>> >>>>>> >>>> _ds_get_signature >>>> >>>> >>>>>> in the /var/log/messages log i get this = >>>>>> >>>>>> Nov 14 15:29:25 kraftwerk1 imap: antispam: /usr/bin/dspam >>>>>> >>>>>> >>>> --source=error >>>> >>>> >>>>>> --class=spam --signature==?iso-8859-1?Q?491d76cb23581804284693=0D?= >>>>>> >> ... >> >>>>>> Nov 14 15:29:25 kraftwerk1 dspam[9189]: Signature retrieval for >>>>>> '=?iso-8859-1?Q?491d76cb23581804284693=0D?=' failed >>>>>> Nov 14 15:29:25 kraftwerk1 dspam[9189]: Unable to find a valid >>>>>> signature. Aborting. >>>>>> Nov 14 15:29:25 kraftwerk1 dspam[9189]: process_message returned >>>>>> >> error >> >>>>>> -5. dropping message. >>>>>> >>>>>> >>>>>> but when i search for the signatur 491d76cb23581804284693 in the >>>>>> >> mysql >> >>>>>> database i can find it there. >>>>>> >>>>>> what is this ?iso-8859-1?Q? , is this causing the error ? >>>>>> >>>>>> >>>>>> >>>>>> >>>>> This means, that the plugin developer used the signature from the body >>>>> >>>>> >>>> instead of the one found in the header. You can verify yourself what >>>> >> the >> >>>> proper signature would be. Just issue this small one liner in sed: >>>> >>>> >>>>> sed -e '/^$/q' -ne 's/^X\-DSPAM\-Signature: >>>>> >>>>> >>>> $\([1-9][0-9]*,$\{0,1\}$[a-f0-9]\{20,32\}$\)$/\1/p' >>>> >> /path/to/your/message >> >>>> >>>> >>>>> This should print out the proper signature DSPAM should receive from >>>>> >> the >> >>>>> >>>>> >>>> plugin. The iso-8859-1 part indicates to me that the message is taken >>>> >> from >> >>>> a iso8859-1 encoded message body. >>>> >>>> >>>>> >>>>> >>>>> >>>>>> do any1 need my dspam.conf , then i can paste it on web. >>>>>> >>>>>> thank for helping. >>>>>> >>>>>> marko >>>>>> >>>>>> >>>>>> >>>>>> >>>>> Steve >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> > >

2 1

[Dovecot] Mailbox Hashing
by Justin Krejci 14 Nov '08

14 Nov '08

First off, the website documentation is really good for Dovecot but while reading I was not able to find anything pertaining to inbox hashing for Maildirs. I saw plenty about hashing the directories that the user mailboxes live in but nothing about specifically hashing an individual user's inbox directory itself. Is there any method for hashing the inbox automatically after say 5,000 messages are stored? Example $Maildir/in/0/message0 $Maildir/in/0/message1 $Maildir/in/0/message2 . $Maildir/in/0/message4999 $Maildir/in/1/message5000 $Maildir/in/1/message5001 etc I am not currently using Dovecot but am interested to know if this is available or does running with 20,000+ messages in a single inbox not affect the performance much? I have looked into other file system tuning techniques such as enabling ext3 dir_index or using ReiserFS (maybe not ReiserFS anymore). There will likely be 15,000 to 20,000 accounts spread out on one or more servers using a 6-drive RAID10 setup. Most accounts are not expected to have high message quantities but there will be lots of concurrent connections via pop and imap (and webmail imap). Any suggestions or feedback would be appreciated.

3 6

Re: [Dovecot] Install Configuring and run.
by Marko Weber 14 Nov '08

14 Nov '08

Hello Thorsten, i got "dovecot-antispam" working now. also moving mails now fully work. but retraining dont seems to work fully. emails that are delivered to mailbox are scanned with user "clamav" 28799: [11/13/2008 23:26:25] DSPAM Instance Startup 28799: [11/13/2008 23:26:25] input args: /usr/bin/dspam --stdout --deliver=spam,innocent --mode=tum --tokenizer=chained,noise --enable-signature-headers --user clamav 28799: [11/13/2008 23:26:25] pass-thru args: --tokenizer=chained,noise --enable-signature-headers 28799: [11/13/2008 23:26:25] processing user clamav 28799: [11/13/2008 23:26:25] uid = 105, euid = 105, gid = 449, egid = 1000 28799: [11/13/2008 23:26:25] loading preferences for user clamav 28799: [11/13/2008 23:26:25] Loading preferences for uid 1 i dont have a maildrop filter ready yet that move spam tagged mails into SPAM folder. all is delivered to "incoming". when i now move the mail to SPAM; this is done by user "postfix" 28917: [11/13/2008 23:40:17] DSPAM Instance Startup 28917: [11/13/2008 23:40:17] input args: /usr/bin/dspam --source=error --class=spam --signature==?iso-8859-1?Q?1,491930ac25229173225906=0D?= 28917: [11/13/2008 23:40:17] pass-thru args: 28917: [11/13/2008 23:40:17] processing user postfix 28917: [11/13/2008 23:40:17] uid = 207, euid = 207, gid = 207, egid = 1000 28917: [11/13/2008 23:40:17] loading preferences for user postfix 28917: [11/13/2008 23:40:17] Loading preferences for uid 6 28917: [11/13/2008 23:40:17] Loading preferences for uid 0 28917: [11/13/2008 23:40:17] Loading preferences for uid 0 28917: [11/13/2008 23:40:17] default preferences empty. reverting to dspam.conf preferences. 28917: [11/13/2008 23:40:17] Loading preferences from dspam.conf 28917: [11/13/2008 23:40:17] using /var/spool/dspam/opt-in/local/postfix.dspam as path 28917: [11/13/2008 23:40:17] using /var/spool/dspam/opt-out/local/postfix.nodspam as path 28917: [11/13/2008 23:40:17] sedation level set to: 0 28917: [11/13/2008 23:40:17] DSPAM Instance Shutdown. Exit Code: 0 Now when i move the mail again into incoming (also done by user postfix), the mail dont get tagged with "innocent" , it is still tagged "spam". How do i get it work all under "clamav" and why are the removed mails from spam to incoming not tagged with "inncoent" ? (do you need any logs?) thank you marko Thorsten Vollmer schrieb: > On Thu, 2008-11-13 at 21:30 +0100, Marko Weber wrote: > >> Sorry to ask, >> so if dovecot-antispam-20080829 dont work for me you mean i should >> install the latest from webpage ? >> > > Yes, dovecot-antispam-1.1 should be sufficient. I already sent you the > link to an updated ebuild. Please be more proactive. > > Thorsten > >

2 6

[Dovecot] Folder aliases
by Daniel L. Miller 14 Nov '08

14 Nov '08

I don't if this has been discussed before - or already implemented - but it would be great if I could define one or more "aliases" for a folder. A particular example might be "Sent" where different (badly-written) clients might have hard-coded "Sent" folder locations like "Sent", "Sent Items", "Sent Mail", "Mail Delivered", etc. - and I'd love to get those consolidated. It would also be nice to show a different set of folders based on the client - or if that's not possible (if the client doesn't identify itself as part of the login sequence) then somehow via login name - maybe by an extension? So where my usual login might be "user(a)domain.com", and that would continue to be supported as the default login, I could now have "user(a)domain.com+thunderbird", "user(a)domain.com+mobile", etc.? -- Daniel

6 10

Re: [Dovecot] Install Configuring and run.
by Marko Weber 13 Nov '08

13 Nov '08

Sorry to ask, so if dovecot-antispam-20080829 dont work for me you mean i should install the latest from webpage ? Thorsten Vollmer schrieb: > On Thu, 2008-11-13 at 00:24 +0100, Marko Weber wrote: > >> Hello Thorsten thank you for taking time. >> I recompiled with -debug, but i also had to set -syslog >> otherwise gentoo dont compile. it says then , "useflag syslog enabled >> needs flag debug also set" >> >> we proceed further in the list then ? >> > > Yes, once you have new results. > > If dovecot-antispam-20080829 does not yet work for you, please try the > latest version and re-enable debugging. > > Thorsten > >

1 0

[Dovecot] New Dovecot server buildout review
by David Jonas 13 Nov '08

13 Nov '08

Hello Everyone, I have the opportunity to do a new mail system build out for about 20k email boxes over a few hundred domains. I already run a system like this, but this time I can do it right :). My thought on storage is to have a 12 drive JBOD system attached via SAS to a 3ware 9690SA raid controller. Drives would be RAID'd (10 or 5) to appear as one for message storage. Dovecot would run on that box and deliver directly from postfix. POP and IMAP connections would come in directly (no proxy). Postfix head boxes in front of this do anti-virus and spamassassin, so this box won't be doing any heavy crunching, just delivery and serving of messages. Does this sound reasonable? Is there details that I'm missing? An obvious bottleneck I'm overlooking? Is anyone running a similar config with success? It all sounds very standard, but I haven't run something like this before. Thanks for the help! David

1 0

[Dovecot] Moving mail between folders
by Ilo Lorusso 13 Nov '08

13 Nov '08

Hi, Ive just installed Dovecot and postfix and Ive created a whole bunch of folders and started moving my mail around. Ive noticed that it copies the mail to the folder I wish to move the mail to and leave the original message waiting to be purged in the original folder Is there a way to have dovecot delete it straight after it has been moved? as I don't think users are going to take well to this change? any help would be appreciated Thanks, Ilo.

3 2

[Dovecot] dovecot pop3 performance, what size installation
by Brandon Lamb 13 Nov '08

13 Nov '08

Hello, I was wondering if anyone could tell me their results and maybe help me get an idea how big our mail system installation and usage is. For dovecot pop3 and nfs backend, I am curious how many machines I should expect to need. Here are my stats for what I can tell. 5-7 connections per second average, 8-12 connections per second peak. 30-50 constant connections between 3 servers (8-20+) on each server. We are servicing about 14,000+ email accounts. These are just the pop3 login stats, a lot of users use both or imap through our webmail. My question I guess is whether this is on the small side, medium or larger side mail setup, and should I be thinking that only 1 or 2 servers should be able to handle the load. Right now I am splitting it between 3. Should a single quad core box be able to handle this? Not sure if this is a simple question or not. Im not including my dovecot -n just because this is more of a generic question.

1 0