Jon Callas wrote:

So I have a tangential question -- am I doing the right thing? Should I rip out Postfix and go to Sendmail?

Personally, I'd stick with Postfix. You'll also get the free SMTP AUTH integration with Dovecot.

And I say this as someone running sendmail and postfix.

-- Curtis Maloney cmaloney@cardgate.net

...

I have presently set up my new server to use Postfix and virtual mailboxes. I want to add Dovecot to that and then do something reasonable on top of it, like SpamAssassin.

I highly recommend that anyone who wants a very lightweight, *very* effective, brain-dead simple (installation *and* maintenance) anti-spam gateway check out ASSP (http://assp.sourceforge.net/)... it was already one of the most effective options available, but there has been a *lot* of activity recently, and the improvements are amazing. Combined with ClamSMTP (http://memberwebs.com/nielsen/software/clamsmtp/) - also very light-weight, very easy to install, configure and maintain - it has reduced the load on my server drastically (through its early HELO checks, and the new PB (PenaltyBox) functionality, and is virtually maintenance free.

It replaces (for me) SpamAssassin, Postgrey and RBL - and I could most likely do away with ClamSMTP as well (haven't had a virus get past ASSP yet, in about 4 months, since I installed ASSP).

1.2 is in the late beta/RC stages, and will most likely be released in the next week or two. The only negative is, it has been changing so rapidly, that the documentatiuon hasn't had a cnahce to catch up.

These recent changes began as unofficial changes by Fritz Borgstedt, who was later made an official developer. You can read all about these changes in the mail list archives.

As for the other parts, I use Postfix, and am in the process of converting from Courier to Dovecot (which is why I'm here)...

--

Best regards,

Charles

Tere.

I highly recommend that anyone who wants a very lightweight, *very* effective, brain-dead simple (installation *and* maintenance) anti-spam gateway check out ASSP (http://assp.sourceforge.net/)... it was already one of the most effective options available, but there has been a *lot* of activity recently, and the improvements are amazing. Combined with ClamSMTP (http://memberwebs.com/nielsen/software/clamsmtp/) - also very light-weight, very easy to install, configure and maintain - it has reduced the load on my server drastically (through its early HELO checks, and the new PB (PenaltyBox) functionality, and is virtually maintenance free. Well, I haven't tested it yet, but did check the code, it's written in Perl, so surely it's not lightweight, and in Your case, maybe configuring the smtp do the HELO check, would improve the situation even without this software. And Assp is also old - June 26, 2005, can't actually trust that old piece of code...

Btw, I just tested ClamSmtp, and it passed true viruses, but Amavis did catch them.

-- Sysadmin

...

Well, I haven't tested it yet, but did check the code, it's written in Perl, so surely it's not lightweight,

Quick, your bias is showing... ;-)

heh... bias? We're not biased in here! ;)

For example, *all* of the inbound email to the apache.org and perl.[org|com] domains is handled by qpsmtpd, which is an advanced MTA written in Perl.

http://smtpd.develooper.com

The next generation qpsmtpd will be capable of handling tens of thousands of simultaneous inbound connections (currently used only for spamtraps).

FULL DISCLOSURE: I am one of the developers of qpsmtpd.

Neat - I'll check it out...

...

Btw, I just tested ClamSmtp, and it passed true viruses, but Amavis did catch them.

Did you make sure to update the ClamAV definitions? Did you make sure that you configured ClamAV correctly (e.g. to scan inside archives)? A quick test done without understanding the tools involved is equivalent to no test at all.

Correct - also, ClamSMTP != ClamAV, though it does use the same definitions.

--

Best regards,

Charles

Tere.

Quick, your bias is showing... ;-)

:).

There are many ways to use Perl so that it is lightweight (in terms of performance, since RAM is cheap).

Well, Ram is cheap but this isn't the main goal. Usually perl needs latest version, all kind of modules etc, and C based software does not. And I have no time to install all that stuff and most of all I have no time to even add more Ram if needed just like that, as servers are running 24/7 and only 4 minutes downtime per year is allowed:(.

You apparently didn't read very closely; the main project hasn't had a recent *official* release, but one of the contributors has been continuing to develop (and his last released code in the 1.1.2 series was Mar 28, 2006 and in the 1.2.0 series was today).

...

You right, I found there link to the latest version, did You mean - http://www.magicvillage.de/~Fritz_Borgstedt/assp/ which gives at this moment:

Internal Server Error

Service Unavailable

Did you make sure to update the ClamAV definitions? Did you make sure that you configured ClamAV correctly (e.g. to scan inside archives)? A quick test done without understanding the tools involved is equivalent to no test at all.

Sure. ClamSmtp did catch some, Amavis did catch all, with identical settings and latest software/definitions, so I choose to use Amavis in future, as I did use it also previously, ClamSmtp was just a test purpose only.

-- Sysadmin

Tere.

This is going badly off topic, but spam & virus scanning on SMTP-level is one of the easiest services to make load balanced and highly available, so you can get 100% uptime per year (really!!) and still do necessary maintenance on your servers.

Yes and not. But this is off topic now.

So perl is bad but still you chose perl written Amavis instead of C written ClamSMTP ..

Heh, You missed the point, that with sendmail I use clamav-milter/milter-greylist mostly, but with postfix I must use perl based stuff (these are also powerful computers).

My point is, that everyone should use the software which he know best.

-- Sysadmin

hi,

i happily live with dspam atm. i just wait for the new version of the dspam plugin from johannes. :)

darix

http://dspam.nuclearelephant.com/

-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org

On 2006-04-18 21:56:49 +0200, Johannes Berg wrote:

...

i happily live with dspam atm. i just wait for the new version of the dspam plugin from johannes. :)

I'm not writing any new version at the moment, nor do I plan to upgrade my dspam. What specifically do you need?

libdspam instead of system("dspam...")

:)

darix

-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org

Johannes Berg wrote:

On Tue, 2006-04-18 at 22:56 +0200, Marcus Rueckert wrote:

...

libdspam instead of system("dspam...")

That's a lot of work, how about calling to the dspam server instead?

johannes

YES PLEASE. ;)

I'm also curious as to whether there is any significant local overhead if you're calling the dspam server... If not, it might be better to just have a server plugin, that's settable to localhost.

-deano

On Tue, 2006-04-18 at 16:13 -0400, Charles Marcus wrote:

I used to use dspam, and agree it is pretty cool - very powerful if you need a very granular solution. But it is also very complex to install and configure - at least as compared to ASSP. It is also VERY I/O intensive, where ASSP is extraordinarily lightweight (except when rebuilding the spamdb).

You can probably trivially rewrite my plugin for that :)

johannes

On Wed, 2006-04-19 at 09:34 -0400, Charles Marcus wrote:

? To rewrite the spamdb?

rewrite my plugin to train assp

johannes

clamAV for virus spamprobe for spam

I highly recommend both of them. Spamprobe is just fantastic. However they both suffer from the fact that they require per-user integration (typically via procmail). Thats great for a 10 user box, its unworkable for a 10,000 user box. My box is a 10 user box so I love them. I know there are site-wide integrations for both of them but I've not tried them ... and with spamprobe its largely against its design ... so I'm not confident of a global spam bayesian database ... you know "my spam" != "your spam" ... I could use a good home refinance and cheep drugs :) Thats why I love spamprobe, I own my spam definitions. Mine all mine !!!!

I am looking at this "assp" ... I like the idea of a configurable smtp layer filter, although I detest RBL's ... being configurable is nice (I can turn OFF the RBL). I dont like that its in perl due to my admited huge bias, but if it really works well I could maybe overlook that ... I try to be language agnostic in theory ... but sometimes I have a hard time ... being that I've written software in most languages, enough to grow a few biased oppinions :)

-David

David A. Lee dlee@calldei.com http://www.calldei.com

On Tuesday 18 Apr 2006 17:57, Marc Perkel wrote:

I'm basically trying to use every trick out there. Here's a detailed explation of everything I'm doing:

http://www.junkemailfilter.com/spam/how_it_works.html

Interesting.

I'm processing about 1.2 million email attempts per day and passing 50,000 good messages on. And most all of it with a single computer that's a Dual Core Athlon 3800 with 4 gigs of ram.

But I'm always looking for new tricks.

We are slightly smaller.

I also use the spamhaus block list.

I've focused almost entirely on trying to reject at SMTP connection time on criteria other than content of email (with the exception of known bad file attachment types (.scr etc)).

I don't see the goal as stopping all viruses, or all spam, but deploying a system to make email liveable again, without burdening the users with additional folders etc. In many cases we are forwarding email on again.

Historically the greylisting was almost perfect, but it is easily defeated and that has started occurring recently.

http://www.debian-administration.org/articles/168

Would love to see so serious analysis of "HELO" based blocking. Whilst I tend to think it is a bad idea, if there are criteria I can exploit in identifying things that aren't genuine mail servers -- it fits the strategy.

On 19/04/2006 9:29 p.m., Tomi Hakala wrote:

Simon Waters wrote:

...

Would love to see so serious analysis of "HELO" based blocking. Whilst I tend to think it is a bad idea, if there are criteria I can exploit in identifying things that aren't genuine mail servers -- it fits the strategy.

Some very broken spam tool sends IP address of an MX host it is speaking to in HELO response, this should never happen with real mail hosts so it is safe to block all such connections. This blocks high amount of spam for us.

Ditto with 'localhost', '127.0.0.1' and your host's own hostname, and apart from what you get from any of your secondary MX's if you have them - their hostname too. Although there's the rule that you should be liberal in what you accept and I believe HELO is something that you're supposed to accept regardless of what the remote end claims, I've never found legitimate hosts using any of these arguments to HELO.

If you're slightly more brave then also add non-FQDN and anything which starts with a '-' such as -1269643152' which I get lots of to invalid addresses. I'm yet to see a false positive from setting all of these in a year or so since I implemented them, but then my system probably isn't as critical as some others...so I can afford to be more brave.

I'd say with a lot of confidence that I've had more false positives from dynamic blocklists tagging email than HELO checking (perhaps not surprising).

reuben

But as regards HELO, I know what the rules for HELO state, and so I know why it should work (even if it is considered RFC infringing). But I've never seen a detailed analysis of HELO strings in the same manner as I've seen studies for the effectiveness of different RBL.

This is one of the things that ASSP has been doing VERY reliably, and with some of the changes that Fritz has made (PenaltyBox being the main one), it is even better.

What I want to see is someone saying "X% of our normal genuine email servers were (would have been) caught" & "it stopped (or would have stopped) Y% of spam".

Lots of anecdotes don't cut it I'm afraid, not that I don't believe the people, just it is different when it is someone elses email you are filtering, especially if you are making a Yes/No decision on accepting email on the basis of the test. I need to know if it is <1%, <0.1%, or <0.001% false positives, as that'll establish how many people get angry. Obviously email sources vary, but I need to know where to concentrate the effort.

If you are really interested in how HELO checks can be used to drastically cut the number of valid connections, I highly suggest trying out ASSP, or at least checking its code.

--

Best regards,

Charles

On Tue, 2006-04-18 at 12:48 -0400, Mark E. Mallett wrote:

Well I'll put in another plug for my own piece of work..

http://www.mvmf.org/

includes several tools among which are:

• mvmda, mail delivery agent with a powerful scripting language, including a SIEVE submode (which can be used exclusively or in conjunction with non-sieve elements);

So when can this use Dovecot directly? ;) I looked at it when first writing Dovecot LDA, but it seemed way too much trouble to use it compared to Cyrus Sieve library.

On Fri, 2006-04-21 at 17:02 -0400, Mark E. Mallett wrote:

...

...

• mvmda, mail delivery agent with a powerful scripting language, including a SIEVE submode (which can be used exclusively or in conjunction with non-sieve elements);

So when can this use Dovecot directly? ;)

I use it with dovecot now :)

If you mean... updating the indexes and so forth, I have been thinking about that again lately. I still think the best thing would be for you to supply a small utility that can be used by other delivery agents (or whatever) to either:

• dumbly deliver a message to a folder, but in a way that's friendly to dovecot's indexing, a la uw's tool the name of which escapes me; or

I guess dovecot-lda already does this.. And I guess I should fix the deliver binary that comes with Dovecot before v1.0.

• accept some info that would update the indexes for a specific message.

This probably wouldn't work that well. With mbox it kind of already does that when opening the mailbox the next time..

Alternatively I was thinking about adding support for dovecot-style indexing, but I'd first have to understand those indexes. The wiki is empty on the subject and I haven't yet attempted to scour the code. One problem here is that the delivery agent and dovecot would both have to agree about what is being indexed;

I think you'd pretty much have to use my mail-storage.h API the same way as Dovecot-LDA does. In any case it's not worth it to try to write code which directly updates the indexes.

I suppose the delivery agent could read the dovecot.conf file but I'd like to avoid that level of connection.

Dovecot-LDA does that, but I hate it too. I don't really see a better solution though. Dovecot 2.0 has a nice config process where the LDA can nicely ask for its configuration. :)