Gnu sieve vs Dovecot sieve-filter - sieve-filter extremely slow at lda (writing emails to local mbox files)

[@lbutlr]

On Sep 12, 2019, at 12:57 AM, Zenaan Harkness <zenaan at freedbms.net> wrote:
> The next step, I throw the email-incoming-unsorted mbox file at a
> sieve processor, to sort the emails from that mbox, into other
> mboxes, according to the sieve rules file.

I would expect mbox is the worst possible format choice for this.

> Gnu sieve balks on emails which have no x-message-id (?? something
> like this) header field, so after a few years, I finally decided to
> switch "up" to Dovecot/Pigeonhole's "sieve-filter" command.
> 
> Using Gnu sieve, this mbox sorting step was even faster than mpop (/
> getmail) - and mpop and getmail are really fast (compared with
> fetchmail), since they pipeline the email downloads.

Perhaps because of its reliance on the header allowing it to index?

> Even with 100s of emails, Gnu sieve would take only 10 to 20 seconds
> at most. Super fast.

That doesn’t sound fast. I processed a few thousand messages through sieve in less than 10 seconds, if I recall correctly.

> See below for details, any ideas appreciated.

The first thing I would do is download to Maildir and see what the difference is.



-- 
What we have here is a failure to communicate.