[Dovecot] Creating Maildir and populating with emails via external Python process

Thu Oct 25 23:18:52 EEST 2012

On Thu, 25 Oct 2012 12:13:34 -0700 (PDT), Bradley Rintoul said:
> This could be good.  I'd never looked into the LDA - I will study up
> on it.
> 
> Someone else was helping out here and I thought I'd shed some more
> light on what I'm doing here...
> 
> Let's  say someone has an account with Yahoo, for example.  My Python
> code is fetching email from the user's Yahoo! account and placing it
> into the  Dovecot Maildir storage for a particular user.  Now when
> the user  retrieves their mail, they are doing so using my Dovecot -
> my Dovecot  instance is acting as a proxy, of sorts...
> 
> Thanks for the responses!
> 
> (Is there an IRC channel?)

Hi Bradley,

I'm doing almost the exact same thing, but with fetchmail and procmail.
I go out and grab my email from about five different places using
fetchmail, which feeds the messages to procmail, with .procmailrc
deciding where in the Dovecot maildir tree to put them.

Your only need I *haven't* accomplished is having different users get
their mail from my Dovecot, and to make sure each users' email goes
where they can get it via IMAP connection to your Dovecot. If you can
get different IMAP mailboxes for different users, you can put each user
in .procmailrc so as to deliver to the correct box.

Anyway, Procmail knows exactly how to submit an email to Dovecot, so
you don't need to worry about actually placing the file into the tree,
or anything like that.

You mention you've written some Python code. If the purpose of your
Python code is just to retrieve from SMTP servers, you can drop your
Python code in favor of Fetchmail and Procmail. If your Python code
actually does something with the emails, you can call a subset of your
Python code from Procmail, to do its magic on each email.

Here's how my fetchmail is running:

29588 ?        Ss     0:21 fetchmail -f /home/slitt/.fetchmailrc

And here's a partial view of my .fetchmailrc, showing my retrieval
from Bluehost and gmail:

===================================
set postmaster "slitt"
set bouncemail
set no spambounce
set properties ""
set daemon 180

#poll mail.a3b3.com protocol POP3:
poll mail.a3b3.com protocol IMAP:
        user 'slitt at troubleshooters.com' there is 'slitt' here
        pass wouldnt_you_like_to_know
        limit 50000000
        warnings 3200
        expunge 60
        ssl                             #Use ssl encryption
        sslcommonname "*.bluehost.com"  
        sslcertck                     
        mda "/usr/bin/procmail -d %T" fetchall;

poll imap.gmail.com protocol IMAP 
        user 'litttest at gmail.com' there is 'slitt' here
        pass 'I_just_cant_say'
        #portnumber 993
        limit 50000000
        warnings 3200
        expunge 60
        mda "/usr/bin/procmail -d %T"
        fetchlimit 50
        ssl;
====================================

Do you notice the "mda" line on both pulls? That means "use procmail as
your mda", which just ships each email to Procmail. Procmail knows
exactly how to deliver stuff to Dovecot.

The following are the top several lines of my .procmailrc:

===================================
DEFAULT=$HOME/mail/Maildir/.INBOX/
MAILDIR=$HOME/mail/Maildir/
LOCKFILE=$HOME/mail/.lock
VERBOSE=no
LOGFILE=$HOME/procmail/log
#GARBAGE=.garbage/
GARBAGE=/dev/null
PURGATORY=.garbage/
SUPREMUM=9876543210    #PROCMAIL SUPREMUM NUMBER, SEE
http://www.perlcode.org/tutorials/procmail/proctut/proctip2.pod

#### HANDLE STUFF FROM littdom at gmail.com and litttest at gmail.com ####

:0:
* ^Delivered-To:.*littdom at gmail.com
.littdom_gmail/

:0:
* ^Delivered-To:.*litttest at gmail.com
.litttest_gmail/
===================================

A few explanations: First, I couldn't include my actual filters,
because they are full of very unflattering comments concerning various
trolls, ignos, blabbermouths, and proudly helpless fools.

The $MAILDIR environment variable is the rood directory of your Maildir
tree. $DEFAULT is the location of the main inbox for that -- I think
it's where you put email that doesn't get routed elsewhere by Procmail.
$GARBAGE is an environment var I made up as code for where filtered
stuff gets sent. It's usually /dev/null because I don't want to see
that junk again. However, I can temporarily change it to an actual IMAP
directory for troubleshooting. $PURGATORY is junk that I actually want
to OK the deletion of. I actually currently have nothing filtered to
$PURGATORY, but it's there. $SUPREMUM is a very large number that is
used in making OR logic, which is otherwise difficult. I couldn't make
the $SUPREMUM env var work, so I had to use a literal, and here's a way
I got all my magazines into one mailbox:

:0:
* 9876543210^0 ^From.*onsale.com
* 9876543210^0 ^From.*pcmag.com
* 9876543210^0 ^From.*itworld.com
* 9876543210^0 ^From.*networkworld.info
* 9876543210^0 ^From.*infoworld.com
* 9876543210^0 ^From.*whatsnewnow.com
* 9876543210^0 ^From.*eweek.com
* 9876543210^0 ^From.*computerworld.com
.mags/

By the way, BE SURE to note the slash after the directory name. That
trailing slash tells Procmail that it's delivering to a Maildir, not to
an (ugh) mbox.

Anyway, I think you and I are doing very similar things, albeit for
very different reasons. My motivation is that I consider all currently
available email clients to be junk, and don't want them holding my
email, so I hold it in a Dovecot hosted Maildir instead.

I'll be interested in how you solve this. Please keep me (and probably
everyone on this list) in the loop.

Thank you so much!

SteveT

Steve Litt                *  http://www.troubleshooters.com/
                          *  http://twitter.com/stevelitt
Troubleshooting Training  *  Human Performance