[Dovecot] Dovecot namespace solved while writing; preparing to refilter

miro.rovis at croatiafidelis.hr miro.rovis at croatiafidelis.hr
Sun Oct 6 07:38:40 EEST 2013


Since I still have to send mail from web, I may not be able to get this 
message as a reply to my previous mail, but it belongs to this same 
thread:

http://www.dovecot.org/list/dovecot/2013-October/092810.html

Some of the work I have now done.
I can now easily, lest some hurdle shows up, use a script based on the 
one I mentioned (here it is again: 
http://wiki.dovecot.org/HowTo/RefilterMail ), but I used maildrop, 
having harkened to the advice by the author of Debian Reference, Osamu 
Aoki:

http://www.debian.org/doc/manuals/debian-reference/ch06.en.html#_the_remote_mail_retrieval_and_forward_utility

http://www.debian.org/doc/manuals/debian-reference/ch06.en.html#_mail_delivery_agent_mda_with_filter

So while, after connecting with the password, I wait some 5 or 10 
minutes (the machines I install/clone Debian onto are old machines) for 
all the mail from the other Dovecot on the other machine, same MBO and 
all, as I explained in the previous message, again; I am preparing this 
before going online, I haven't see if there are any replies in the last 
maybe 40 hours)...

me at mybox2:~$ getmail
getmail version 4.43.0
Copyright (C) 1998-2012 Charles Cazabon.  Licensed under the GNU GPL 
version 2.
SimpleIMAPSSLRetriever:me at mybox1.myfantasydomain:993: read_all and not 
delete -- all messages will be retrieved each time getmail is run
SimpleIMAPSSLRetriever:me at mybox1.myfantasydomain:993:
Enter password for 
SimpleIMAPSSLRetriever:me at mybox1.myfantasydomain:993:

The:

me at mybox2:~$ getmail

command that I issued has just churned out all the 2945 messages into 
my homefolder here.
And this is the previously prepared

----%<--------------------------------------------------------
$ cat ~/.getmail/getmailrc
[retriever]
type = SimpleIMAPSSLRetriever
server = mybox1.myfantasydomain
username = mr
mailboxes= ALL

[destination]
type = MDA_external
path = /usr/bin/maildrop
unixfrom = True

[options]
verbose = 2
delete = False
read_all = True
delivered_to = False
message_log = ~/.getmail/Mrinet_refilter.log
----%<--------------------------------------------------------

Not much talk needed about the getmailrc above, because it's all in the 
getmail manual, faq, and places.

Sure, I also, prepared .mailfilter, for the maildrop, the fine program 
that nicely put all mails where I decided I wanted them!

----%<--------------------------------------------------------
$ cat ~/.mailfilter

# ~/.mailfilter
# based on:
# http://www.wonkity.com/~wblock/docs/html/maildrop.html

TYPE="maildir"

logfile "$HOME/.mailfilter.log"

ECHO="/bin/echo"
MAIL="/usr/bin/mail"
MAILDIRMAKE="/usr/bin/maildirmake"
REFORMAIL="/usr/bin/reformail"

MBOX=($TYPE =~ /mbox/)
MAILDIR=($TYPE =~ /maildir/)

# use mbox by default
DEFAULT="$HOME/mail"
FOLDERS="$DEFAULT/"

if ( $MAILDIR )
{
     DEFAULT="$HOME/Maildir"
     FOLDERS="$DEFAULT/."
}

if ( $MAILDIR )
{
     `${MAILDIRMAKE} "$DEFAULT"`
     if ( $RETURNCODE == 0 )
     {
         # notify the user when new folders are created
         NEWFOLDERMSG="${FOLDERS} folder created"
         `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" $LOGNAME`
     }
}

# filter out duplicate messages
`${REFORMAIL} -D 8192 $HOME/.duplicate.cache`
if ( $RETURNCODE == 0 )
{
     log "File: (duplicate)     ($SIZE)\n"
     exit
}

if 
(/^(To|Cc|Envelope-To|Delivered-To):.*(miro\.rovis at croatiafidelis\.hr)/)
{
     MYFOLDERDOT="$MATCH2"
     MYFOLDER=`echo $MYFOLDERDOT|sed 's/\.//g'`

     if ( $MAILDIR )
     {
         `${MAILDIRMAKE} -f "$MYFOLDER" "$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}$MYFOLDER list folder created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
     }

# handle mailing list messages automatically
if ( /^List-Id:.*<([0-9A-Za-z_\.\-]+)/ )
  {
     LISTNAMEDOT="$MATCH1"
     LISTNAME=`echo $LISTNAMEDOT|sed 's/\.//g'`

     # don't create a folder for Mailman status messages, just deliver 
them
     if ( $LISTNAME =~ /Mailman/ )
         to $DEFAULT

     if ( $MAILDIR )
     {
         `${MAILDIRMAKE} -f "$MYFOLDER.$LISTNAME" "$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}$MYFOLDER.$LISTNAME list folder 
created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
     }
     to ${FOLDERS}$MYFOLDER.$LISTNAME
  }
  to ${FOLDERS}$MYFOLDER
}

if (/^(To|Cc|Envelope-To|Delivered-To):.*(m.rovis at inet\.hr)/)
{
     MYFOLDERDOT="$MATCH2"
     MYFOLDER=`echo $MYFOLDERDOT|sed 's/\.//g'`

     if ( $MAILDIR )
     {
         `${MAILDIRMAKE} -f "$MYFOLDER" "$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}$MYFOLDER list folder created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
     }

  if (/^Return-Path:.*@(facebookmail\.com)/)
  {
     MY2FOLDERDOT="$MATCH1"
     MY2FOLDER=`echo $MY2FOLDERDOT|sed 's/\.//g'`

     if ( $MAILDIR )
     {
         `${MAILDIRMAKE} -f "${MYFOLDER}.${MY2FOLDER}" "$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}${MYFOLDER}.${MY2FOLDER} list 
folder created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
     }

# handle Facebook groups/lists messages automatically
if ( /^List-Id:.*<([0-9A-Za-z_\-]+)\.+/ )
    {
     LIST2NAME="$MATCH1"

     # don't create a folder for Mailman status messages, just deliver 
them
     if ( $LIST2NAME =~ /Mailman/ )
         to $DEFAULT

     if ( $MAILDIR )
      {
         `${MAILDIRMAKE} -f "${MYFOLDER}.${MY2FOLDER}.$LIST2NAME" 
"$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}${MYFOLDER}.${MY2FOLDER}.$LIST2NAME 
list folder created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
      }
     to ${FOLDERS}${MYFOLDER}.${MY2FOLDER}.$LIST2NAME
    }
   to ${FOLDERS}${MYFOLDER}.$MY2FOLDER
  }

  if (/^(To|From):.*(childrenofmedugorje at gmail\.com)/)
  {
     MY2FOLDERDOT="$MATCH2"
     MY2FOLDER=`echo $MY2FOLDERDOT|sed 's/\.//g'`

     if ( $MAILDIR )
     {
         `${MAILDIRMAKE} -f "${MYFOLDER}.${MY2FOLDER}" "$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}${MYFOLDER}.${MY2FOLDER} list 
folder created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
     }
   to ${FOLDERS}${MYFOLDER}.$MY2FOLDER
  }
if ( /^From:\s*EWTN\s*<(wings at ewtn.com)>/ )
{
     MY2FOLDERDOT="$MATCH1"
     MY2FOLDER=`echo $MY2FOLDERDOT|sed 's/\.//g'`

     if ( $MAILDIR )
     {
         `${MAILDIRMAKE} -f "${MYFOLDER}.${MY2FOLDER}" "$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}${MYFOLDER}.${MY2FOLDER} list 
folder created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
     }
     to ${FOLDERS}${MYFOLDER}.$MY2FOLDER
  }

# handle mailing list messages automatically
if ( /^List-Id:.*<([0-9A-Za-z_\.\-]+)/ )
  {
     LISTNAMEDOT="$MATCH1"
     LISTNAME=`echo $LISTNAMEDOT|sed 's/\.//g'`

     # don't create a folder for Mailman status messages, just deliver 
them
     if ( $LISTNAME =~ /Mailman/ )
         to $DEFAULT

     if ( $MAILDIR )
     {
         `${MAILDIRMAKE} -f "$MYFOLDER.$LISTNAME" "$DEFAULT"`
         if ( $RETURNCODE == 0 )
         {
             # notify the user when new folders are created
             NEWFOLDERMSG="${FOLDERS}$MYFOLDER.$LISTNAME list folder 
created"
             `${ECHO} "$NEWFOLDERMSG" | ${MAIL} -s "$NEWFOLDERMSG" 
$LOGNAME`
         }
     }
     to ${FOLDERS}$MYFOLDER.$LISTNAME
  }
  to ${FOLDERS}$MYFOLDER
}
----%<--------------------------------------------------------

Here is my .muttrc, to show nearly all the components of my setup, 
which I
hope to be using for mail.. I really understand only now, that the 
basis is
really e-mail, not the browsers, or other things, for the life on the 
internet... That the basis is really e-mail these days on Russia Today 
said also Ladar Levison, the Lavabit.com founder that fought for his 
users' privacy all the way to losing his business. I admire people of 
such integrity and humanity! Not like the Canadian Hushmail which sold 
NSA's whistleblower Thomas Drake... Who says I don't like people from 
NSA. I highly regard him, and Ross Tice, and some others!
It's the usual perception most of people have, that email is less 
important/less advanced, whatnot, not just me, but not delving into it 
here.
I'm yet to see if my "broader picture" provoked any angry reactions... 
(Note while proofreading: No reactions, neither good nor bad, I just 
saw.)
But I hope this will be technical enough now and to the completion of 
this thread...
I haven't found anywhere other then on the www.wonkity.com, the address 
that I gave above, examples for maildrop, so I hope other users will 
find mine useful, which will be source of content for me... I invested 
time and effort to arrange those lines.
Just let me give you the .muttrc (and completely without the unrelated 
settings for this story), because the dominant talk is about the 
maildrop, not the other two configuration files.

----%<--------------------------------------------------------
$ cat ~/.muttrc

set mbox_type="maildir"
set spoolfile=imaps://me@mybox2/
set folder=imaps://me@mybox2/

set ssl_starttls=yes
set ssl_force_tls=yes
set imap_passive

----%<--------------------------------------------------------

What I got with that maildrop in my Dovecot, and it would need some 
functions set, because I used too much pastion to get same work repeated 
on different input, but I don't know hot to do it, and it is by no means 
urgent, since I achived all the refiltering and in the way I wanted it, 
is I decided I wanted all mail in folders by the e-mail address by which 
I recieve them, and used the usual List-Id header to sort mails into up 
to two levels deeper, exactly for the famous yukky Stasimail, oh, sorry! 
I meant zucky Facebookmail.

Also it would be great to check for existance of a foled before 
attemting to create it... It hasn't been yet urgent to me, and I am 
rather exhausted now, not only from this coding, but also because I got 
the tumbleweed pollen allergy (American native plant, brought here as 
ornamental plant in Columbo's time, but nauseating/suffocating (but 
rarely to death), fever causing to lots of people in Europe, through 
allergy, so I was also bed-ridden much of the time... and I am not well 
neither as I am writing this, so not much of proofreading before sending 
this.

And to check for existance I guess it's with ... ~= ( some expression) 
of folders, so not to get an error on every fail attempt at creating 
folders

as Warren Block explained here:
http://www.wonkity.com/~wblock/docs/html/maildrop.html#_em_maildir_em_compatibility

, and maybe improve those lines by using some foreach loop or some such 
means...

Ah, notice that I start with no folder whatsoever, neither mbox not 
maildir type, I let the script do it all in my /home/mr!

The .mailfilter settings above for maildrop upon completion of 
refiltering gave me, in my mutt, not going back to no GUIs no more, 
thanx so kindly!, but no love lost there, with all your Java and 
Javascript and stinking popup windows out of nowhere and rummiging in my 
machine which I in no way provoked, let alone initiated, and most 
certainly not acquiesced to... So this is, with a few copy-pastes (just 
the Mutt fine colors will be missing):


y:Exit  c:Chdir  m:Mask  ?:Help
  1     IMAP +                                INBOX/
  2     IMAP +                                
mirorovis at croatiafidelishr/
  3     IMAP +                                mirovis at inethr/
  4     IMAP +                                mrovis at inethr/

  /\        /\        /\        /\        /\        /\        /\        
/
/  \      /  \      /  \      /  \      /  \      /  \      /  \      /
     \  /      \  /      \  /      \  /      \  /      \  /      \  /
      \/        \/        \/        \/        \/        \/        \/

-- Mutt: Directory [imaps://me@mybox2/], File mask: !^\.[^.]



y:Exit  c:Chdir  m:Mask  ?:Help
  1     IMAP                                  ../
  2     IMAP                                  dovecotdovecotorg

  /\        /\        /\        /\        /\        /\        /\        
/
/  \      /  \      /  \      /  \      /  \      /  \      /  \      /
     \  /      \  /      \  /      \  /      \  /      \  /      \  /
      \/        \/        \/        \/        \/        \/        \/

-- Mutt: Directory [=mirorovis at croatiafidelishr/], File mask: !^\.[^.]



y:Exit  c:Chdir  m:Mask  ?:Help
  1     IMAP                                  ../
  2     IMAP                                  
dmo-discussiondeb-multimediaorg
  3     IMAP                                  gnewsense-usersnongnuorg
  4     IMAP                                  helpdeskiskonhr

  /\        /\        /\        /\        /\        /\        /\        
/
/  \      /  \      /  \      /  \      /  \      /  \      /  \      /
     \  /      \  /      \  /      \  /      \  /      \  /      \  /
      \/        \/        \/        \/        \/        \/        \/

-- Mutt: Directory [=mirovis at inethr/], File mask: !^\.[^.]



y:Exit  c:Chdir  m:Mask  ?:Help
  1     IMAP                                  ../
  2     IMAP                                  
alsa-userlistssourceforgenet
  3     IMAP                                  
blfs-supportlinuxfromscratchorg
  4     IMAP                                  
childrenofmedugorje at gmailcom
  5     IMAP                                  cinelerraskolelinuxno
  6     IMAP                                  cjeniklosthr
  7     IMAP                                  
dailydavelistsimmunityinccom
  8     IMAP                                  
debburn-devellistsaliothdebianorg
  9     IMAP +                                facebookmailcom/
10     IMAP                                  ffmpeg-userffmpegorg
11     IMAP                                  gnewsense-usersnongnuorg
12     IMAP                                  listalosthr
13     IMAP                                  mencoder-usersmplayerhqhu
14     IMAP                                  mplayer-usersmplayerhqhu
15     IMAP                                  userslistsclaws-mailorg
16     IMAP                                  wings at ewtncom
17     IMAP                                  
wireshark-announcewiresharkorg
18     IMAP                                  
wireshark-userswiresharkorg

  /\        /\        /\        /\        /\        /\        /\        
/
      \/        \/        \/        \/        \/        \/        \/

-- Mutt: Directory [=mrovis at inethr/], File mask: !^\.[^.]



It's this last directory, i.e. this e-mail address that has the most of 
the folders.
So I'll show how I got all the Facebookmail in one folder, and devided 
by its' groups/lists you name what they are.


y:Exit  c:Chdir  m:Mask  ?:Help
  1     IMAP                                  ../
  2     IMAP                                  109806486481
  3     IMAP                                  114674985270726
  4     IMAP                                  116954758434576
  5     IMAP                                  132773866800887
  6     IMAP                                  141591592620540
  7     IMAP                                  147623871992079
  8     IMAP                                  148721478521575
  9     IMAP                                  158927160843505
10     IMAP                                  166374096744320
11     IMAP                                  177486422318814
12     IMAP                                  178566045493622
13     IMAP                                  191938787541916
14     IMAP                                  195731330459694
15     IMAP                                  199083346818377
16     IMAP                                  203424869671315
17     IMAP                                  248203298232
18     IMAP                                  259057640810411
19     IMAP                                  312893169439
20     IMAP                                  37080471529
21     IMAP                                  77789534513
22     IMAP                                  australiacroatiaunited
23     IMAP                                  domovinausrcu
24     IMAP                                  hrperoistine
25     IMAP                                  hrvatirimokatolici
26     IMAP                                  hrvatskastraza
27     IMAP                                  nacionalist
28     IMAP                                  stopkrscanofobiji
29     IMAP                                  zakonoovrsi

-- Mutt: Directory [=mrovis at inethr/facebookmailcom/], File mask: 
!^\.[^.]

I didn't need any rudimentary ASCII drawing to represent the blank 
lines cut out, because this is all the Stasi lists/groups here. The fact 
that some are named in numbers and not in [a-z] is only because that is 
the first part of the address, before @facebookmail.com part. Not 
touching that, no need to spend time on those...

If there is some user who would like to have any more explanaion on 
this, I can try and tell more, but I know most of the users here know 
more and can do better that I, so I leave the talk now. I'll take a few 
looks back to see if I got any replies, just I can't do it in very 
frequent periods at all.

I also have a few issues probably for the Mutt people to solve, the 
issues I suspect are with Mutt, but they are not urgent either.

And also, not major issue, I tried sticking  :DIRNAME=mAildir to the 
end of my mail_location line, but it simply didn't work, I mean, as per:

http://wiki2.dovecot.org/MailLocation/Maildir

I will try and get the stunnel to get my outgoing mail on the 465 port 
with TLS, but that is not an issue related here at all I think.

Thanks again to you fine developers for these great programs. Timo 
Sirainen, you are greatly appreciated here, and my respect for your fine 
country Finland, and for your nation!

Miroslav Rovis
Zagreb, Croatia



More information about the dovecot mailing list