Glean all from addresses from a users mailbox?

Austin Witmer

1 Jul 2024 1 Jul '24

4:01 a.m.

Hello all!

Is there a quick and easy way to search through an entire mailbox for a user on my dovecot server and glean all the “from” email addresses?

This user would like a record of all the email addresses who have contacted him.

Thanks in advance for your ideas.

Austin Witmer

Show replies by date

Aki Tuomi

1 Jul 1 Jul

7:55 a.m.

...

On 01/07/2024 04:01 EEST Austin Witmer via dovecot <dovecot@dovecot.org> wrote:

Hello all!

Is there a quick and easy way to search through an entire mailbox for a user on my dovecot server and glean all the “from” email addresses?

This user would like a record of all the email addresses who have contacted him.

Thanks in advance for your ideas.

Austin Witmer

doveadm fetch -u user hdr.from all

Aki

Paul Kudla (SCOM.CA Internet Services Inc.)

2:28 p.m.

please find python imap script below i use to make csv files etc

it will search all folders, sub folders etc and make a file based on display name and email address, i had started on a dav cal format but csv is all that is supported at this time. it was also designed to import hotmail address books for conversion?

was origionally intended to import to thunderbird.

it will give you a starting point.

#!/usr/local/bin/python2 # encoding: utf-8

import sys, os, socket,traceback import getopt,base64 import time,datetime from datetime import date import select from urllib import unquote_plus

import psycopg2,imaplib from email.header import decode_header import imaplib2 from threading import *

from lib import *

from optparse import OptionParser

USAGE_TEXT = '''
usage: %%prog %s[options] '''

parser = OptionParser(usage=USAGE_TEXT % '', version='0.4') parser.add_option("-e", "--email", dest="email_address", help="Email Adress to Process") parser.add_option("-o", "--outlook", dest="hotmail_import_from", help="Filespec for Hotmail Address Book") parser.add_option("-p", "--password", dest="password", help="Email Account Password") parser.add_option("-f", "--filespec", dest="file_out", default = 'addressbook.csv', help="Output File") parser.add_option("-v", "--vcard", dest="vcard", default = False, action = 'store_true', help="Output File type vCard") options, args = parser.parse_args()

if options.email_address == None : print 'No Email Specified ..., Exiting' sys.exit()

if options.password == None : #Go get the password print 'Email Account Not Found, Exiting ....' sys.exit()

print 'Processing Email : %s' %options.email_address print 'Sending To : %s' %options.file_out print 'Processing Hotmail Addressbook : %s' %options.hotmail_import_from

print 'Logging in to Account : %s with Password : %s\n\n' %(options.email_address,options.password)

# Set the following two lines to your creds and server M = imaplib2.IMAP4("mail.local.scom.ca") status = M.login(options.email_address,options.password) # We need to get out of the AUTH state, so we just select # the INBOX. if 'OK' not in status : print 'Bad Username or Password : %s / %s' %(options.uemail_address,options.password) sys.exit()

address_book = []

#Ok import hotmail address book into system if avaliable

if options.hotmail_import_from != None : #go get the file in csv format f = open(options.hotmail_import_from,'r') data = f.read() f.close

     #Now start importing the data

     data = data.split('\r\n')

     #print data
     #sys.exit()

     for n in range ( 1,len(data) ) :
             entry = data[n]
             entry = entry.split(',')
             #print
             #print n,entry[0]


             try :
                     email_address = entry[8].lower()
                     display_name = entry[0] + ' ' + entry[2]

                     print 'Importing Email          : %s'

%email_address print 'Importing Display Adress : %s' %display_name

                     #sys.exit()

                     b = []
                     b.append( email_address )
                     b.append( display_name )
                     b.append( display_name.split(' ')[0] )

                     try :
                             b.append( display_name.split(' ')[1] )
                     except:
                             b.append( display_name )

             except :
                     pass

#sys.exit()

#Go get the folder list

for i in M.list()[1]: mbox = (i.split('"/" ')[1]) print '\nProcessing Mbox : %s' %mbox select_mbox = M.select(mbox) if 'OK' not in select_mbox : print 'Error on MBOX : %s, skipping ...' %mbox continue #sys.exit()

     result, data = M.search(None, "ALL")
     ids = data[0] # data is a list.
     id_list = ids.split() # ids is a space separated string
     #print id_list
     #print
     #print
     #Ok process each email one at a time trying to look up a match


     for ii in range( 0,len(id_list) ):
             id = id_list[ii]
             print
             print 'Processing Message ID Number : %s' %str(id)
             result, message_header = M.fetch(id, '(BODY.PEEK[HEADER])')
             #print message_header
             print
             #print
             message_header = str(message_header[0]).split('\\r\\n')
             print 'Message Header for ID : %s:\n' %id
             for n in range (0,len(message_header)) :
                     a = message_header[n]

                     if a[0:4] == 'To: ' or a[0:6] == 'From: ' :

                             if 'To: ' in a :
                                     a = a.split('To: ')[1]
                             else :
                                     a = a.split('From: ')[1]

                             print '\nFound Email Address Data : %s' %a

                             if '&lt;' in a and '>' in a : #ok we have

a header that has a Display Address display_name = a.split('<')[0].rstrip() print display_name email_address = str(a.split('<')[1].split('>')[0]) print email_address

                             else :
                                     email_address = str (a)
                                     display_name = email_address


                             #Do i need to decode UTF?/ISO-1




                             if '=?utf-8' in email_address or \
                                     '=?ISO-8859-1?' in

email_address or
'=?iso-8859-1?' in email_address or
'=?Windows-1252?' in email_address or
'=?UTF-8' in email_address : #found a utf/iso header to decode) print 'Decoding UTF-8 Email Address ...' email_address = decode_header(email_address)

                                     if '=?ISO-8859-1?' in

email_address or '=?iso-8859-1?' in email_address : email_address = email_address.decode('iso-8859-1').encode('utf8') #convert iso->>utf

                                     if '=?Windows-1252?' in

email_address : email_address = email_address.decode('Windows-1252').encode('utf8') #convert windows->>utf

                                     email_address =

utftoascii(email_address).ascii

                             if '=?utf-8?' in display_name or \
                                     '=?ISO-8859-1?' in display_name

or
'=?iso-8859-1?' in display_name or
'=?Windows-1252?' in display_name or
'=?UTF-8' in display_name : #found a utf header to decode) print 'Decoding UTF-8 Display Name : %s' %display_name

                                     #Dump the quotes
                                     display_name =

display_name.replace('"','')

                                     print 'Non-Quoted Display Name

UTF? : %s' %display_name

                                     display_name =

decode_header(display_name)

                                     print display_name

                                     if '=?ISO-8859-1?' in

display_name or '=?iso-8859-1?' in display_name : display_name = display_name.decode('iso-8859-1').encode('utf8') #convert iso->>utf #print 'Converted from ISO : %s' %display_name

                                     if '=?Windows-1252?' in

display_name : display_name = display_name.decode('Windows-1252').encode('utf8') #convert iso->>utf #print 'Converted from Windows-1252 : %s' %display_name

                                     display_name =

utftoascii(display_name[0][0]).ascii

                                     print 'UTF->ASCII : %s'

%display_name

                                     display_name = ''.join([x for x

in display_name if ord(x) < 127])

                                     print 'Display Name (Stripped)

: %s' %display_name

                                     #sys.exit()

                             email_address =

str(email_address.split(',')[0]).lower() #just get the first email address?

                             display_name = display_name.replace('"','')
                             display_name = display_name.replace("'","")

                             #Now check email address is spam/valid etc
                             if 'bounces' in email_address or \
                                     'noreply' in email_address or \
                                     'no-reply' in email_address or \
                                     '+' in email_address or \

len(email_address.split('@')[0]) > 32 :

                                     continue


                             #print 'Looking for       : %s'

%email_address #print 'With Display Name : %s' %display_name

                             #sys.exit()

                             #if email_address != 'claudia@edgetec.ca' :
                             #       continue

                             #go see if email already in array, if

so skip, if email in array but no display (same as email), update display skip = False for nx in range(0,len(address_book)) : #Go get current entries add_data = address_book[nx]

                                     #print 'With Email Info   : %s'

%add_data[0] #print 'With Display Info : %s' %add_data[1]

                                     if add_data[0] == email_address

: #found a duplicate email address, go see if we need display if add_data[1] != email_address : skip = True #print 'Found Email Address but Display Name Seems OK' break else : #ok found email, update display_name, then break #print 'Email Address Found, Processing Better Display Name'

address_book[nx][1] = display_name skip = True break

                                             sys.exit()


                             if skip == True :
                                     #Skip dup entry
                                     continue


                             email_address = email_address.split(',')[0]

                             if display_name == '' :
                                     display_name = email_address

                             email_address = email_address.split(',')[0]


                             if display_name == '' or email_address

== '' : print 'Blank EMail or Display Name, Skipping ....'

                                     continue

                             print '\nCreating Address Book Entry

with :\n\nEmail : %s\nDisplay Name : %s' %(email_address,display_name)

                             if display_name[0] == '=' :
                                     print 'Break Bad'
                                     sys.exit()



                             b = []
                             b.append( email_address )
                             b.append( display_name )
                             b.append( display_name.split(' ')[0] )

                             try :
                                     b.append( display_name.split('

')[1] ) except: b.append( display_name )

                             #print b
                             address_book.append(b)
                             #print address_book

M.logout()

#print address_book

#Write the Address Book

#Am i making a Vcard file ?

if options.vcard == True : # yes f = open(options.file_out,'w')

else : #CSV File f = open(options.file_out,'w') f.write('Primary Email\tDisplay Name\tFirst Name\tLast Name\n')

     for n in range(0,len(address_book)) :
             f.write(str( address_book[n][0] + '\t' +

address_book[n][1] + '\t' + address_book[n][2] + '\t' + address_book[n][3] + '\n') ) #tab delimited

     f.close()

''' VCARD Formats

BEGIN:VCARD VERSION:3.0 PRODID:-//Apple Inc.//iOS 17.0//EN N:Hanna;Ed;;; FN:Ed Hanna ITEM1.EMAIL;TYPE=work;PREF=1:ed.hanna@eks.ca ITEM2.EMAIL;TYPE=work:ed.hanna@electrokinetic.ca ITEM3.TEL;TYPE=pref:16133792289 ITEM3.X-ABLABEL:Cottage ITEM4.TEL:1 (647) 256-3460 ITEM4.X-ABLABEL:DSS Management TEL;TYPE=HOME,VOICE:1 (647) 256-3472 ITEM5.TEL:+14035372392 ITEM5.X-ABLABEL:Calgary TEL;TYPE=IPHONE,CELL,VOICE:+19057670409 ITEM6.ADR;TYPE=HOME,pref:;;;;;;Canada ITEM6.X-ABADR:ca ITEM7.URL;TYPE=pref:https://maps.apple.com/?ll=44.489801,-77.027956&q=Dropp ed%20Pin&t=h ITEM7.X-ABLABEL:_$!<HomePage>!$_ UID:621429c9-f0d8-4dc0-bc8b-6561176e85b1 X-IMAGETYPE:PHOTO REV:20230802T165327Z EMAIL;TYPE=work:ed.hanna@dssmgmt.com EMAIL;TYPE=work:ed.hanna@ekst.ca EMAIL;TYPE=work:ed.hanna@ek-solutions.ca END:VCARD

'''

Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.)

Have A Happy Sunday !!!

Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3

Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca

On 2024-07-01 12:55 a.m., Aki Tuomi via dovecot wrote:

...

...
On 01/07/2024 04:01 EEST Austin Witmer via dovecot <dovecot@dovecot.org> wrote:

Hello all!

Is there a quick and easy way to search through an entire mailbox for a user on my dovecot server and glean all the “from” email addresses?

This user would like a record of all the email addresses who have contacted him.

Thanks in advance for your ideas.

Austin Witmer

doveadm fetch -u user hdr.from all

Aki

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

175

Age (days ago)

175

Last active (days ago)

List overview

2 comments

3 participants

participants (3)

Aki Tuomi
Austin Witmer
Paul Kudla (SCOM.CA Internet Services Inc.)