[Dovecot] Vpopmail + Expire Plugin
Hi,
I've changed my running qmail/vpopmail/procmail/courier-imap/sqwebmail installation to qmail/vpopmail/siege/dovecot/horde imp.
So far, I've got everything running. Authentification with vpopmail as passdb is working, Mail delivery with siege into different Mail folders is working as well.
One of the reasons why I switched to dovecot was the expire plugin and right now I'm trying to set it up but I don't get it to run.
I created a MySQL User as advised
I created a database and a table and granted SELECT, INSERT, DELETE, UPDATE privileges for that table to user dovecot
# mysql -udovecot -p dovecot Enter password: mysql> desc expires; +--------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+--------------+------+-----+---------+-------+ | username | varchar(75) | NO | PRI | NULL | | | mailbox | varchar(255) | NO | PRI | NULL | | | expire_stamp | int(11) | NO | | NULL | | +--------------+--------------+------+-----+---------+-------+ 3 rows in set (0.00 sec)
mysql> select * from expires; Empty set (0.00 sec)
I added the following lines to my dovecot.conf:
dict { expire = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext }
plugin { expire = Test expire_dict = proxy::expire }
I created the dovecot-dict-sql.conf.ext
# cat /usr/local/etc/dovecot/dovecot-dict-sql.conf.ext connect = host=localhost dbname=dovecot user=dovecot password=XXXX map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire_stamp
fields { username = $user mailbox = $mailbox } } #
When restarting dovecot I had a dict error because the socket was owned by root. I changed that so the socket is owned by the user vpopmail.
Then I created a folder "Test" via IMAP for one of my accounts
Then I copied via Horde IMP (so via IMAP) a message to that new folder
I requeried dovecot.expires in my database and it is still empty
I enabled logging for my MySQL DB and with that I found out, that no connect with the "dovecot" user is made at all so of course the table is empty.
I created a siege rule to move a mail directly to that Test folder when a specific Subject is used. lda moves the mail on arrival but still no MySQL connection is made.
I've read that doveadm -A must work to get the expire plugin workinj, but it does not:
# doveadm search -A mailbox INBOX subject test doveadm(olivleh1): Error: User listing returned failure doveadm: Error: Failed to iterate through some users #
The expire plugin is loaded on startup:
Jan 04 11:35:50 imap: Debug: Loading modules from directory: /usr/local/lib/dovecot Jan 04 11:35:50 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib20_expire_plugin.so
When using doveadm I'm getting a strange error: # doveadm -v expire -A mailbox INBOX subject test doveadm(olivleh1): Fatal: Unknown command 'expire', but plugin expire exists. Try to set mail_plugins=expire Exit 89 #
I've no idea how to get it to work. I guess I need your help guys to get it working ;)
# dovecot -n # 2.0.7: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 7.3-STABLE i386 auth_debug = yes auth_verbose = yes debug_log_path = /var/log/dovecot/dovecot-debug.log dict { expire = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_uid = 89 info_log_path = /var/log/dovecot/dovecot-info.log last_valid_uid = 89 listen = 127.0.0.1 log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_plugins = " expire" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { driver = vpopmail } plugin { expire = Test 1 expire_dict = proxy::expire sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap sieve service auth { unix_listener auth-userdb { group = vchkpw mode = 0600 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0600 user = vpopmail } } ssl_cert = </usr/local/etc/dovecot/ssl/certs/dovecot.pem ssl_key = </usr/local/etc/dovecot/ssl/private/dovecot.pem userdb { args = quota_template=quota_rule=*:backend=%q driver = vpopmail } protocol lda { debug_log_path = syslog info_log_path = syslog log_path = syslog mail_plugins = " sieve" }
This message was sent using IMP, the Internet Messaging Program.
On Tue, 2011-01-04 at 11:46 +0100, Oliver Lehmann wrote:
One of the reasons why I switched to dovecot was the expire plugin and right now I'm trying to set it up but I don't get it to run.
So you have thousands of users to make expire plugin useful?
I've read that doveadm -A must work to get the expire plugin workinj, but it does not:
# doveadm search -A mailbox INBOX subject test doveadm(olivleh1): Error: User listing returned failure doveadm: Error: Failed to iterate through some users
This won't work with userdb vpopmail. I'm not sure if vpopmail API allows for a way to list all users. (I see there's a way to list one domain's users.)
You could switch from userdb vpopmail to accessing vpopmail's sql(?) database directly with userdb sql.
When using doveadm I'm getting a strange error: # doveadm -v expire -A mailbox INBOX subject test
doveadm expunge, not doveadm expire.
Hi Timo,
thanks for your reply.
Timo Sirainen <tss@iki.fi> wrote:
On Tue, 2011-01-04 at 11:46 +0100, Oliver Lehmann wrote:
One of the reasons why I switched to dovecot was the expire plugin and right now I'm trying to set it up but I don't get it to run.
So you have thousands of users to make expire plugin useful?
Nope - just me but I want an automatic deletion of old mailinglist emails which I don't want to do manually and I definitly do not like find -exec rm solutions. Are there other ways for deleting old emails instead of expire?
You could switch from userdb vpopmail to accessing vpopmail's sql(?) database directly with userdb sql.
I don't have vpopmail running with SQL - I'm using the vpopmail internal userdatabase.
When using doveadm I'm getting a strange error: # doveadm -v expire -A mailbox INBOX subject test
doveadm expunge, not doveadm expire.
Ok - point granted but still - dovecot does not fill the expires table in my database on a mail arrival which would be the first step at all before starting with doveadm, or?
This message was sent using IMP, the Internet Messaging Program.
On Tue, 2011-01-04 at 12:51 +0100, Oliver Lehmann wrote:
One of the reasons why I switched to dovecot was the expire plugin and right now I'm trying to set it up but I don't get it to run.
So you have thousands of users to make expire plugin useful?
Nope - just me but I want an automatic deletion of old mailinglist emails which I don't want to do manually and I definitly do not like find -exec rm solutions. Are there other ways for deleting old emails instead of expire?
There's no need to use expire plugin for that with v2.0. You can just run doveadm. Do you mean you only have a single user? Then it's really simple:
doveadm expunge -u user@domain mailbox dovecot-list savedbefore 30d
Zitat von Timo Sirainen <tss@iki.fi>:
doveadm expunge -u user@domain mailbox dovecot-list savedbefore 30d
I'm trying to evaluate this at first with "search" but I'm just getting nothing returned from the command. I've a folder "Test" with many messages in it with the subject "test". I'm running the doveadm search command:
# doveadm -Dv search -u lehmann@ans-netz.de mailbox Test subject 'test'
doveadm(user): Debug: Loading modules from directory: /usr/local/lib/dovecot
doveadm(user): Debug: Module loaded:
/usr/local/lib/dovecot/lib20_expire_plugin.so
doveadm(user): Debug: Loading modules from directory:
/usr/local/lib/dovecot/doveadm
doveadm(user): Debug: Module loaded:
/usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so
doveadm(user): Debug: Skipping module doveadm_quota_plugin, because
dlopen() failed:
/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so:
Undefined symbol "quota_user_module" (this is usually intentional, so
just ignore this message)
doveadm(user): Debug: Skipping module doveadm_zlib_plugin, because
dlopen() failed:
/usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: Undefined
symbol "i_stream_create_deflate" (this is usually intentional, so just
ignore this message)
doveadm(user): Debug: expire: Iterating only a single user, ignoring
expire database
doveadm(lehmann@ans-netz.de): Debug: auth input: lehmann@ans-netz.de
uid=89 gid=89 home=/usr/local/vpopmail/domains/ans-netz.de/lehmann
doveadm(lehmann@ans-netz.de): Debug: Effective uid=89, gid=89,
home=/usr/local/vpopmail/domains/ans-netz.de/lehmann
doveadm(lehmann@ans-netz.de): Debug: maildir++:
root=/usr/local/vpopmail/domains/ans-netz.de/lehmann, index=,
control=, inbox=/usr/local/vpopmail/domains/ans-netz.de/lehmann
#
As you can see - only debug information - no messages listed. Attached you'll find the telnet output to show that there are messages:
# telnet 0 143 Trying 0.0.0.0... Connected to 0. Escape character is '^]'.
- OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE STARTTLS AUTH=PLAIN] Dovecot ready. . login lehmann@ans-netz.de password . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND
UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1
CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH
LIST-STATUS] Logged in . SELECT "Test" - FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)]
Flags permitted. - 7 EXISTS
- 2 RECENT
- OK [UIDVALIDITY 1294061093] UIDs valid
- OK [UIDNEXT 8] Predicted next UID
- OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. . fetch 1:7 (body[header.fields (subject)])
- 1 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {17} Subject: test
)
- 2 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {17} Subject: test
)
- 3 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {17} Subject: test
)
- 4 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {17} Subject: test
)
- 5 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {17} Subject: test
)
- 6 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {17} Subject: test
)
- 7 FETCH (BODY[HEADER.FIELDS (SUBJECT)] {17} Subject: test
) . OK Fetch completed.
This message was sent using IMP, the Internet Messaging Program.
On Tue, 2011-01-04 at 13:13 +0100, Oliver Lehmann wrote:
doveadm(lehmann@ans-netz.de): Debug: Effective uid=89, gid=89,
home=/usr/local/vpopmail/domains/ans-netz.de/lehmann doveadm(lehmann@ans-netz.de): Debug: maildir++:
root=/usr/local/vpopmail/domains/ans-netz.de/lehmann, index=,
control=, inbox=/usr/local/vpopmail/domains/ans-netz.de/lehmann
So the mails exist in /usr/local/vpopmail/domains/ans-netz.de/lehmann/.Test/cur/ directory?
You could also try if doveadm sees any mailboxes:
doveadm mailbox list -u lehmann@ans-netz.de
Timo Sirainen <tss@iki.fi> wrote:
On Tue, 2011-01-04 at 13:13 +0100, Oliver Lehmann wrote:
doveadm(lehmann@ans-netz.de): Debug: Effective uid=89, gid=89, home=/usr/local/vpopmail/domains/ans-netz.de/lehmann doveadm(lehmann@ans-netz.de): Debug: maildir++: root=/usr/local/vpopmail/domains/ans-netz.de/lehmann, index=, control=, inbox=/usr/local/vpopmail/domains/ans-netz.de/lehmann
So the mails exist in /usr/local/vpopmail/domains/ans-netz.de/lehmann/.Test/cur/ directory?
You could also try if doveadm sees any mailboxes:
doveadm mailbox list -u lehmann@ans-netz.de
This just gave me "dovecot.sieve", so "/Maildir" was missing from the
directory.
I added it by specifying mail_location:
mail_location = maildir:/usr/local/vpopmail/domains/%d/%n/Maildir
I've read that specifying this is not needed with vpopmail as vpopmail is used
to get the mail location but obviously it is not working 100% correctly.
With defining mail_location doveadm search works.
# doveadm search -u lehmann@ans-netz.de mailbox Test subject 'test' 4f62f61f93e9224df2830000a41df4f7 1 4f62f61f93e9224df2830000a41df4f7 2 4f62f61f93e9224df2830000a41df4f7 3 4f62f61f93e9224df2830000a41df4f7 4 4f62f61f93e9224df2830000a41df4f7 5 4f62f61f93e9224df2830000a41df4f7 6 4f62f61f93e9224df2830000a41df4f7 7 #
I could now probably set up a cron job doing the "expunging" but - what is expire now for at all? What is the benefit? I thought that expunging is only possible with expire but from what I understood now it also works without expire.
This message was sent using IMP, the Internet Messaging Program.
On Tue, 2011-01-04 at 13:29 +0100, Oliver Lehmann wrote:
doveadm mailbox list -u lehmann@ans-netz.de
This just gave me "dovecot.sieve", so "/Maildir" was missing from the
directory. I added it by specifying mail_location: mail_location = maildir:/usr/local/vpopmail/domains/%d/%n/Maildir I've read that specifying this is not needed with vpopmail as vpopmail is used to get the mail location but obviously it is not working 100% correctly. With defining mail_location doveadm search works.
Hmm. It should have worked exactly the same for imap and doveadm.
I could now probably set up a cron job doing the "expunging" but - what is expire now for at all? What is the benefit? I thought that expunging is only possible with expire but from what I understood now it also works without expire.
In v1.x it was necessary, but in v2.0 it's only an optimization for skipping users who don't have anything to expunge (which is useful when you have millions of users).
participants (2)
-
Oliver Lehmann
-
Timo Sirainen