[Dovecot] Sieve > Pigeonhole > external storage with LDAP or other data source available to dovecot

[Martin F. Foster]

> On 5.1.2010, at 10.02, Martin F. Foster wrote:
>
>    
>> I am looking at porting and generalizing an old in-house patch that I'm using for the CMU Sieve plugin.  It allows sieve script to test&  lookup arguments from LDAP.
>>      
> ..
>    
>> Related work:
>>
>>   1. Pigeonhole low priority TODO would like to implement alternate
>>      script storage, eg: LDAP&  SQL.  I'm not immediately interested in
>>      alternate types of script storage, but for what I want to acheive,
>>      I need to sanely access at least an LDAP directory.
>>   2. draft-ietf-sieve-external-lists
>>      (http://tools.ietf.org/html/draft-ietf-sieve-external-lists-01)
>>      proposes a mechanism to pull mailing list addresses from external
>>      storage mechanisms such as LDAP, ACAP or relational databases.      I'm interested in this, but would like to extend this
>>      functionality beyond just lists as the example above demonstrates.
>>      
> I haven't looked into that draft or thought much about this, but would doing the lookups via Dovecot's lib-dict be ok? That would of course need a dict-ldap backend implemented, but it would be a generic way to solve this, if its API is good enough.
>    

After another day of looking into this, I believe that there are two 
concerns to be addressed:
1. dovecot implementation: how dovecot/pigeonhole provides access to 
these external sources in a sane way. I do like the dict interface for 
this, which could keep LDAP url's (or SQL queries, etc) out of sieve 
scripts, and into nicely compartementalized configuration files. Being 
very ldap focussed at my site, I hadn't looked at dict before today.
2. sieve language: an extension which would allow scripts to get 
information from an external source. I'm asking the 
draft-ietf-sieve-external-lists authors a few questions about this.


The end result would be something like what follows:

Dovecot implementation (proposed):
-------------------------------------

<dovecot.conf>
dict {
dict_sieve = ldap:/etc/dovecot/dovecot-sieve-ldap.conf
}

plugin {
# TODO: proxy this dict?
sieve_extsrc = dict
}
</dovecot.conf>

then I'm not sure how you specify multiple hosts with the dict-sql 
format, so I'm using the postfix convention. there's probably a better 
way for dovecot.
<dovecot-sieve-ldap.conf>
map {
name = responder_mode
server_host = ldap://server1.mycorp.com, ldap://server2.mycorp.com
search_base = o=mailstuff
query_filter = (&(objectclass=mailschema)(uid=%u))
result_attribute = responder_mode
result_scope = sub
bind = no
version = 3
}
map {
name = responder_text
server_host = ldap://server1.mycorp.com, ldap://server2.mycorp.com
search_base = o=mailstuff
query_filter = (&(objectclass=mailschema)(uid=%u))
result_attribute = responder_text
result_scope = sub
bind = no
version = 3
}
map {
name = forward_keep
server_host = ldap://server1.mycorp.com, ldap://server2.mycorp.com
search_base = o=mailstuff
query_filter = (&(objectclass=mailschema)(uid=%u))
result_attribute = mailForwardKeep
result_scope = sub
bind = no
version = 3
}
map {
# in this case we can have many responses
name = forward_addresses
server_host = ldap://server1.mycorp.com, ldap://server2.mycorp.com
search_base = o=mailstuff
query_filter = (&(objectclass=mailschema)(uid=%u))
result_attribute = mailForwardAddress
result_scope = sub
bind = no
version = 3
}
</dovecot-sieve-ldap.conf>

Sieve language:
---------------------
then with proposed new functions of the sieve language (all names 
plucked out of my head 2hrs ago, would have to think about the semantics 
some more):

1. a “:list” argument that expects a string-list

2. a “:extsrc” argument that expects an <ext-name: string> and returns a 
string-list

3. a “:extsrc1” argument that expects an <ext-name: string> and returns 
a string

4. a new test that uses information that is not in the message, say 
“arbitrary [comparator] string string” (syntax needs work)


sample uses in scripts are then:

1. redirect to a mailing list sourced from ldap (or wheverver):

# mailForwardAddress attribute can be present >=0 times

redirect :list :extsrc “dict:forward_addresses”

2. decide whether to keep or discard forwarded messages (eg: after the 
above rule)

# mailForwardKeep attribute should only be present once, if at all

if arbitrary :is :extsrc1 “dict:forward_keep” “TRUE” {

keep;

}

3. decide vacation action and text based in information in an ldap dir:

# mailResponderMode & mailResponderText attrs should only be present 
once, if at all

if arbitrary :is :extsrc1 “dict:responder_mode” “alwaysreply” {

vacation :days 1 :extsrc1 “dict:responder_text";

}

if arbitrary :is :extsrc1 “dict:responder_mode” “holiday” {

vacation :days 30 :extsrc1 “dict:responder_text”;

}


... where the "dict:mapname" syntax would be dovecot implementation 
specific. could also support the URL & TAG-URI schemes as proposed by 
draft-ietf-sieve-external-lists. This would allow query URLs as a more 
general case, using whatever's configured in /etc/ldap/ldap.conf (or 
whatever the platform's ldap client lib uses). Doing this does raise 
some resource consumption concerns, hence there should probably be a 
means of restricting external queries to global scripts only (or 
sieve_before, sieve_after), and a means of providing timeout values, etc.

eg:

if arbitrary :is :extsrc1 “ldap:///o=myorg?mailForwardKeep” “TRUE” {

keep;

}



I'll keep on thinking about the design, staring intently at the dovecot 
sources and http://wiki.dovecot.org/Design for another couple days 
before attempting to write some code to test this. Feedback on the 
design of both the sieve language extension, & dovecot interface very 
appreciated.

Cheers,

-Martin Foster