On Wed, Apr 21, 2010 at 10:47 AM, Patrick Nagel mail@patrick-nagel.netwrote:

I think /etc/passwd is as close as it gets to your requirements... why not just add the users as system users, and set their shell to /bin/false?

There would be conflicts in this, especially with multiple domain names (sorry, forgot to mention that ... there will be about 10 domain names).

On Wed, Apr 21, 2010 at 1:30 PM, Rodolfo Gonzalez rgonzalez@gnt.cc wrote:

Phil Howard escribió:

On Wed, Apr 21, 2010 at 10:47 AM, Patrick Nagel

...

...

wrote:

I think /etc/passwd is as close as it gets to your requirements... why

...

not just add the users as system users, and set their shell to /bin/false?

There would be conflicts in this, especially with multiple domain names (sorry, forgot to mention that ... there will be about 10 domain names).

Then I think MySQL will do the job. Both postfix and dovecot support MySQL, and you can use SASL to authenticate SMTP with Dovecot, so Dovecot would do all the auth work. Finally, you could use Postfix's VDA patch if you want to use Maildir++.

Hope this helps.

I don't want to use any other server engine of any kind with this. I'm trying to keep it small and lean, and minimize what the people that have to monitor and fix it need to know. So at the present time, I am excluding all databases like any SQL or LDAP or anything else that needs to run as a daemon/engine/service.

Ideal would be a single file both can read, be it a flat file, or a Berkeley DB file. Next to that would be two files where one is authoritative and the other (for Postfix since it only needs a list) is generated from it. That looks like what I ultimately will do (the script that adds users will just generate the files and test them).

I was hoping there might be a way to get Postfix to use the legacy Unix passwd file format, but with a different file name. It doesn't seem to have that ability. It would make things very simple if it did. It would also be simple if the system /etc/passwd file could be used, but it can't.

On Wed, Apr 21, 2010 at 3:30 PM, Thomas Leuxner tlx@leuxner.net wrote:

I'm running a setup that should be good enough for what you are trying to achieve. All user information is stored in flat files per domain and you may override per user settings individually:

passdb { args = username_format=%u /var/vmail/auth.d/%d/passwd driver = passwd-file }

userdb { args = username_format=%u /var/vmail/auth.d/%d/passwd driver = passwd-file }

What does it take to get Postfix to read this?

$ cat passwd

user@domain.tld:{scheme}<password>:5000:5000::/var/vmail/domain.tld/user::userdb_quota_rule=*:storage=5G userdb_acl_groups=PublicMailboxAdmins

In which directory was this?

I would vote against storing aliases in these files though. Reason being the Postfix alias files are more flexible, because you would need to setup NULL password/No Login users or similar in the Dovecot backend. Another reason to keep them in Postfix is to completely separate alias management from the user management and use the same for login checks.

See how aliases are used for routing and to authenticate valid mail from senders with one file:

$ cat virtual alias@domain.tld login@domain.tld postmaster@domain.tld login@domain.tld

I suspect I will want to be maping virtuals between different domains, so I might have

abuse@example.com mailadmin@example.net abuse@example.net mailadmin@example.net postmaster@example.com mailadmin@example.net postmaster@example.net mailadmin@example.net

[main.cf]

virtual_mailbox_domains = domain.tld, domain1.tld virtual_mailbox_base = /var/vmail virtual_minimum_uid = 100 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/virtual virtual_transport = lmtp:unix:private/dovecot-lmtp […] smtpd_sender_login_maps=hash:/etc/postfix/virtual

One thing I need to watch out for, and am concerned with because the last time I used Postfix there were a bunch of "virtual" configurations that really didn't work for me for a reason I cannot recall right now ... is that the same user name in different domains is NOT always the same user. E.g. bob@example.com is NOT the same person as bob@example.net while bob@example.org doesn't even exist. So there needs to be distinct entries for bob@example.com and bob@example.net (and not any for bob@example.org and have Postfix reject that during incoming SMTP sessions).

There can also be cases where mike@example.com and mike@example.net are the same person, and Mike wants to have mail to these two addresses kept in separate mail boxes (and presumably must do separate logins, so he'd have to set up 2 accounts in his MUA) ... as well as steve@example.com and steve@example.net also being the same person, but Steve wants everything in one mailbox (so he'd have to pick between steve@example.com and steve@example.net and I'd have to set up a virtual map for the other to be delivered to the mailbox of his choice ... in a separate lookup table in Postfix).

If this seems suitable I can send more details to you.

It might well be as long the domains are fully distinct. I'll have to go read up on each of the virtual_* configuration parameters to be sure of the effects. I was thinking to use:

mailbox_command = /usr/lib/dovecot/deliver

in Postfix main.cf. Is that workable instead of "virtual_transport = lmtp:unix:private/dovecot-lmtp" Or would running LMTP be a better way?

On Thu, Apr 22, 2010 at 3:33 AM, Thomas Leuxner tlx@leuxner.net wrote:

On Wed, Apr 21, 2010 at 04:34:30PM -0400, Phil Howard wrote:

...

...

userdb { args = username_format=%u /var/vmail/auth.d/%d/passwd driver = passwd-file } What does it take to get Postfix to read this?

Basically these two parameters in 'main.cf':

[main.cf] smtpd_sasl_type=dovecot smtpd_sasl_path=private/auth

Since this will have implications when Dovecot is not running/unavailable as Authtentication Backend, Postfix will reject legit incoming mail in that case, it is better to put this in the master configuration actually and have Postfix use a dedicated submission port for SASL clients:

[master.cf] submission inet n - - - - smtpd smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth -o smtpd_sasl_security_options=noanonymous -o smtpd_sasl_local_domain=$myhostname -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtpd_sender_login_maps=hash:/etc/postfix/virtual -o smtpd_sender_restrictions=reject_sender_login_mismatch -o smtpd_recipient_restrictions=reject_unknown_recipient_domain,reject_non_fqdn_recipient,permit_sasl_authenticated,reject

So what would local_recipient_maps look like in this case? At this point, I don't understand what is happening for this. I would be expecting Postfix to be asking Dovecot if a user is valid. This is for mail incoming from outside, to make the rejection decision during the SMTP session. This looks more like a configuration to provide a submission interface and authenticate through Dovecot. That's fine, and probably what is needed. But I'm trying to sort out the local_recipient_maps at this time. Can this solve both issues at the same time?

...

read up on each of the virtual_* configuration parameters to be sure of

It might well be as long the domains are fully distinct. I'll have to go the

...

effects. I was thinking to use:

mailbox_command = /usr/lib/dovecot/deliver

in Postfix main.cf. Is that workable instead of "virtual_transport = lmtp:unix:private/dovecot-lmtp" Or would running LMTP be a better way?

LMTP would be better long-term as it is more flexible and robust, e.g. allowing multiple recipient deliveries in parallel and has a real protocol handshake compared to piping into the LDA, but both is feasible. Hower LMTP is available with Dovecot 2.0 only.

I'm doing this on Ubuntu 9.10 and it has Dovecot 1.1.11 right now (we need to get this mail server up before we will be ready to eval Ubuntu 10.04 or another distro).

On Thu, Apr 22, 2010 at 10:43 AM, Rainer Frey rainer.frey@inxmail.dewrote:

If you can't wait for Dovecot 2.0, you need to use dovecot deliver, but you should set it up as a pipe transport in master - see http://wiki.dovecot.org/LDA/Postfix for virtual users. mailbox_command again is for real system users only.

Basically what I expect to be doing is:

1. Postfix listens on SMTP for incoming MX mail to local (as in virtual, not system) users.

2. Postfix listens on Submission, encrypted only, and authenticates users to submit mail for delivery anywhere.

3. Dovecot listens on encrypted IMAPS and POP3S for user authenticated access to mailboxes.

4. Everything but MX to SMTP on port 25 shall be encrypted only. If I can force the use of STARTTLS on the non-encrypted ports, then it would be OK to use them that way. But I do not want to give any user an option to not be encrypted.

5. Passwords stored encrypted, such as MD5. And it should be a scheme that both Postfix and Dovecot can use so I don't have keep two different encryption schemes.

6. I'd prefer not to, but it looks like I will have to copy data from one format to another format so Dovecot can read it and Postfix can read it. I will most likely be using the CDB format (the constant database file format from Dan Bernstein ... which I'd think should be easy enough for a future version of Dovecot to support).

7. I am wondering if I can trick Postfix into reading virtual user info by running it chrooted where I substitute /etc/passwd and /etc/shadow with stuff I generate from Dovecot files.

On Fri, Apr 23, 2010 at 2:49 AM, Rainer Frey rainer.frey@inxmail.de wrote:

On Thursday 22 April 2010 18:15:18 Phil Howard wrote: [ ... all standard stuff that is well documented ... ]

...

5. Passwords stored encrypted, such as MD5. And it should be a scheme that both Postfix and Dovecot can use so I don't have keep two different encryption schemes.

Postfix doesn't need any password directly. It only authenticates a user with a password in one case: SMTP SASL authentication of submission users, and it uses the dovecot auth service for that and does not read the password database itself.

OK, so all that is in whatever SASL configuration, and so I'll be doing that in Dovecot.

It is not about supporting a certain database library. There is simply a

difference in what Dovecot and Postfix need from a user/recipient database. Dovecot needs information like path to mailbox, uid/gid with which to put files into the mailboxes, extra configuration fields ... Therefore it uses a structured multi-value format like the passwd-file. CDB or similar don't work like this, so dovecot can't easily support using the same CDB files as postfix.

Why not have all those fields combined into the value string you get from CDB? If it can do it from a line out of /etc/passwd then it can do it from CDB (the value could simply be that same line ... or defined to be something else more convenient).

Postfix only supports name:value tables, either to use the value

(table-style lookup) or to check whether there is an entry for a name (list-style lookup). Therefore it only uses file databases with such a mapping. In the case of valid recipients which will be handed off to dovecot for delivery, it is such a simple list lookup.

The value can be a tuple (see above). We're lucky that what Postfix needs is "does this exist". So it can just completely ignore the value and not need any code to parse it.

(If it interests you: the postfix virtual delivery agent needs very similar information to dovecot, but it uses several key:value maps for the different information, all with the recipient as key.)

I've run into that before. I think it was the source of some difficulty. In programming I've done, I've stored structured data in Berkeley DB and never needed but one table/file per class of index.

On Wednesday 21 April 2010 21:30:12 Thomas Leuxner wrote:

I'm running a setup that should be good enough for what you are trying to achieve. All user information is stored in flat files per domain and you may override per user settings individually:

passdb { args = username_format=%u /var/vmail/auth.d/%d/passwd driver = passwd-file }

userdb { args = username_format=%u /var/vmail/auth.d/%d/passwd driver = passwd-file }

$ cat passwd user@domain.tld:{scheme}<password>:5000:5000::/var/vmail/domain.tld/user::u serdb_quota_rule=*:storage=5G userdb_acl_groups=PublicMailboxAdmins

[...]

See how aliases are used for routing and to authenticate valid mail from senders with one file:

$ cat virtual alias@domain.tld login@domain.tld postmaster@domain.tld login@domain.tld

[main.cf] virtual_mailbox_domains = domain.tld, domain1.tld virtual_mailbox_base = /var/vmail virtual_minimum_uid = 100 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/virtual virtual_transport = lmtp:unix:private/dovecot-lmtp

What I don't see here at all (and neither in your Wiki Howto) is how Postfix determines the valid recipients for the domains in virtual_mailbox_domains.

The correct parameter would be virtual_mailbox_maps, but AFAIK there is no lookup table that read the passwd format from an arbitrary file. So a script that generates a hash/whatever postfix lookup file from the passwd-files would still be necessary.

Or do you use recipient validation via LMTP? (I didn't notice a reject_unverified_recipient though) This at least won't work with deliver, I'm not even sure about LMTP.

Regards Thomas

Rainer

Please honour the Reply-To header next time. Thanks.

On Thursday 22 April 2010 11:42:01 Thomas Leuxner wrote:

On Thu, Apr 22, 2010 at 11:18:09AM +0200, Rainer Frey wrote:

...

What I don't see here at all (and neither in your Wiki Howto) is how Postfix determines the valid recipients for the domains in virtual_mailbox_domains.

Postfix will expand possible aliases first and determine the final recipient handing over to Dovecot:

...

...

$ cat virtual alias@domain.tld login@domain.tld postmaster@domain.tld login@domain.tld

virtual_alias_maps = hash:/etc/postfix/virtual

Do you define all valid recipients there (e.g. in you example virtual file login@domain.tld)?

It will query the recipients by connecting to the socket in its chroot provided by Dovecot:

service auth {

This is wrong. The auth service is not queried for recipient, only for valid SASL users (that connect to the submission service as *senders*). I'm talking about determining valid *recipients* for the virtual_mailbox_domains.

[...]

Once it has the homedir it will send it off via LMTP or deliver, whichever you configured via:

virtual_transport = lmtp:unix:private/dovecot-lmtp or virtual_transport = dovecot

But this is at the delivery stage, when the mail has already been accepted. This means, if no homedir/mailbox is found, bounce mails are sent, to potentially forged senders. That is backscatter.

...

The correct parameter would be virtual_mailbox_maps, but AFAIK there is no lookup table that read the passwd format from an arbitrary file. So a script that generates a hash/whatever postfix lookup file from the passwd-files would still be necessary.

There is no such thing as a correct parameter from my perspective. I did not say that alias creation was to be unified/automated.

I'm not talking about aliases, I'm talking about recipient addresses of virtual mailboxes. You need to verify whether a mailbox exists for a recipient address in the SMTP server before accepting the message.

Instead I said I did not even think this is good practice to me. Anyone with at least a bit of sed/awk knowledge can kludge it from the flat-files anyway.

Indeed, but you offered the original poster your solution as one that "should be good enough for what you are trying to achieve", but your solution leaves out the aspect of the valid recipient list for the virtual mailbox domain address class.

...

Or do you use recipient validation via LMTP? (I didn't notice a reject_unverified_recipient though) This at least won't work with deliver, I'm not even sure about LMTP.

This is not required in the example and optional at least:

Of course, but it would be a viable alternative to a lookup table for the recipients.

Rainer

On Thu, Apr 22, 2010 at 09:48:36AM -0400, Phil Howard wrote:

The ideal would be a complete mail server package that handled it all in one ... SMTP, submission, IMAP(S), POP3(S). But what I've seen as attempts to do that so far were less than promising (even though I have no need for what most of the MTAs do).

The only one like that I know of is Courier, and I found it incredibly frustrating to work with. I use dovecot and postfix, but don't have the virtual user issue -- all of my users (for twelve domains) have real accounts.

Jim

Jim Trigg, Lord High Everything Else O- /"
\ / ASCII RIBBON CAMPAIGN Hostmaster, Huie Kin family website X HELP CURE HTML MAIL Verger, All Saints Church - Sharon Chapel / \

On Thu, Apr 22, 2010 at 01:12:24PM +0200, Rainer Frey wrote:

Do you define all valid recipients there (e.g. in you example virtual file login@domain.tld)?

Yes.

But this is at the delivery stage, when the mail has already been accepted. This means, if no homedir/mailbox is found, bounce mails are sent, to potentially forged senders. That is backscatter.

I'm not talking about aliases, I'm talking about recipient addresses of virtual mailboxes. You need to verify whether a mailbox exists for a recipient address in the SMTP server before accepting the message.

Possibly. But this could then be fixed by adding another recipient restriction, is that what is bothering you?

Indeed, but you offered the original poster your solution as one that "should be good enough for what you are trying to achieve", but your solution leaves out the aspect of the valid recipient list for the virtual mailbox domain address class.

This was not meant to say this is the ultimate one and only solution. See for recipient_restrictions esspecially, everyone may have different needs. But at least someone *may* a starting point. Feel free to refine the setup.

Of course, but it would be a viable alternative to a lookup table for the recipients.

Will look into it, but maybe you can add your thoughts how you would do.

Thomas

On Thu, 22 Apr 2010 17:03:00 +0200 Rainer rainer.frey@inxmail.de articulated:

Well, it leaves out the *one tricky part* of using a flat file database for virtual users with dovecot and postfix: there is no common format that both understand directly.

I have not been following this thread as closely as I probably should have; however, I was wondering what the OP's problem was with using MySQL? It would greatly simplify the job of constructing and maintaining databases. It is even possible to create tables that both Postfix and Dovecot can use jointly if desired. I use MySQL for several projects, and would never go back to using 'flat files" unless there was no other way to achieve my goal.

If the OP does not know how to construct a usable table in MySQL, I would be willing to work with him as I am sure would others on this list.

The thread subject implies that the OP wants the best choice for a database file system and that would be, IMHO, a SQL one.

Just my 2¢.

-- Jerry Dovecot.user@seibercom.net

Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.

On Thu, 22 Apr 2010 12:18:41 -0400 Phil ttiphil@gmail.com articulated:

The administration is going to be handed off to less technical people, and my goal is to mimize the number of elements in this. It's not about MySQL itself ... it's about not running yet another server/daemon.

Excuse my ignorance here; however, it appears that what you are trying to accomplish is exceedingly harder to implement than simply using SQL for the job. The are numerous GUIs to administer SQL with. Personally, I use "phpMyAdmin" with MySQL and have found it quite a good fit. There are several others of course, and other flavors of SQL that you might want to investigate.

I have personally set up MySQL with phpMyAdmin for some rather technically challenged individuals and they have not experienced any problems that I am aware of. There is also a PostfixAdmin http://sourceforge.net/projects/postfixadmin/ that might simplify your life. I don't use it myself; however, I have heard it is quite good.

Again, just my 2¢.

-- Jerry Dovecot.user@seibercom.net

Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.

Phil Howard put forth on 4/22/2010 11:18 AM:

On Thu, Apr 22, 2010 at 12:12 PM, Jerry dovecot.user@seibercom.net wrote:

...

On Thu, 22 Apr 2010 17:03:00 +0200 Rainer rainer.frey@inxmail.de articulated:

...

Well, it leaves out the *one tricky part* of using a flat file database for virtual users with dovecot and postfix: there is no common format that both understand directly.

I have not been following this thread as closely as I probably should have; however, I was wondering what the OP's problem was with using MySQL? It would greatly simplify the job of constructing and maintaining databases. It is even possible to create tables that both Postfix and Dovecot can use jointly if desired. I use MySQL for several projects, and would never go back to using 'flat files" unless there was no other way to achieve my goal.

The administration is going to be handed off to less technical people, and my goal is to mimize the number of elements in this. It's not about MySQL itself ... it's about not running yet another server/daemon.

With this many lookup table types supported by Postfix, is it true that it has no "simple" table type in common with Dovecot?

          btree  A sorted, balanced tree structure.  This  is
                 available on systems with support for Berke-
                 ley DB databases.

          cdb    A read-optimized structure with  no  support
                 for  incremental updates.  This is available
                 on systems with support for CDB databases.

          cidr   A table that associates values  with  Class-
                 less  Inter-Domain  Routing (CIDR) patterns.
                 This is described in cidr_table(5).

          dbm    An indexed file type based on hashing.  This
                 is available on systems with support for DBM
                 databases.

          environ
                 The  UNIX  process  environment  array.  The
                 lookup  key is the variable name. Originally
                 implemented for testing,  someone  may  find
                 this useful someday.

          hash   An indexed file type based on hashing.  This
                 is available on  systems  with  support  for
                 Berkeley DB databases.

          internal
                 A non-shared, in-memory hash table. Its con-
                 tent are lost when a process terminates.

          ldap (read-only)
                 Perform lookups  using  the  LDAP  protocol.
                 This is described in ldap_table(5).

          mysql (read-only)
                 Perform  lookups  using  the MYSQL protocol.
                 This is described in mysql_table(5).

          pcre (read-only)
                 A lookup table based on Perl Compatible Reg-
                 ular   Expressions.   The   file  format  is
                 described in pcre_table(5).

          pgsql (read-only)
                 Perform lookups using the PostgreSQL  proto-
                 col. This is described in pgsql_table(5).

          proxy (read-only)
                 A  lookup  table that is implemented via the
                 Postfix proxymap(8) service. The table  name
                 syntax is type:name.

          regexp (read-only)
                 A lookup table based on regular expressions.
                 The file format is described  in  regexp_ta-
                 ble(5).

          sdbm   An indexed file type based on hashing.  This
                 is available on  systems  with  support  for
                 SDBM databases.

          static (read-only)
                 A  table  that  always  returns  its name as
                 lookup result.  For  example,  static:foobar
                 always  returns  the string foobar as lookup
                 result.

          tcp (read-only)
                 Perform lookups using a simple request-reply
                 protocol  that is described in tcp_table(5).
                 This feature is not included with the stable
                 Postfix release.

          unix (read-only)
                 A  limited way to query the UNIX authentica-
                 tion  database.  The  following  tables  are
                 implemented:

                 unix:passwd.byname
                        The  table is the UNIX password data-
                        base. The key is a login  name.   The
                        result  is  a  password file entry in
                        passwd(5) format.

                 unix:group.byname
                        The table is the UNIX group database.
                        The  key is a group name.  The result
                        is a group  file  entry  in  group(5)
                        format.

-- Stan

Phil Howard put forth on 4/22/2010 3:07 PM:

I would think the general (insert some character here)-separated file should be doable: comma-delimited:/some/file/name colon-delimited:/some/file/name tab-delimited:/some/file/name space-delimited:/some/file/name But it hasn't been done that I can see (some third party patch not withstanding).

It's not exactly what you're asking for here, but it sounds like SQLite would fit your needs. If only both Postfix and Dovecot had an SQLite driver for user and pwd lookup. At this time I believe neither does.

well be the description given early to use Dovecot SASL to authenticate Postfix submission users would be just fine, as long as Postfix does a SOFT

This is a very populate way to integrate the two, and works well.

rejection when Dovecot isn't up, so it requeues files. On the SMTP end, I just want an up front rejection decision to avoid backscatter. The one backscatter hole where an existing user is being deleted while incoming mail is being queued is probably insignificant.

Postfix is very configurable in this regard and is smart enough to return a 4xx when it can't talk to Dovecot (or any "remote" auth provider) and a 5xx when Dovecot says unknown user. I'm not looking at docs now, but upon a brief scan earlier I recall at least 3 different rejection codes Postfix will return depending on the reason an auth attempt fails. Some of these are configurable. One nice thing about Postfix is that the documentation is _very_ thorough, even if sometimes hard to digest.

Good luck, and please keep us up to date. This is an interesting topic and the solution may be useful to others.

-- Stan

Phil Howard skrev:

On Thu, Apr 22, 2010 at 12:12 PM, Jerry dovecot.user@seibercom.net wrote:

...

On Thu, 22 Apr 2010 17:03:00 +0200 Rainer rainer.frey@inxmail.de articulated:

...

Well, it leaves out the *one tricky part* of using a flat file database for virtual users with dovecot and postfix: there is no common format that both understand directly.

I have not been following this thread as closely as I probably should have; however, I was wondering what the OP's problem was with using MySQL? It would greatly simplify the job of constructing and maintaining databases. It is even possible to create tables that both Postfix and Dovecot can use jointly if desired. I use MySQL for several projects, and would never go back to using 'flat files" unless there was no other way to achieve my goal.

The administration is going to be handed off to less technical people, and my goal is to mimize the number of elements in this. It's not about MySQL itself ... it's about not running yet another server/daemon.

Have you looked into Postfix Admin? http://postfixadmin.sourceforge.net/

It might be a good solution for you. I'm using it for a a growing database of users and I'm very happy with it. The setup with postfix, dovecot and mysql was quite straight forward, and this interface requires no particular technical know-how.

Should be perfect if you can do the initial setup, then just give the admisn the password to this interface.

Arne

On 22/04/2010 17:18, Phil Howard wrote:

...

I have not been following this thread as closely as I probably should have; however, I was wondering what the OP's problem was with using MySQL? It would greatly simplify the job of constructing and maintaining databases. It is even possible to create tables that both Postfix and Dovecot can use jointly if desired. I use MySQL for several projects, and would never go back to using 'flat files" unless there was no other way to achieve my goal.

The administration is going to be handed off to less technical people, and my goal is to mimize the number of elements in this. It's not about MySQL itself ... it's about not running yet another server/daemon.

You need to look at where you are going with this.... One way or another you need a database - call it a banana if you prefer, but it's still a database whether it's a flat file or a BDB file or whatever

Your requirements appear to be badly phrased. What you *appear* to want is a black box system which is as simple to maintain as possible.
However, you have stated a bunch of hard to meet criteria such as not allowing any long running code to support your needs (a daemon). I really don't see that you should exclude additional daemons, simply evaluate the complete system with and without and choose the one which is "easiest" to maintain

1. Yes you CAN setup something which uses a plain text database, I tried it for a while but at 50 users I decided it was too complicated and switched to a DB.

2. You are optimistic if you don't think some user will accidently add a <CR> halfway through the text file, completely breaking it and have the skills to realise what they did. Or they will add a "," in the middle of a password and change the meaning of all your fields. Or they will miss a field save the file and then never realise their mistake. Better yet are errors which cause some external (grep,sed,awk) function to choke on the input and cause some really wierd effects downstream...

3. Why should an sql database need any monitoring and maintenance over the next X years? Yes you can corrupt the files, but I would hope for very decent uptimes and after all they aren't going to repair a corrupted boot sector so you need some kind of maintenance plan in place, simply work in a full OS backup into the plan and this saves your DB at the same time? (Offlist we can discuss, but a simple rsync to some separate partition should do it)

4. If you really, really don't want a daemon based database then you will have to look at bdb (if dovecot supports it?) or sqlite which I think postfix+dovecot both support. You can then add convenience command line functions to examine and alter the data. Those convenience functions can error check the input also.

Good luck

Ed W

On Fri, Apr 23, 2010 at 5:23 AM, Ed W lists@wildgooses.com wrote:

P.S. The "idiot" who kept breaking the plain text format file in my original setup was .... da da ... me ... So given I think of myself as reasonably technical, I would claim that text format "databases" are way more fragile than you might expect.... (good luck getting a non technical user not to break the text format file from time to time...)

But this is more an issue of exposing the system to human failure than it is a choice of using a flat file. If the flat file is generated by some other system, and maybe merged with other data, and never handled by a human except in extraordinary situations, it should be fine. This is used as a justification for XML being a text format (that a human can read it and even edit it, in non-routine situations).

FYI, I have done hand editing of /etc/{group,passwd,shadow} to add users. So far I've never killed root. I still have to at times. Most recently I did so with Ubuntu because the user management tool couldn't get it right (I needed a specific UID/GID arrangement for compatibility with other systems, and the GUI tool refused ... it would let me do one or the other, but not both at the same time).

On Fri, 23 Apr 2010 09:03:02 -0400 Phil ttiphil@gmail.com articulated:

You rsync the files an SQL database (like MySQL) works from, and don't expect corruption? That's only safe if the database is synced and shut down. I don't want to be doing that. If I did run a database engine, it would have to import everything ... as a single massive transaction (or maybe a live table switch scenario between two tables). To back it up I'd either export the entire table to a file (and send that off to the archive), or just back up the file I used to import with.

There are numerous ways to export/backup a live MySQL database. I have employed several of them myself. You might want to check out this URL for starters:

http://www.noupe.com/how-tos/10-ways-to-automatically-manually-backup-mysql-...

Your statement that the only safe way to backup/export an SQL database is to stop it, etc is totally incorrect. No offense, but I question your knowledge of SQL, and MySQL in particular. If you need help with SQL, I know plenty of individuals, including myself, who would be willing to help you design a usable one for your needs.

I am somewhat perplex by you insinuation that the end users of this database system you are designing are total novices. Depending on the table:type you choose, you may very well have to invoke 'postmap' and perhaps 'postaliase' on them to make them usable by Postfix. If your intended audience is so incompetent that they cannot handle a simple SQL database, how do you expect them to handle the intricacies of getting this database you are designing serviceable?

It is my own opinion; however, I think you are basing your decision on a fear of SQL more than on simplifying the final database decision. A simple SQL would greatly simplify maintenance and security. Then again, you are going to be the one who takes the heat if this blows up in your face.

-- Jerry Dovecot.user@seibercom.net

Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.

Entropy requires no maintenance.

Markoff Chaney

On Fri, 23 Apr 2010 16:07:20 +0100 Ed lists@wildgooses.com articulated:

Err to be fair, I was about to reply and say that it's IS achievable, but on the flip side I would concede that it does require a bit of thought to make this stuff work correctly. I think you are being a bit harsh to suggest it's *that* simple to make an automated push work reliably?

Sorry, I did not mean that it was trivial, although it is not exactly rocket science either. I was simply countering the statement that the whole MySQL daemon process would have to be shut down to achieve a reliable backup/export was incorrect.

So in summary, it IS possible, but yes it does need a bit of thought, fair enough.

Again, I did not mean to offend anyone. Sorry!

-- Jerry Dovecot.user@seibercom.net

Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.

The chief enemy of creativity is "good" sense.

Picasso

On Thu, Apr 22, 2010 at 5:42 AM, Thomas Leuxner tlx@leuxner.net wrote:

On Thu, Apr 22, 2010 at 11:18:09AM +0200, Rainer Frey wrote:

...

What I don't see here at all (and neither in your Wiki Howto) is how Postfix determines the valid recipients for the domains in virtual_mailbox_domains.

Postfix will expand possible aliases first and determine the final recipient handing over to Dovecot:

virtual_alias_maps = hash:/etc/postfix/virtual

It will query the recipients by connecting to the socket in its chroot provided by Dovecot:

service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = doveauth }

Once it has the homedir it will send it off via LMTP or deliver, whichever you configured via:

virtual_transport = lmtp:unix:private/dovecot-lmtp or

virtual_transport = dovecot

...

The correct parameter would be virtual_mailbox_maps, but AFAIK there is

...

lookup table that read the passwd format from an arbitrary file. So a

no script

...

that generates a hash/whatever postfix lookup file from the passwd-files would still be necessary.

There is no such thing as a correct parameter from my perspective. I did not say that alias creation was to be unified/automated. Instead I said I did not even think this is good practice to me. Anyone with at least a bit of sed/awk knowledge can kludge it from the flat-files anyway.

So your suggestion for local_recipient_maps is to read from a table extracted from the file used by Dovecot. And then use the previously described setup (now in the Wiki) for a proper submission interface (using SASL and asking Dovecot).'

I have Dovecot 1.1.11, and for now using package versions outside of what Ubuntu offers isn't an option (that's something I'll be doing later with a different distribution, but it isn't an option right now). So I can't have submission ask via LMTP. If I can get Postfix to authenticate via passwords in a btree or cdb, I guess I can just copy BOTH usernames AND passwords from the file Dovecot reads from and build the btree or cdb db file for Postfix from there.

Any chance Dovecot can read either btree or cdb files directly?

On 23/04/2010 10:30, Gábor Lénárt wrote:

On Wed, Apr 21, 2010 at 01:45:35PM -0400, Phil Howard wrote:

...

...

Then I think MySQL will do the job. Both postfix and dovecot support MySQL, and you can use SASL to authenticate SMTP with Dovecot, so Dovecot would do all the auth work. Finally, you could use Postfix's VDA patch if you want to use Maildir++.

Hope this helps.

I don't want to use any other server engine of any kind with this. I'm trying to keep it small and lean, and minimize what the people that have to monitor and fix it need to know. So at the present time, I am excluding all databases like any SQL or LDAP or anything else that needs to run as a daemon/engine/service.

Aham, yes, but as soon as you need some management interface, like a web based one, it will be more and more complicated to deal with this decision, you must edit/generate those files with that interface, care about locking because of the possibility of multiple admin access at the same time, you must parse them when you want to show the user list and so on. Sure, if you are very sure that it's not a need and it won't be either, then maybe you're right about "keeping minimal" solution, but based on my experience at an ISP, it's always the situation that sooner or later somebody want to extend the usage of a system which sooner or later needs to use some kind of database to avoid the complexity with dealing local "databases" as flat files or other solution (or keeping them as "system users").

Yeah, I PROMISE that someone will eventually corrupt the flat file text "database" by accident. So now you need to add a backup system (git? rsync?), plus some editing sanity checks? (a la visudo?)

In the end I personally went for a mysql + small rails frontend (using activescaffold which allows you to generate an admin interface in a couple of lines of code)

We don't know the exact requirements of the OP, but based on what we read here, *I* personally would evaluate either:

• sqlite based database with command line shell scripts to edit/examine
• mysql based system with remote control editing from some centralised location (ie you admin it for them)
• Some black box office distro where you just push the button and out it rolls. I didn't look, but there must be some Centos/Ubuntu derivative thing which comes with a web interface, webmail, etc?

With mysql use innodb databases and all the logging options set to sync writes. It should be near impossible to kill even if you keep ripping the power out.

Good luck

Ed W

On 04/22/10 00:58, Peter Hessler wrote:

postfix:

smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth

dovecot:

auth default { socket listen { client { path = /var/spool/postfix/private/auth user = _postfix group = wheel mode = 0660 } } }

Then configure your favorite auth mechanism for dovecot. Let bake for 20 minutes, add salt for taste.

I'll second this.

Why bother finding a file format that's compatible, when Postfix has built-in support for asking Dovecot to take care of it?

I've been running my own mail server this way for some time now...

-- Curtis