On 2010-05-10 1:59 PM, Romer Ventura wrote:

I had to chmod 777 for it to work..

That's pretty much *never* a reasonable solution.

--

Best regards,

Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax

I dont know what else.

I tried to chwon postfix:postfix, vmail:vmail, postfix:vmail,
vmail:postfix and none of them worked. I had to go with chmod 777

Thanks

Romer Ventura

On May 10, 2010, at 1:42 PM, Charles Marcus wrote:

On 2010-05-10 1:59 PM, Romer Ventura wrote:

...

I had to chmod 777 for it to work..

That's pretty much *never* a reasonable solution.

--

Best regards,

Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax

Could it be that selinux or apparmor are playing games with you???

just my 2 $cents Egbert Jan (NL)

-----Oorspronkelijk bericht----- Van: dovecot-bounces+egbert=vandenbussche.nl@dovecot.org [mailto:dovecot-bounces+egbert=vandenbussche.nl@dovecot.org] Namens Romer Ventura Verzonden: maandag 10 mei 2010 21:08 Aan: Dovecot Mailing List Onderwerp: Re: [Dovecot] dovecot/deliver ... Can't open logfile /var/log/dovecot/error.log: Permission denied

I dont know what else.

I tried to chwon postfix:postfix, vmail:vmail, postfix:vmail,
vmail:postfix and none of them worked. I had to go with chmod 777

Thanks

Romer Ventura

On May 10, 2010, at 1:42 PM, Charles Marcus wrote:

...

On 2010-05-10 1:59 PM, Romer Ventura wrote:

...

I had to chmod 777 for it to work..

That's pretty much *never* a reasonable solution.

--

Best regards,

Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax

What about your postfix conf..? mine is set to: virtual_gid_maps = static:1001 virtual_mailbox_base = /srv/mail/vmail/ virtual_mailbox_domains = $mydomain virtual_mailbox_maps = ldap:/etc/postfix/ldap_users.cf virtual_transport = dovecot virtual_uid_maps = static:1001

Thanks

Romer Ventura

On May 10, 2010, at 1:45 PM, Phil Howard wrote:

Just realized my email was not going to the list.

On Mon, May 10, 2010 at 14:20, Romer Ventura rventura@h-st.com
wrote:

...

I am using static uids: mail_uid = vmail mail_gid = vmail user = vmail group = vmail

else it will do what you describe.

I have that, too. But it's not running the right userid. Deliver is running as the userid Postfix starts it as. How could it be any
different since deliver is not suid root (nor should it be, afaik). It seems
that I need to tell Postfix a specific userid to run it as (and tell it
that userid is vmail). I haven't found how to do that, yet.

I'm also getting wrong mail_location. The variable %d comes up
empty. I verified that Postfix actually is passing the full user@domain, in the message header, and in the -a argument (as coded in main.cf
mainbox_command =).

Maybe I need to make /usr/lib/dovecot/deliver be suid vmail? That
would open it up to logged in system users injecting into mailboxes.

...

Thanks

Romer Ventura

On Mon, May 10, 2010 at 13:59, Romer Ventura rventura@h-st.com
wrote:

...

I had to chmod 777 for it to work..

I did chmod 777 to see what it would do, and especially, what
userid the log files were owned by. Bad news from that ... they are owned
by the first user I sent email to. That seems to me to be a Postfix issue
where Postfix still thinks I mailing to local system users, and running the
deliver program under such a userid. When I start adding users which
don't have local system user equivalents, that's going to be a problem

Also, I'm finding that in mail_location = the variable %d is
empty. It should be the domain. Again, this seems like Postfix is treating
local delivery as all-users-are-equivalent for any local domain (and
that is definitely not the case). So I need to look at some Postfix
config now to see how to make it pass the full email address (user@domain ...
so %n@%d represents the email address), and to run dovecot/deliver as user
vmail.

At least I'm not using sendmail :-)

This old legacy "system user" thing is sure a PITA. It should
either be ON or OFF. log files automatically named by the date (and maybe time) ...
kind of like in a shell script I would do: date +/path/to/tree/%Y/%m/% d.log or such.

Yes, 1001 are the uid and gid in my system, check your /etc/passwd
to get the ones for your system.

mailbox_command = /usr/lib/dovecot/deliver

Thanks

Romer Ventura

On May 10, 2010, at 1:56 PM, Phil Howard wrote:

On Mon, May 10, 2010 at 14:44, Romer Ventura rventura@h-st.com
wrote:

...

What about your postfix conf..? mine is set to: virtual_gid_maps = static:1001 virtual_mailbox_base = /srv/mail/vmail/ virtual_mailbox_domains = $mydomain virtual_mailbox_maps = ldap:/etc/postfix/ldap_users.cf virtual_transport = dovecot virtual_uid_maps = static:1001

I saw conflicts in the docs for some of that and what I was doing.
What is uid 1001 on yours? vmail?

What does Postfix do with virtual_mailbox_base ... or why should it
care if it is passing all deliverables to dovecot/deliver. I'm not using
LDAP, so that's out. Auth is via Dovecot, and a test to a non-existant user
was rejected as expected, so it seems the userdb lookup worked. What does "virtual_transport = dovecot" mean that ...

mailbox_command = /usr/lib/dovecot/deliver -c /etc/dovecot/dovecot-postfix.conf -a "${RECIPIENT}"

... does not?

On Mon, May 10, 2010 at 15:25, Jerry dovecot.user@seibercom.net wrote:

From my 'master.cf' file:

dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${user}@${nexthop}

From 'main.cf' file: (snippet)

virtual_gid_maps = static:1002 virtual_minimum_uid = 100 virtual_transport = dovecot virtual_uid_maps = static:1002

Obviously, I have that user:

pw showuser 1002 vmail:*:1002:1002::0:0:Virtual Mail User:/nonexistent:/usr/sbin/nologin

My log file is has 0600 permissions and its owner/group is 'vmail'.

So what does this make Postfix do? Run one instance of dovecot/deliver and pipe email to it? Maybe that is the right solution and running it via mailbox_command is wrong?

So what is virtual_minimum_uid doing for you if virtual_uid_maps is static? Or why are any of these even relevant if everything is being piped to a process started via master.cf?

And (problem I posted in a separate thread) does %d get assigned correctly with the domain name for mail_location = if this method of running dovecot/deliver is used?

On Mon, May 10, 2010 at 15:58, Jerry dovecot.user@seibercom.net wrote:

See: http://wiki.dovecot.org/LDA/Postfix

Be sure to read the entire page.

I have a few times. But now I'm getting a bit of a different perspective on part of it. The parameters are:

-d <username>: Destination username. If given, the user information is looked up from dovecot-auth. Typically used with virtual users, but not necessarily with system users. -a <address>: Destination address (e.g. user+ext@domain). Default is the same as username. (v1.1+ only)

Well, that was actually confusing. I was passing the address via -a instead of -d because -d was described as username. That, and I know that the first cases of "virtual users" (in sendmail and earlier postfix) was actually just a twisted variant of system users, where the left hand side of @ was used alone, and it didn't support distinct domains (e.g. bob@example.com and bob@example.net were both just bob ... even if not the same as bob in /etc/passwd). And that's why I didn't use -d because in my case, I do have different domains, where fred@example.com and fred@example.net are different people. So they are separate mailboxes and separate IMAP and submit logins. Oh, and their passwords may be different, too :-)

It's easy to continue to tie in virtual users to system users when uniqueness is only on the LHS. So if jerry@example.com and jerry@example.net are the same user, and likewise for all users, then storing the password in /etc/passwd or /etc/shadow suffices (for those not wanting to use LDAP, SQL, etc). But when the users need to be different across different domains, even though the LHS is the same, now we have issues with connecting them to system users. And I have seen people map username@domainname to someothername to lookup in /etc/passwd (that would be a nightmare) or just put username@domainname in /etc/passwd (not sure how well that would work).

But there is more than one semantic for "virtual users". I believe I have seen at least four. In my case it will be unrelated to system users in /etc/passwd or the setuid() or seteuid() calls. Security will depend on the mail application codes, not the underlying OS, to keep one user out of another's mailbox (or sieve scripts,etc).

...

So what is virtual_minimum_uid doing for you if virtual_uid_maps is static? Or why are any of these even relevant if everything is being piped to a process started via master.cf?

Not really sure. I was told it has something to do with Postfix itself.

The description of virtual_minumum_uid seemed to suggest that it was a bound applied to what you get from virtual_uid_maps in case something was bad in the map.

...

And (problem I posted in a separate thread) does %d get assigned correctly with the domain name for mail_location = if this method of running dovecot/deliver is used?

You can either try it or perhaps ask on the Postfix forum.

Maybe it's related to -d vs -a in dovecot/deliver. Postfix was sending the full user@domain to dovecot/deliver, and the %d should have been filled in from that by dovecot/deliver. But I was using -a and that may be wrong. I'll try with -d instead. Now I get a new error I didn't get before:

Error: Can't connect to auth server at /var/run/dovecot//auth-master: Permission denied

It's not really clear how it is that worked before.

On Mon, May 10, 2010 at 17:11, Romer Ventura rventura@h-st.com wrote:

try using -d ${recipient}, but change the format of the username in dovecot.conf

What does "change the format of the username" mean?

What i did was to set the mail attribute for each user in AD, then perform a query for it and have dovecot group users by domain, this way i can have user1@example.net and user1@example.com

Sorry, now I'm just not following this at all. I don't know what mail attribute apply here, and I don't know what "in AD" means.

I have the following in in my dovecot-postfix.conf file:

mail_location = maildir:/home/mail/dnamesum=%12MLd/dname=%Ld/unamesum=%12MLn/uname=%Ln/mail

And this was working until I switched to the "virtual_transport = dovecot" method ... although %d was coming up empty (and %12MLd was the md5 of empty). Ultimately my intention is to have:

mail_location = maildir:/home/mail/%2MLd/%Ld/%2MLn/%Ln/mail

On Mon, May 10, 2010 at 18:16, Jerry dovecot.user@seibercom.net wrote:

Please post the output of "dovecot -n" and "postconf -n". Better, provide output from the postfinger tool. This can be found at http://ftp.wl0.org/SOURCES/postfinger.

I have redacted external IP addresses and domain names.

from dovecot -n:

# 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31-19-server x86_64 Ubuntu 9.10 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/error.log info_log_path: /var/log/dovecot/info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 imaps pop3s listen: 172.30.0.24, [fc00::18], [${MYIPV6}::18], 127.0.0.1, [::1] ssl_cert_file: /etc/ssl/certs/ssl-mail.pem ssl_key_file: /etc/ssl/private/ssl-mail.key ssl_parameters_regenerate: 24 ssl_cipher_list: ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no mail_chroot: /home/mail mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 verbose_proctitle: yes mail_privileged_group: mail mail_uid: vmail mail_gid: vmail mail_location: maildir:/home/mail/dnamesum=%12MLd/dname=%Ld/unamesum=%12MLn/uname=%Ln/mail mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_process_size: 768 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login username_format: %Ln@Ld passdb: driver: passwd-file args: username_format=%Ln /etc/mailauth/%Ld.deny deny: yes passdb: driver: passwd-file args: scheme=crypt username_format=%Ln /etc/mailauth/%Ld.passwd userdb: driver: passwd-file args: username_format=%Ln /etc/mailauth/%Ld.passwd socket: type: listen client: path: /var/spool/postfix/private/dovecot-auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail

from postconf -n:

command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/lib/postfix data_directory = /var/lib/postfix default_destination_concurrency_limit = 2 default_privs = nobody in_flow_delay = 1s inet_interfaces = 172.30.0.25 inet_protocols = ipv4, ipv6 local_destination_concurrency_limit = 2 mail_owner = postfix mydestination = 17.DOMAIN.NAMES.REDACTED mydomain = 1.DOMAIN.NAME.REDACTED myhostname = mail.1.DOMAIN.NAME.REDACTED mynetworks = ${MYIPV4}.80/28, 127.0.0.0/8, 172.16.0.0/16, 172.20.0.0/16, 172.30.0.0/16, [::1]/128, [fc00::]/48, [${MYIPV6}::]/48, [fe80::]/48 myorigin = $mydomain proxy_interfaces = ${MYIPV4}.90 queue_directory = /var/spool/postfix recipient_delimiter = - relay_domains = $mydestination smtp_bind_address = 172.30.0.25 smtp_bind_address6 = fc00::25, ${MYIPV6}::25 smtpd_banner = $myhostname ESMTP smtpd_sasl_path = private/dovecot-auth smtpd_sasl_type = dovecot soft_bounce = yes unknown_local_recipient_reject_code = 450 virtual_gid_maps = static:252 virtual_minimum_uid = 100 virtual_transport = dovecot virtual_uid_maps = static:252

from postfinger:

postfinger - postfix configuration on Tue May 11 09:29:55 EDT 2010 version: 1.30

Warning: postfinger output may show private configuration information, such as ip addresses and/or domain names which you do not want to show to the public. If this is the case it is your responsibility to modify the output to hide this private information. [Remove this warning with the --nowarn option.]

--System Parameters-- mail_version = 2.6.5 hostname = marconi uname = Linux marconi 2.6.31-19-server #56-Ubuntu SMP Thu Jan 28 03:40:48 UTC 2010 x86_64 GNU/Linux

--Packaging information-- looks like this postfix comes from deb package: postfix-2.6.5-3

--main.cf non-default parameters-- default_destination_concurrency_limit = 2 inet_interfaces = 172.30.0.25 inet_protocols = ipv4, ipv6 mydestination = 17.DOMAIN.NAMES.REDACTED mydomain = 1.DOMAIN.NAME.REDACTED myhostname = mail.1.DOMAIN.NAME.REDACTED mynetworks = ${MYIPV4}.80/28, 127.0.0.0/8, 172.16.0.0/16, 172.20.0.0/16, 172.30.0.0/16, [::1]/128, [fc00::]/48, [${MYIPV6}::]/48, [fe80::]/48 myorigin = $mydomain proxy_interfaces = ${MYIPV4}.90 recipient_delimiter = - smtp_bind_address = 172.30.0.25 smtp_bind_address6 = fc00::25, ${MYIPV6}::25 smtpd_banner = $myhostname ESMTP smtpd_sasl_path = private/dovecot-auth smtpd_sasl_type = dovecot soft_bounce = yes unknown_local_recipient_reject_code = 450 virtual_gid_maps = static:252 virtual_transport = dovecot virtual_uid_maps = static:252

--master.cf-- smtp inet n - - - - smtpd pickup fifo n - - 60 1 pickup cleanup unix n - - - 0 cleanup qmgr fifo n - n 300 1 qmgr tlsmgr unix - - - 1000? 1 tlsmgr rewrite unix - - - - - trivial-rewrite bounce unix - - - - 0 bounce defer unix - - - - 0 bounce trace unix - - - - 0 bounce verify unix - - - - 1 verify flush unix n - - 1000? 0 flush proxymap unix - - n - - proxymap proxywrite unix - - n - 1 proxymap smtp unix - - - - - smtp relay unix - - - - - smtp -o smtp_fallback_relay= showq unix n - - - - showq error unix - - - - - error retry unix - - - - - error discard unix - - - - - discard local unix - n n - - local virtual unix - n n - - virtual lmtp unix - - - - - lmtp anvil unix - - - - 1 anvil scache unix - - - - 1 scache maildrop unix - n n - - pipe flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient} uucp unix - n n - - pipe flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient) ifmail unix - n n - - pipe flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient) bsmtp unix - n n - - pipe flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient scalemail-backend unix - n n - 2 pipe flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension} mailman unix - n n - - pipe flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${user} dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${user}@${nexthop}

-- end of postfinger output --

On Tue, May 11, 2010 at 12:59, Gerard Seibert dovecot.user@seibercom.netwrote:

I have to admit that I am somewhat confused. You have "postfix" listed as user/group in the dovecot.conf file, yet you have "vmail" listed as the user in 'master.cf". That doesn't look right.

I'm not sure which way things are supposed to be, and I've been trying changes, some of which worked (which confirmed I didn't understand it to begin with). There are too many different usernames (although I can understand the need to have a certain number of distinct isolation users) to keep straight. Which ones go where. There being a lack of a thorough document to configure BOTH postfix and dovecot together makes this harder.

Do you actually have a user with uid 252?

dovecot:x:250:250:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false postfix:x:251:251::/var/spool/postfix:/bin/false vmail:x:252:252::/tmp:/bin/false

I am assuming that you are not using "mydestination =

17.DOMAIN.NAMES.REDACTED" as a virtual delivery address. Virtual domains must not be listed in "my destination". I could not find a "virtual_mailbox_domains"

virtual_mailbox_domains ($virtual_mailbox_maps) Postfix is final destination for the specified list of domains; mail is delivered via the $vir- tual_transport mail delivery transport.

All 17 (and more to come) domains are destined for this MACHINE. And I have gotten email delivered through Dovecot already, which was addressed to one of these machines.

I do think Postfix has always had a somewhat different notion of what "virtual user" is than what I'm doing. If it does make the distinction between Postfix itself as a destination (defined by "mydestination") and Dovecot as a different destination, then the obvious question now is, what do I assign the list of domains (which includes the domain this machine's hostname is in) that go to Dovecot (which, BTW, for now, is all of them)? For temporary reference for discussion purpose, until someone tells me the real variable name to use, I'll call it "dovecotdestination =". So what is that real variable name?

I've done virtual users in Postfix before ... this way. And this way is the only one that worked. What was described as virtual users in Postfix docs did not work. When I discussed this with Postfix people on the list a long time ago when I set that up (elsewhere), they said that it was not really "virtual users" in the sense that Postfix thinks of virtual users. I had "local" delivery going by other means. For THIS setup, I am trying to make local deliver go to Dovecot. And that is successful. And the deliver program is getting what I think it should be getting (the domain name is in the message headers and on the command line).

Personally, I think that you should take this up on the Postfix forum.

Post the output of the "postfinger" and "dovecot -n" and I think that you will be able to get your problem solved. It looks to me like you have "virtual" configured incorrectly. It certainly is not configured like I have it on my system. I use purely "vitual" users and have "mydestination = " in main.cf on my system.

I'm not seeing how this is a Postfix issue, yet. The mail is getting to Dovecot, and it includes the domain name in the addresses. But Dovecot is not filling in the %d variable. I don't see how that is a Postfix issue.

FYI, I am on the Postfix list, already. Maybe what is needed is a list just for combination users, those using Dovecot and Postfix together?

Jerry

dovecot.user@seibercom.net

On Tue, May 11, 2010 at 12:59, Gerard Seibert dovecot.user@seibercom.netwrote:

Virtual documentaion: http://www.postfix.org/virtual.8.html

This seems to be a delivery agent of its own. I don't want Postfix to do the delivery. I want Dovecot to do the delivery so it can create the additional cache/index files (whatever they were ... Dovecot documentation encourages this). So that means handing it off to the /usr/lib/dovecot/deliver program.

On Tue, May 11, 2010 at 14:38, Bradley Giesbrecht < bradley.giesbrecht@gmail.com> wrote:

On May 11, 2010, at 11:26 AM, Phil Howard wrote:

On Tue, May 11, 2010 at 12:59, Gerard Seibert

...

...

wrote:

Virtual documentaion: http://www.postfix.org/virtual.8.html

...

This seems to be a delivery agent of its own. I don't want Postfix to do the delivery. I want Dovecot to do the delivery so it can create the additional cache/index files (whatever they were ... Dovecot documentation encourages this). So that means handing it off to the /usr/lib/dovecot/deliver program.

Basically postfix just needs to know that a username/email address is local and how to deliver.

And it did seem to do that already. Mail was sent to dovecot/deliver. It included the domain name. But deliver just didn't construct the mail_location correctly due to %d being empty. The resulting path with the empty space where the domain name should have been was used to actually deliver the mail. I read that file and the domain name was also in the headers. The domain was there, but %d didn't get it.

If you are using virtual users in main.cf this works for me. virtual_transport = dovecot

In master.cf this works for me.

dovecot unix - n n - - pipe flags=DRhu user=_vmail:_vmail argv=/opt/local/libexec/dovecot/deliver -d ${recipient}

I tried it, but effectively, nothing happened. Maybe the other virtual_* stuff also needs to be configured. I've used that virtual_* stuff before many years ago without success. At the time, from what I remember, the concept of "virtual" the way they were using it just wasn't the same as my idea of "virtual". The way I read the Dovecot docs, virtual for Dovecot seemed to be the same. Now I don't know. I do know I have run across at least 4 different concepts called "virtual email users".

Now dovecot needs to know where to deliver to. I use a database backend so

postfix and dovecot can look the information up in the same place. They just need queries to return the values they require. In dovecot docs look for userdb and passworddb.

I'm using "passwd-file" to authenticate, and mail_location = to compose a pattern of where each maildir will be found. I won't be using a backend database (that's the last thing I want to do).

777 you log file till you figure out which users need to write to it. I have _vmail as my user and group name and my dovecot-deliver.log has _vmail:_vmail for owner and group and dovecot.log has root:_vmail for owner and group.

I got the log file working. I had to tell Postfix to run dovecot/deliver as user:group vmail:vmail and that did it. It WAS running dovecot/deliver as some user whose name just happened to match (even though the mail didn't belong to the person who had that system account).

I'm looking over the Postfix virtual_* stuff again. Maybe there's new stuff since I last did Postfix about 6 years ago or so.

Summary of what I want to accomplish:

There are many domains and many users in each domain. Where the user part of a domain happens to be the same as the user part of another domain, that is NOT to be considered the same mainbox at all. All mail to a set of domains (currently all the domains) is to be delivered to maildir format mailboxes via Dovecot. IMAP users will login as "user@domain". The userdb/authdb is in passwd-file format, with different files for each domain, and user names w/o domain being the index. If necessary, I can change the format of that to one big passwd-file format with user@domainindex. The mailboxes will be located in /home/mail/XX/domain/YY/user/home/mail where XX will be 2 hex digits from the MD5 of the domain, and YY will be 2 hex digits from the MD5 of the username. The part of the path before the final "mail" directory is the "home" for the "user", and the last "mail" subdirectory is in maildir format. Domains and users are to be translated to lower case before composing that path and before taking theur MD5 hash for XX and YY. The delivery into that path is to be done by Dovecot's deliver program so it builds whatever indexes and stuff are needed to make IMAP access faster. Also running POP3 is a plus, but not required. The same userdb/passdb is to also be used for submission of mail, via Dovecot's SASL support exported back to Postfix.

On Tue, May 11, 2010 at 19:25, Noel Butler noel.butler@ausics.net wrote:

...

And it did seem to do that already. Mail was sent to dovecot/deliver. It included the domain name. But deliver just didn't construct the mail_location correctly due to %d being empty. The resulting path with the empty space where the domain name should have been was used to actually deliver the mail. I read that file and the domain name was also in the headers. The domain was there, but %d didn't get it.

interesting...

%d is derived from the right hand side of a username, dovecot's deliver couldn't care less about verifying the domain, since that is the MTA's job.

No doubt. However, the big question is WHICH particular instance of user@domain does it derive domain from? There is more than one inside the headers. There are also options -a and -d and maybe it gets it from ONE of those. Or maybe it looks around more than once source for an address to derive the RHS from. I doubt it would verify any more so than whether it should deliver. But it did deliver, so clearly it believed it could. That big question can also be in the form of "where should the domain be provided that it so far was not provided in?".

I tried it, but effectively, nothing happened. Maybe the other virtual_*

...

stuff also needs to be configured. I've used that virtual_* stuff before

it certainly does

That's a different mode of operation of Postfix that I have had troubles with in the past. The big one I remember having (of more than one) was that it treated all the domains as equivalent. That is, bob@example.com and bob@example.net were the same. OTOH, that may have been due to mishandling of, or by, the NON-Dovecot delivery agent I was using back then.

So I'll try this with Dovecot deliver. Been out of the office for a couple days, so I hope I'll have some time today to give it a shot.

...

I'm using "passwd-file" to authenticate, and mail_location = to compose a pattern of where each maildir will be found. I won't be using a backend database (that's the last thing I want to do).

why not? it simplifies virtual users, you're trying to use a method primarily designed for system accounts, as demonstrated over the past several days you are only giving yourself pain for no reason.

I don't see how one database lookup method vs. another database lookup method has anything to do with whether email users are virtual or not. The actual DATA that comes back from the lookup might. But the method itself should be transparent to the mail delivery decisions. In another thread, CDB was asked for, for a future Dovecot. How do you feel about CDB? Does using CDB make users virtual or system?

On Fri, May 14, 2010 at 17:48, Noel Butler noel.butler@ausics.net wrote:

CDB, oh dear god, you want to go back in time? CDB is no better than any other flatfile based system, it was horrible with qmail and it'll be horrible with anything else above a couple thousand users, you clearly dont add/del users all the time, rebuilding its DB can take some time (I've seen some take 3 minutes, tuff luck if your clients want to add a few users,... so using that is something you cant afford to do as a SP.

I see no problem with CDB. I designed my own variation of that a couple decades ago. Mine isn't quite as fast as CDB, but it was along the basic idea. It was very useful in its time. Had CDB not come along, I'd probably have tried to add mine into things like Postfix and Dovecot at some point.

MySQL makes it such a dream, even with customers adding aliases and so on, its a simple instruction to mysql via the web portal from them, and using replication means every front end has its own local copy, and able to fallback to the master if for some reason it becomes unavailable (never seen that in all the years been using it tho, but its nice insurance)

MySQL (or PostgreSQL, etc) has its place. And for things like CRM with a lot of different aspects, that is the way to go. But even then, I would (and have for other purposes) just export the data out of the SQL database and build a fast index like CDB. However long it takes to build CDB is NOT downtime; it's just lag from data entry to activation. And there are ways to work around that if the lag is an issue, such as having a CDB first, followed by another lookup that may have the most recent data. For example, when the list of new users arrives, add them to a Berkeley DB that is queried next when the CDB has no match. Thus they work even while the CDB (think of it has a static cache) is being rebuilt. One thing I would NOT do is have mail servers hitting the CRM database (or its replicas) directly. It's not a performance issue; it's a security issue. The larger the operation, the more important this is.

its your network (I hope for your sake).. its up to you how efficient it

is.

CDB is very fast.

On 05/18/2010 12:14 AM, Noel Butler wrote:

On Mon, 2010-05-17 at 09:28 -0400, Phil Howard wrote:

...

    its your network (I hope for your sake).. its up to you how
    efficient it
    is.

CDB is very fast.

yes it is, if you only have a small number of users.

We use Michael Tokarev's tinycdb:

• stable on disk format
• has atomic updates
• has a tiny library in case you want to link it statically
• uses a lot less memory when the process count is large, i.e. scales well
• rebuilding the database a few times a day is just fine with a few million records. If you go into tens or hundreds of millions, test rebuild times before putting into production

Using cdb is by no means a MUST but don't dissmiss it out of hand. It has its uses.

-- Eray

On Mon, May 17, 2010 at 17:14, Noel Butler noel.butler@ausics.net wrote:

I guess you've never used it with tens and tens of thousands of users, let alone user numbers well into 6 figures and why on gods (or any) earth would I use that load of crap being backed up by another form? that clearly makes no sense, we have backup provisions being mysql replications sure but thats nothing like what you do. your method is pure insanity in this day and age.

I've used a like technique with over 45 million records present. Was extremely fast. Beat the pants off SQL for the kinds of things this is good for, which is: ... simple key:value lookups

I guess you've been bitten by a proper database solution given your

apprehension for using one.

It's called experience. I could explain many cases where SQL is overkill and overhead. But I don't do mail servers very much, so it would all be off-topic for this list. This is not the SQL/NOSQL battle zone.

yes it is, if you only have a small number of users.

Why would it be any slower if the 45 million records represented users instead of document IDs?

(please use reply to list, not reply to all)

No such button. That's one of the reasons why mailing lists are lousy. Oh, since this is a list about an aspect of mail servers, I suppose it seems natural to communicate over a mailing list. OTOH, some people might need to communicate when mail isn't working. That's one of the reasons I acquired a Gmail account for this and Postfix subscription. So do you know a freemail service where there is a "reply to list" button?

On Thu, May 20, 2010 at 18:45, Noel Butler noel.butler@ausics.net wrote:

like you said, you dont really do mail servers

Id LOVE to see you try even 100K users in mail server situation that is ever changing, you'd soon open your eyes up.

I know it would mean more hits to the DB.

The reason we moved from qmail/vpopmail CDB to qmail/vpopmail/mysql was for a MASSIVE IMPROVEMENT in performance, then added dovecot in for even more performance improvements, I too was hesitant, but a large university having similar problems to us made the change and it was like comparing a snail V porsche, I made our change based on their results and never looked back, of course we then had the sense to migrate to postfix and remove vpopmail from the equation altogether. Best move ever, so yes experience counts.

CDB can still run circles around any *SQL DB. Sounds to me like you were using CDB wrong or other factors in your situation made CDB impractical. One such situation could be frequent updates. If you need to do frequent updates, and with 100K users that may well be the case, then CDB can be a loss. Did you try Berkeley DB?

Now show me how this means CDB is bad for lookups.

...

    (please use reply to list, not reply to all)

No such button. That's one of the reasons why mailing lists are lousy. Oh, since this is a list about an aspect of mail servers, I suppose it seems natural to communicate over a mailing list. OTOH, some people might need to communicate when mail isn't working. That's one of the reasons I acquired a Gmail account for this and Postfix subscription. So do you know a freemail service where there is a "reply to list" button?

well I dont need two copies, and often yours get here first and accepted, therefor the list copy is discarded as duplicate, I prefer my lsit mail to be sorted by evolution into its respective mail folder. The fact gmail dont offer this is no excuse most other gmail users dont have this problem, its just another mess gmail creates, like their hopeless quoter segment handling, but , if you use a service you dont pay for then you cant bitch, but often because "some" dont know how to cut quoting, it ends up being 15 pages long and you have NFI who said what.

Where is the "reply to list" button on Evolution? I don't see one there, either. All it has are "reply" and "reply to all". The reply sends to the sender alone if it's a case where there are 2 addresses to send to (reply to all would send to 2 in that case).

There are a number of posters on the list where the reply goes directly to the list alone. I don't know what it is they do with the headers to get it to come out that way. Maybe you can ask them what they do, then do that.

I have a gmail a/c only for testing when someone whinges the cant get mail from them, i'd never rely on them for day to day communications, those that do, need to stop being lazy and make the extra effort, Evolution (my version anyway) has no short cut button., I have to hit the key combo manually, its not killing me to do so.

I don't know what key you are talking about.

anyway, I guess you'll think your way, and I'll think mine, going to be pointless continuing this thread it seems, since your not by your own admission a mail admin and obviously have not had to deal with the situations we have.

It's not about thinking ... it's about seeing. I see faster lookup performance from CDB and similar technology than from MySQL or PostgreSQL. I haven't tried Oracle (nor is that ever likely to happen). I have tried SyBase and Ingres, and they were both quite bad. In the Sybase case, updates were fortunately only daily, so I could literally run a cron job to download all the records at night, and build a CDB-like DB, and have the lookups be done from there. With 34 million records, the download took about 4 hours. Lookups to Sybase took about 20 seconds each (and it was an indexed table). Lookups on the DB file were a tiny fraction of a second. I see problems with big database engines all the time. Sure, if you are running a big massive mail server with lots of updates, and SQL DB might well be the only choice. Tell me what DB GMAIL uses.

Threads like this are one of the reasons I'm posting from GMAIL.

Charles Marcus put forth on 5/21/2010 10:17 AM:

On 2010-05-21 9:04 AM, Phil Howard wrote:

...

Where is the "reply to list" button on Evolution? I don't see one there, either. All it has are "reply" and "reply to all". The reply sends to the sender alone if it's a case where there are 2 addresses to send to (reply to all would send to 2 in that case).

Thunderbird has it now (yay!), although you have to manually add the button to the toolbar. Keybd equiv is CTRL-SHIFT-L...

Reply-to-List is in the right click menu for those who read messages in the preview pane. On Win32 TBird anyway. This is how I do it.

-- Stan

On Fri, 2010-05-21 at 09:04 -0400, Phil Howard wrote:

...

The reason we moved from qmail/vpopmail CDB to qmail/vpopmail/mysql was for a MASSIVE IMPROVEMENT in performance, then added dovecot in for even more performance improvements, I too was hesitant, but a large university having similar problems to us made the change and it was like comparing a snail V porsche, I made our change based on their results and never looked back, of course we then had the sense to migrate to postfix and remove vpopmail from the equation altogether. Best move ever, so yes experience counts.

CDB can still run circles around any *SQL DB. Sounds to me like you were using CDB wrong or other factors in your situation made CDB impractical. One such situation could be frequent updates.

yes, correct very frequent updates (as mentioned earlier in this thread, seems you have memory) it simply couldnt handle it.

If you need to do frequent updates, and with 100K users that may well be the case, then CDB can be a loss. Did you try Berkeley DB?

eh? WTF is it with you and old methods, this is the year 2010, not 1990

Now show me how this means CDB is bad for lookups.

*sigh* I think ive done that several times and I hate repeating myself.. though I think i have woken up to you now.. did you write part of the CDB code? you must have to be such a fanboi after you've been told a circumstance where it just doesnt cut it you sir are bordering on trolling

Where is the "reply to list" button on Evolution? I don't see one there, either. All it has are "reply" and "reply to all". The reply sends to the

yup trolling, or you have a reading comprehension problem, given I already told you in my last post.

dont bother replying, ive played your troll games long enough. and as its the weekend Im not sitting here wasting it (lovely day outside) on idiots like you.

On Mon, May 10, 2010 at 15:25, Jerry dovecot.user@seibercom.net wrote:

From my 'master.cf' file:

dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${user}@${nexthop}

From 'main.cf' file: (snippet)

virtual_gid_maps = static:1002 virtual_minimum_uid = 100 virtual_transport = dovecot virtual_uid_maps = static:1002

I want to give this approach a try. But I can't find anything in the docs on what ${nexthop} means. I'm not doing any hopping. I don't know what to put in here. Is this just the domain? Would that be ${domain} that I need to use?

I just gave it a shot. Nothing is happening. Postfix comes up. Email into port 25 goes in. But nothing shows up in a mailbox and the log files are not created. It's as if deliver doesn't even get run.

Oops ... just found that the mail is showing up in /var/mail/${USER} ... totally wrong place like Postfix is ignoring this and not running dovecot/deliver at all.

On Mon, May 10, 2010 at 17:23, Romer Ventura rventura@h-st.com wrote:

man pipe

${nexthop} This macro expands to the next-hop hostname.

                This information is modified by the h flag for case

folding.

But what is next hop? I don't have any next hop that I'm aware of. These are local domains being kept distinct.