How to configure read-only access during a migration?

Wed Aug 20 19:54:16 UTC 2014

Hi.  I would appreciate your advice about these

TWO PROBLEMS: ======================================================= 
Find the simplest way to

A) configure a Dovecot 2.2.10 IMAP server to access
(existing) email users' (mbox-format) mail folders 
****in read-only mode.****

B) Do the above while allowing a master_user to access mail on behalf
of real users. (I've got master_user access working in read-write mode.)

Simplest because while I'm proficient at UNIX systems administration,
I'm under time pressure and dovecot has a lot of configuration
flexibility. Identifying the right way to do this is proving time-
consuming.  Namespaces? ACL's? passdb's?
I thought it was time to bring in the dovecot wizards. You!

GOAL: =======================================================
Without knowing or changing users' passwords,
migrate several hundred users of dovecot IMAP to MS Exchange
using the commercial product Transend. I'm manging the dovecot side,
someone else is doing the Transend/Exchange piece. Migration would
occur in batches due to slowness of the Transend side when dealing
with users with email collections over a few gigabytes.

I have a production IMAP server (dovecot 1.0.13 on 32-bit RHEL 4.9).
Let's call it PROD for short.  Users INBOXes are stored locally on
PROD. Their other mail folders are stored on a second machine and
accessed via NFS. Those other folders contain the great majority of
the email to be migrated. Let's call the NFS server MIGR for short.

On MIGR, I've set up a migration IMAP server (dovecot 2.2.10 on
64-bit RHEL 5.10). MIGR is a dedicated NFS server with newer
hardware than PROD. It has an unused Ethernet NIC I can tell
Transend to use.

Both PROD and MIGR use NIS for authentication. Therefore PAM
is consulted for passdb information, normally. No SQL db's used.

I set up the second server (MIGR) for several reasons:

1) there are some users mail folders larger than the 32-bit file size
limit found on the PROD server.
2) the bulk of the email is stored locally on the MIGR server.
3) the version of dovecot is much newer and therefore more likely
to have fewer bugs and more configuration options.
4) I don't want to modify the production environment except where
really necessary.

Pointing Transend at MIGR instead of PROD seems like it will be
faster and better.

Telling end-users to use MIGR instead of PROD would allow them to
verify that their email came across okay (into MS Exchange) without
being able to modify the original copy of email (in Dovecot). In
the days and weeks after migration.

I may need to re-enable write access on PROD for certain users if the
migration does not succeed.

Transend has been dying on a number of test users despite tweaking its
parameters in consultation with the vendor. "Corrupt" messages seem to
be the issue. Users (or sysadmins) might be required to identify and
delete the offending messages before we retry their migration. (Which
folders and which messages are causing problems will hopefully be
discernible in Transend logs.)

Ha! If you've read this far, then I'd also say that I welcome any
advice on changes to increase performance on MIGR, such as whether
trying to minimize indexing as I've done is useful.

Current configurations of the two servers follow.
===========================================================================
PRODUCTION:
===========================================================================

# 1.0.13: /etc/dovecot.conf
log_path: /var/log/dovecot.log
info_log_path: /var/log/dovecot.log
protocols: imaps imap
listen: *:143
ssl_listen: *:993
ssl_cert_file: /etc/pki/dovecot/certs/dovecot.pem
ssl_key_file: /etc/pki/dovecot/private/dovecot.key
login_dir: /var/run/dovecot/login
login_executable: /usr/libexec/dovecot/imap-login
login_processes_count: 20
login_max_processes_count: 500
max_mail_processes: 1524
first_valid_uid: 100
mail_location: mbox:~/Mail:INBOX=/var/mail/%u:INDEX=/var/spool/dovecot-indexes/%u/
mbox_read_locks: dotlock
mbox_write_locks: dotlock
mbox_dotlock_change_timeout: 60
mbox_min_index_size: 2048
namespace:
    type: private
    separator: /
    inbox: yes
namespace:
    type: private
    separator: /
    prefix: Mail/
    hidden: yes
namespace:
    type: private
    separator: /
    prefix: ~/Mail/
    hidden: yes
namespace:
    type: private
    separator: /
    prefix: mail/
    hidden: yes
namespace:
    type: private
    separator: /
    prefix: ~/mail/
    hidden: yes
auth default:
    master_user_separator: *
    verbose: yes
    debug: yes
    debug_passwords: yes
    passdb:
      driver: passwd-file
      args: /etc/dovecot.transend
      master: yes
    passdb:
      driver: shadow
    passdb:
      driver: pam
    userdb:
      driver: passwd
===========================================================================
MIGRATION:
===========================================================================

# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-348.18.1.el5 x86_64 Scientific Linux release 5.10 (Boron) 
auth_debug = yes
auth_debug_passwords = yes
auth_master_user_separator = *
auth_verbose = yes
auth_verbose_passwords = plain
first_valid_uid = 100
log_path = /var/log/dovecot.log
mail_debug = yes
mail_location = mbox:~/Mail:INBOX=/var/spool/concmail/%u:INDEX=MEMORY
mbox_min_index_size = 50048 B
mbox_write_locks = fcntl
namespace four {
    hidden = yes
    location =
    prefix = ~/mail/
    separator = /
}
namespace inbox {
    inbox = yes
    location =
    mailbox Drafts {
      special_use = \Drafts
    }
    mailbox Junk {
      special_use = \Junk
    }
    mailbox Sent {
      special_use = \Sent
    }
    mailbox "Sent Messages" {
      special_use = \Sent
    }
    mailbox Trash {
      special_use = \Trash
    }
    prefix = 
}
namespace one {
    hidden = yes
    location =
    prefix = Mail/
    separator = /
}
namespace three {
    hidden = yes
    location =
    prefix = mail/
    separator = /
}
namespace two {
    hidden = yes
    location =
    prefix = ~/Mail/
    separator = /
}
passdb {
    args = /etc/dovecot/dovecot.transend
    driver = passwd-file
    master = yes
}
passdb {
    driver = pam
}
protocols = imap
service imap-login {
    process_min_avail = 15
    service_count = 10
}
service imap {
    process_limit = 2048
}
ssl = required
ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
ssl_key = </etc/pki/dovecot/private/dovecot.pem
userdb {
    driver = passwd
}
verbose_proctitle = yes
protocol imap {
    mail_max_userip_connections = 1000
}