How to configure read-only access during a migration?

[Jeffrey Tunison]

Hi.  I would appreciate your advice about this

PROBLEM: =======================================================
Find the simplest way to

configure a Dovecot 2.2.10 IMAP server to access
existing email users' (mbox-format) mail folders via a master_user
****in read-only mode.**** (I've got the master_user access working.)

Simplest because while I'm proficient at UNIX systems administration,
I'm under time pressure and dovecot has a lot of configuration
flexibility. Identifying the right way to do this is proving time-
consuming.  Namespaces? ACL's? Passdb's?

Mounting the mail-containing file systems in read-only mode did not work.
(Would it have worked if I had disabled all file-locking rules (if so, how?)?)

I thought it was time to bring in the dovecot wizards. You!

GOAL: =======================================================
Without knowing or changing users' passwords,
migrate several hundred users of dovecot IMAP to MS Exchange
using the commercial product Transend. I'm manging the dovecot side,
someone else is doing the Transend/Exchange piece. Migration would
occur in batches due to slowness of the Transend side when dealing
with users with email collections over a few gigabytes.

I have a production IMAP server (dovecot 1.0.13 on 32-bit RHEL 4.9).
Let's call it PROD for short.  Users INBOXes are stored locally on
PROD. Their other mail folders are stored on a second machine and
accessed via NFS. Those other folders contain the great majority of
the email to be migrated. Let's call the NFS server MIGR for short.

On MIGR, I've set up a migration IMAP server (dovecot 2.2.10 on
64-bit RHEL 5.10). MIGR is a dedicated NFS server with newer
hardware than PROD. It has an unused Ethernet NIC I can tell
Transend to use.

Both PROD and MIGR use NIS for authentication. Therefore PAM
is consulted for passdb information, normally. No SQL db's used.

I set up the second server (MIGR) for several reasons:

1) there are some users mail folders larger than the 32-bit file size
limit found on the PROD server.
2) the bulk of the email is stored locally on the MIGR server.
3) the version of dovecot is much newer and therefore more likely
to have fewer bugs and more configuration options.
4) I don't want to modify the production environment except where
really necessary.

Pointing Transend at MIGR instead of PROD seems like it will be
faster and better.

Telling end-users to use MIGR instead of PROD would allow them to
verify that their email came across okay (into MS Exchange) without
being able to modify the original copy of email (in Dovecot). In
the days and weeks after migration.

I'd like to disable access ***for a batch of users***
on the PROD server at the time the migration begins for that batch.

I may need to re-enable access on PROD for certain users if the
migration does not succeed.

Transend has been dying on a number of test users despite tweaking its
parameters in consultation with the vendor. "Corrupt" messages seem to
be the issue. Users (or sysadmins) might be required to identify and
delete the offending messages before we retry their migration. (Which
folders and which messages are causing problems will hopefully be
discernible in Transend logs.)

Ha! If you've read this far, then I'd also say that I welcome any
advice on changes to increase performance on MIGR, such as whether
trying to minimize indexing as I've done is useful.

Current configurations of the two servers follow.
===========================================================================
PRODUCTION:
===========================================================================

# 1.0.13: /etc/dovecot.conf
log_path: /var/log/dovecot.log
info_log_path: /var/log/dovecot.log
protocols: imaps imap
listen: *:143
ssl_listen: *:993
ssl_cert_file: /etc/pki/dovecot/certs/dovecot.pem
ssl_key_file: /etc/pki/dovecot/private/dovecot.key
login_dir: /var/run/dovecot/login
login_executable: /usr/libexec/dovecot/imap-login
login_processes_count: 20
login_max_processes_count: 500
max_mail_processes: 1524
first_valid_uid: 100
mail_location: mbox:~/Mail:INBOX=/var/mail/%u:INDEX=/var/spool/dovecot-indexes/%u/
mbox_read_locks: dotlock
mbox_write_locks: dotlock
mbox_dotlock_change_timeout: 60
mbox_min_index_size: 2048
namespace:
  type: private
  separator: /
  inbox: yes
namespace:
  type: private
  separator: /
  prefix: Mail/
  hidden: yes
namespace:
  type: private
  separator: /
  prefix: ~/Mail/
  hidden: yes
namespace:
  type: private
  separator: /
  prefix: mail/
  hidden: yes
namespace:
  type: private
  separator: /
  prefix: ~/mail/
  hidden: yes
auth default:
  master_user_separator: *
  verbose: yes
  debug: yes
  debug_passwords: yes
  passdb:
    driver: passwd-file
    args: /etc/dovecot.deny
    deny: yes
  passdb:
    driver: passwd-file
    args: /etc/dovecot.transend
    master: yes
  passdb:
    driver: shadow
  passdb:
    driver: pam
  userdb:
    driver: passwd
===========================================================================
MIGRATION:
===========================================================================

# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-348.18.1.el5 x86_64 Scientific Linux release 5.10 (Boron)
auth_debug = yes
auth_debug_passwords = yes
auth_master_user_separator = *
auth_verbose = yes
auth_verbose_passwords = plain
disable_plaintext_auth = no
first_valid_uid = 100
log_path = /var/log/dovecot.log
mail_debug = yes
mail_location = mbox:~/Mail:INBOX=/var/spool/concmail/%u:INDEX=MEMORY
mbox_min_index_size = 50048 B
mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
namespace one {
  hidden = yes
  location =
  prefix = Mail/
  separator = /
}
namespace three {
  hidden = yes
  location =
  prefix = mail/
  separator = /
}
passdb {
  args = /etc/dovecot/deny-users
  deny = yes
  driver = passwd-file
}
passdb {
  args = /etc/dovecot/dovecot.transend
  driver = passwd-file
  master = yes
}
passdb {
  driver = pam
}
protocols = imap
service imap-login {
  process_min_avail = 15
  service_count = 10
}
service imap {
  executable = imap postlogin
  process_limit = 2048
}
service postlogin {
  executable = script-login -d rawlog
}
ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
ssl_key = </etc/pki/dovecot/private/dovecot.pem
userdb {
  driver = passwd
}
  deny = yes
  driver = passwd-file
}
passdb {
  args = /etc/dovecot/dovecot.transend
  driver = passwd-file
  master = yes
}
passdb {
  driver = pam
}
protocols = imap
service imap-login {
  process_min_avail = 15
  service_count = 10
}
service imap {
  executable = imap postlogin
  process_limit = 2048
}
service postlogin {
  executable = script-login -d rawlog
}
ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
ssl_key = </etc/pki/dovecot/private/dovecot.pem
userdb {
  driver = passwd
}
verbose_proctitle = yes
protocol imap {
  mail_max_userip_connections = 1000
}