- dovecot - dovecot.org

[Dovecot] Problems with POP3 UIDL when migrating from MBOX to Maildir
by Jonas Olsson 14 Feb '06

14 Feb '06

Hello, I am in the process of migrating a hosting setup from UW-IMAP to Dovecot. The protocols available to mail clients are IMAP and POP3 both before and after migration. I also wanted to change the mail storage format from MBOX to Maildir. However, at this point I hit a major snag with Dovecot and POP3 UIDL (unique identification listing for POP3 mailboxes). Several of the customers connecting to the mailserver in question are using POP3 clients and leave their e-mails on the server. To avoid downloading all messages every time the clients connect they use the UIDL command in the POP3 specification to determine if any new messages have arrived since last time they checked. In order for this to work the unique identifier of each message must be static and can never change. If the identifiers change, every client will download all messages again, mixing old messages with new, in the belief that all messages are new since their identifiers changed. The first step, migrating from UW-IMAP to Dovecot, did not pose any problems in this regard. Connecting to the POP3 service manually to show the UIDL output revealed: With UW-IMAP: uidl +OK Unique-ID listing follows 1 43ef424e00000001 2 43ef424e00000002 With Dovecot: uidl +OK 1 43ef424e00000001 2 43ef424e00000002 However, after performing a migration to Maildir, storing all messages in ~/Maildir for the user in question, I received a different UIDL output: uidl +OK 1 43ef523100000001 2 43ef523100000002 I used the mb2md utility available from http://batleth.sapienti-sat.org/projects/mb2md/ to do the conversion. A diff between the new Maildir messages and the old mailbox reveals no other difference than the leading "From " line of each message missing from the Maildir messages. ----------- snip ------------------------------ --- /var/mail/testuser 2006-02-12 15:13:01.000000000 +0100 +++ maildir-concat.txt 2006-02-12 15:21:54.000000000 +0100 @@ -1,4 +1,3 @@ -From test(a)example.com Sun Feb 12 15:09:49 2006 Return-Path: <test(a)example.com> X-Original-To: testuser Delivered-To: testuser(a)localhost.localdomain @@ -16,7 +15,6 @@ Testing 1 -From test(a)example.com Sun Feb 12 15:10:23 2006 Return-Path: <test(a)example.com> X-Original-To: testuser Delivered-To: testuser(a)localhost.localdomain ----------- snip ------------------------------ When removing the ~/Maildir directory of the user to redo the conversion I discovered that Dovecot returned another set of UIDL output this time around: uidl +OK 1 43ef528300000001 2 43ef528300000002 This output is for the same messages as in the previous example, with preserved message filenames. Doing a diff between the old ~/Maildir and the new reveals the following differences: ----------- snip ------------------------------ $ diff -ur Maildir.example1 Maildir.example2 Binary files Maildir.example1/dovecot.index and Maildir.example2/dovecot.index differ Binary files Maildir.example1/dovecot.index.cache and Maildir.example2/dovecot.index.cache differ Binary files Maildir.example1/dovecot.index.log and Maildir.example2/dovecot.index.log differ diff -ur Maildir.example1/dovecot-uidlist Maildir.example2/dovecot-uidlist --- Maildir.example1/dovecot-uidlist 2006-02-12 16:20:17.000000000 +0100 +++ Maildir.example2/dovecot-uidlist 2006-02-12 16:21:39.000000000 +0100 @@ -1,3 +1,3 @@ -1 1139757617 3 +1 1139757699 3 1 1139757559.000001.mbox:2, 2 1139757559.000000.mbox:2, ----------- snip ------------------------------ Questions: ---------------- How does Dovecot generate the POP3 UIDL? Is there any workaround that I could use to make Dovecot generate a consistent POP3 UIDL output, enabling a migration from MBOX to Maildir, or am I stuck with using MBOX? My environment ----------------------- OS: CentOS 4.2 Dovecot version: 1.0 beta3 (built locally to an RPM package, available on request) I have tested with earlier versions of Dovecot, getting the same behaviour. Relevant entries in /etc/dovecot.conf: ----------- snip ------------------------------ protocols = imap imaps pop3 pop3s listen = * disable_plaintext_auth = no protocol pop3 { pop3_uidl_format = %08Xv%08Xu pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } ----------- snip ------------------------------ -- /Jonas Olsson

4 6

[Dovecot] IMAP timeout
by Eugeny N Dzhurinsky 14 Feb '06

14 Feb '06

Hi there! Is it possible to set timeout for IMAP connection somehow? The problem is when I'm using mutt to access IMAP mailbox, everything works fine, but after some idle time mutt tells mailbox is closed and next connection crashes it. With courier mutt works just fine, so I think may be dovecot have some timeout which need to be adjusted? I checked manuals and configuration, but no luck... In both cases (dovecot/courier) I'm using IMAP over SSL -- Eugene N Dzhurinsky

3 3

Re: [Dovecot] drac.c
by Matthias Andree 14 Feb '06

14 Feb '06

"Takehiko Abe" <keke(a)gol.com> writes: > I am using dovecot [1.0 beta3] as a pop server with drac. It is Drac is an abomination that needs to go away. I am requesting to unsupport it before Dovecot 1.0. Drac is insecure by design, in that it doesn't authenticate the same channel (but POP3/IMAP instead) that is being privileged (SMTP for relay), and without any clue of how long dynamic IP mappings remain stable. Better solutions are available (and have been for years), namely SMTP AUTH, supported by all relevant mail transfer agents(*) and - IMO - all relevant clients. ________ (*) I do not consider qmail relevant any more. <http://home.pages.de/~mandree/qmail-bugs.html> -- Matthias Andree

4 4

Re: [Dovecot] Solaris 10 U1 (x86?) LDAP gotcha
by Alex Moore 14 Feb '06

14 Feb '06

On Tue, 14 Feb 2006 12:25:15 +0900 Peter Evans <peter+dovecot(a)ixp.jp> wrote: > This is not actually a problem with dovecot, but something with the > way the library in /opt/sfw/lib/libldap.* is compiled. If you find > yourself eating cpu, compile your own ldap from openldap, it is NOT > dovecot's fault. > > It looks like pilot error with the sun freeware edition. Or get openldap from www.blastwave.org:> Although I have not tried openldap with dovecot, I do use it for virtual accounts with another mail product and have never seen what you describe. The blastwave package is typically quite up to date with the current stable from openldap.org. Alex -- --

1 0

[Dovecot] imap process spinning (ix86/OpenBSD 3.8, dovecot 1.0b3)
by Joseph Kiniry 13 Feb '06

13 Feb '06

Hi Dovecot-ers, Immediately on accessing our new dovecot 1.0b3 build, we see processes spinning, much like the old dovecot 0.9x builds did for us. Here is a ktrace of the spinning imap process: topos:tmp# kdump 18248 imap EMUL "native" 18248 imap PSIG SIGALRM caught handler=0x1c03ff88 mask=0x0 18248 imap CALL setitimer(0,0xcfbcb3bc,0xcfbcb3ac) 18248 imap RET setitimer 0 18248 imap CALL sigreturn(0xcfbcb3ec) 18248 imap RET sigreturn JUSTRETURN 18248 imap PSIG SIGALRM caught handler=0x1c03ff88 mask=0x0 18248 imap CALL setitimer(0,0xcfbcb3bc,0xcfbcb3ac) 18248 imap RET setitimer 0 18248 imap CALL sigreturn(0xcfbcb3ec) 18248 imap RET sigreturn JUSTRETURN If any additional config information is needed to debug, just ask. I am now subscribed to (the digest version of) this list. Thanks, Joe --- Joseph Kiniry School of Computer Science and Informatics UCD Dublin http://secure.ucd.ie/ http://srg.cs.ucd.ie/

1 0

[Dovecot] locking problems with RHEL 3 and dovecot
by Dan Bongert 13 Feb '06

13 Feb '06

I just migrated from a FreeBSD/Sendmail/UW-IMAP setup to RHEL 3/Postfix/Dovecot (1.0a5 currently). Both systems used Procmail as the final delivery agent (into inboxes or . Since the changeover, things have *mostly* been going ok, but I've been having some problems that I think are locking-related. I'm getting reports from users about IMAP hangs (mostly from Thunderbird). Sometimes I see this in the Dovecot logs: ovecot: Feb 06 11:30:45 Error: imap(dbongert): fcntl() failed with mbox file /home/d/dbongert/mail/worklists/tech-partners: Resource temporarily unavailable Now, I have read that one should have everything that touches a mailbox use the same sort of locking, but that's the thing--these mailboxes are only accessed via dovecot. I'm sure a few users use procmail to filter their mail, but they're in the strict minority here. As as experiment, I changed these lines to flock rather than fcntl: mbox_read_locks = flock mbox_write_locks = dotlock flock But all that seems to have done is removed the fcntl errors from the log file. These errors seem to be Thunderbird only (1.07-1.5, Linux and Windows clients), but I'm not sure what that means. I've enabled the only thunderbird-specific IMAP workaround I know of: tb-extra-mailbox-sep, but I doubt that has anything to do with this problem. -- Dan Bongert dbongert(a)ssc.wisc.edu SSCC Unix System Administrator

2 2

Re: [Dovecot] drac.c
by Udo Rader 13 Feb '06

13 Feb '06

On Mon, 2006-02-13 at 22:02 +0900, Takehiko Abe wrote: > Udo Rader wrote: > > > a question from an uninitated: what is drac after all? > > Dynamic Relay Authorization Control. > <http://mail.cc.umanitoba.ca/drac/> > > [it is simple... and i think it is clean <duck>] Hmm, SMTP-after-POP ... I would not use it these days. Before we switched to SMTP AUTH, we were also using a POP-before-SMTP model, yet we dropped it immediately when one of our customer's network had been compromized by a virus and due to POPb4SMTP the virus distributed itself using our mailserver without limitation. Udo Rader -- BestSolution.at EDV Systemhaus GmbH http://www.bestsolution.at

1 0

Re: [Dovecot] drac.c
by Matthias Andree 13 Feb '06

13 Feb '06

On Mon, 13 Feb 2006, Takehiko Abe wrote: > Matthias Andree wrote: > > > Drac is an abomination that needs to go away. > > I do not share that view. Are there any arguments that might support your view? -- Matthias Andree

1 0

[Dovecot] drac.c
by Takehiko Abe 13 Feb '06

13 Feb '06

Hello, I am using dovecot [1.0 beta3] as a pop server with drac. It is working nicely, but I am getting the following message in a log: dovecot: pop3(keke): file ioloop.c: line 154 (timeout_remove): \ assertion failed: (*timeout != NULL) dovecot: child 15752 (pop3) killed with signal 6 Adding '&' to 'to_drac' in drac_deinit of drac.c stops the message: void drac_deinit(void) { if (to_drac != NULL) timeout_remove(&to_drac); /* was to_drac */ } But I am not sure if the mod is correct. Thanks, T.

1 0

[Dovecot] Maximum Folder Size/Number of Messages
by Ropetin 13 Feb '06

13 Feb '06

I hope I'm asking this question in the right place, I don't want to offend. I've recently put together a proof of concept Dovecot IMAP server, and I'm very happy with its performance up to this point. I'm looking into using it to replace our existing POP3 server, allowing users to store email on the server rather than on their local machine, hopefully avoiding loss of data. The big question I have is, is there a realistic maximum realistic folder size, either in reference to number of messages or total file size with Dovecot? I'm using Maildir and running it on Debian Linux x86. Right now the filesystem is Ext3, although once the system goes live I'd be looking at something a little more fault tolerant and able to handle the large amount of data. Any recommendations you may have on Filesystem choice are appreciated too! Thanks, Ropetin.

2 1