April 2005 - dovecot - dovecot.org

[Dovecot] Failed to create storage with data:
by Matthieu Imbert 17 Apr '05

17 Apr '05

Hi, i'm using dovecot-1.0-stable. All my mail accounts are virtual accounts that come from pam with the following configuration in auth_default: passdb = pam userdb = static uid=5000 gid=5000 home=/var/mail/vhost/example.com/%n mail=/var/mail/vhost/example.com/%n for a user foo, if the directory /var/mail/vhost/example.com/foo already exists, then all is ok, but if the directory doesn't exists, it seems dovecot is unable to create it "on the fly". I get the following error message: Failed to create storage with data: /var/mail/vhost/example.com/foo i'm quite sure it is not a permission related problem (i made /var/mail/vhost/example.com world writable to be sure). I'm currently trying to understand the maildir code... But could someone tell me if the maildir directory should exist for dovecot to work ? In the case that the answer is yes, i would say in advance that it's a problem for sites hosting a lot of virtual users, since it means that these users must receive at least one mail for their mail account to work correctly (i use postfix as MTA, and if the user's maildir doesn't exist, postfix takes care of creating it). I'm not very fond of solutions with scripts taking care of creating users. Thanks in advance for your answers :) -- \|||/ (o o) Matthieu Imbert ooO-(_)-Ooo------------------------------------------ équipe administration réseau et systèmes Ecole normale superieure lettres et sciences humaines 15, Parvis René Descartes BP 7000 69342 LYON cedex 07 tél 0437376127, secrétariat 0437376122 fax 0437376123 contact équipe réseau et systèmes <reseau(a)ens-lsh.fr> contact personnel <matthieu.imbert(a)ens-lsh.fr> -----------------------------------------------------

5 8

[Dovecot] test-67 index issues.
by Dan Sully 17 Apr '05

17 Apr '05

I've been seeing a lot of these in my logs: dovecot: Apr 11 17:48:24 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:24 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:24 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:24 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:24 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:24 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:24 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:25 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:25 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:25 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 dovecot: Apr 11 17:48:25 Error: IMAP(fenner): Lost transaction log file /home/f/fenner/Maildir/dovecot.index.log seq 3 Agent is Apple Mail. Previously removing the indexes and letting them rebuild fixed the issue. Now I've had to restart the server and remove the indexes. Linux, 2.6.x, glibc 2.3.3 -D -- It's the wrong trousers Gromit, and they've gone wrong!

2 1

[Dovecot] Invalid mbox file /var/spool/mail/username: No such file or directory
by Obantec Support 17 Apr '05

17 Apr '05

Hi dovecot-0.99.11-1.FC3.4.i386.rpm on FC3 when i create a user account with pop3 email i find that if i don't send and email to the account (so that sendmail creates the file under /var/spool/mail/testuser) where "testuser" is the users username. If the user logs in with Outlook or Outlook express they get an error and maillog shows. Apr 11 14:18:58 proteus2a pop3(testuser): Invalid mbox file /var/spool/mail/testuser: No such file or directory Apr 11 14:18:58 proteus2a pop3(testuser): Failed to create storage with data: mbox:/var/spool/mail/testuser if "testuser" file does not exist it would be better if Dovecot created an empty file IMHO. as a temp fix i now fire off emails to new accounts to avoid this. Mark

2 1

[Dovecot] Trying to allocate 0 bytes in index-mail-headers.c
by Todd Burroughs 17 Apr '05

17 Apr '05

Here's a new one. It just started with the latest (April 8) CVS version. In index-mail-headers.c, line 622: fields = t_new(struct mail_cache_field, count); count is 0 I'm seeing this with several mailboxes. We're using mbox. Here's the error message and backtrace. dovecot: Apr 09 10:53:18 Error: 3348 IMAP(chairman): Trying to allocate 0 bytes dovecot: Apr 09 10:53:18 Error: child 3348 (imap) killed with signal 6 Core was generated by `imap'. Program terminated with signal 6, Aborted. (gdb) bt #1 0xbffff5c8 in ?? () #2 0x00000006 in ?? () #3 0x00000d14 in ?? () #4 0xb7ef1d41 in raise () from /lib/tls/libc.so.6 #5 0xb7ef3529 in abort () from /lib/tls/libc.so.6 #6 0x080985ba in i_internal_panic_handler ( fmt=0x6 <Address 0x6 out of bounds>, args=0xd14 <Address 0xd14 out of bounds>) at failures.c:374 #7 0x080989d4 in i_panic (format=0x80add1b "Trying to allocate %u bytes") at failures.c:173 #8 0x08097e37 in t_malloc_real (size=Variable "size" is not available. ) at data-stack.c:214 #9 0x08097f54 in t_malloc0 (size=0) at data-stack.c:282 #10 0x08074d34 in index_header_lookup_init (box=0x80c2990, headers=0xbffff840) at index-mail-headers.c:622 #11 0x0808c6f1 in mailbox_header_lookup_init (box=0x0, headers=0xbffff840) at mail-storage.c:376 #12 0x0805e095 in imap_sort (cmd=0x80bb9a8, charset=0x80bbd38 "US-ASCII", args=0x80d72a0, sort_program=0x80b2098) at imap-sort.c:228 #13 0x08058363 in cmd_sort (cmd=0x80bb9a8) at cmd-sort.c:123 #14 0x080595e5 in _client_input (context=0x80bb968) at client.c:334 #15 0x0809d74a in io_loop_handler_run (ioloop=0x80ba5b0) at ioloop-poll.c:184 #16 0x0809ce69 in io_loop_run (ioloop=0x80ba5b0) at ioloop.c:218 #17 0x080604a7 in main (argc=1, argv=0xbffffa74, envp=0xbffffa7c) at main.c:228 (gdb) bt full #0 0xffffe410 in ?? () No symbol table info available. #1 0xbffff5c8 in ?? () No symbol table info available. #2 0x00000006 in ?? () No symbol table info available. #3 0x00000d14 in ?? () No symbol table info available. #4 0xb7ef1d41 in raise () from /lib/tls/libc.so.6 No symbol table info available. #5 0xb7ef3529 in abort () from /lib/tls/libc.so.6 No symbol table info available. #6 0x080985ba in i_internal_panic_handler ( fmt=0x6 <Address 0x6 out of bounds>, args=0xd14 <Address 0xd14 out of bounds>) at failures.c:374 No locals. #7 0x080989d4 in i_panic (format=0x80add1b "Trying to allocate %u bytes") at failures.c:173 No locals. #8 0x08097e37 in t_malloc_real (size=Variable "size" is not available. ) at data-stack.c:214 block = Variable "block" is not available.

2 2

[Dovecot] dovecot and evolution
by Dennis Lambe 17 Apr '05

17 Apr '05

I am using dovecot 0.99.13 and Evolution 2.0.2 on Fedora Core 3. Recently, Ximian stopped being able to access my local INBOX via IMAP, failing with the following error: Error while Opening folder imap://malsyned@localhost/INBOX. Incomplete server response: no information provided for message 1. I got a transcript of the conversation between Evolution and dovecot. The relavent section follows: ========================== [>>] Reading from client A00015 SELECT INBOX [<<] Reading from server * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 7668 EXISTS * 0 RECENT * OK [UNSEEN 7660] First unseen. * OK [UIDVALIDITY 1101063317] UIDs valid * OK [UIDNEXT 334080] Predicted next UID A00015 OK [READ-WRITE] Select completed. [>>] Reading from client A00016 UID FETCH 1:* (FLAGS RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS.NOT (RECEIVED)]) [<<] Reading from server * 2 FETCH (UID 326401 FLAGS (\Seen) INTERNALDATE "09-Apr-2005 13:40:44 -0400" RFC822.SIZE 698 BODY[HEADER.FIELDS.NOT (RECEIVED)] {335} Date: 09 Apr 2005 13:40:44 -0400 From: Mail System Internal Data <MAILER-DAEMON(a)malsyned.dyndns.org> Subject: DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA Message-ID: <1113068444(a)malsyned.dyndns.org> X-IMAP: 1113068168 0000078081 Status: RO X-IMAPbase: 1101063317 334079 X-UID: 326401 Content-Length: 260 X-Keywords: ) * 3 FETCH (UID 326402 FLAGS (\Seen) INTERNALDATE "24-Mar-2003 01:45:45 -0500" RFC822.SIZE 1794 BODY[HEADER.FIELDS.NOT (RECEIVED)] {680} Return-Path: <mindless(a)cif.rochester.edu> X-Original-To: malsyned@localhost Delivered-To: malsyned(a)localhost.cif.rochester.edu Delivered-To: malsyned(a)cif.rochester.edu Date: Fri, 19 Apr 2002 09:36:25 -0400 Subject: Re: I can't login Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v481) From: Michael Maskalans <mindless(a)cif.rochester.edu> To: <problem(a)cif.rochester.edu> Content-Transfer-Encoding: 7bit In-Reply-To: <003501c1e6f0$d4859bb0$ef441842@yi0b4plk53mjqz> Message-Id: <71B00250-539A-11D6-A72E-00039303ECDC(a)cif.rochester.edu> X-Mailer: Apple Mail (2.481) Content-Length: 301 Status: RO X-UID: 326402 ) ========================== It looks like dovecot really isn't sending any information for message #1, starting right off at message #2. The headers it gives for message #2 are the same as the first message in my mailspool, and the headers for message #3 are those of the second message in my mailspool. It goes on from there. This doesn't happen to other folders I try to access via IMAP, only my INBOX. Since this only seems to affect Evolution and not pine, I don't know whether this is technically a bug in dovecot or Evolution. I guess that's up to the RFC, which I haven't read all the way through. Anybody got an idea about how to fix this or what's causing it? (Please CC me directly on any replies, I'm not subscribed to this mailing list) Thanks, --Dennis Lambe, <malsyned(a)gmail.com> P.S. Is there a way to get a session transcript out of dovecot? It would save me a fair bit of perl hackery.

2 1

[Dovecot] Upcoming configuration file changes
by Timo Sirainen 17 Apr '05

17 Apr '05

For those who might be interested. Comments welcome. A few weeks ago I had started redesigning how Dovecot handles configuration files. I wrote a new parser and converted dovecot-auth to use the new way. Today I finally managed to convert imap process as well, which wasn't that easy because I didn't want mbox/maildir-specific settings to be hardcoded in any common code. So, the changes will be: - Dovecot master process will need a new completely different configuration file which tells what processes to run and their uids/chroots/memory limits. Very much like Postfix's master.cf. I'm not sure about its format yet (suggestions welcome), but maybe something like it's already: process imap-login { executable = /usr/lib/dovecot/imap-login chroot = /var/run/dovecot/login uid = dovecot vsz_limit = 32M } ... Hopefully after these changes the master process will become very small and simple. - One of the processes will be a configuration process, which is the one that reads and parses the configuration for non-master processes. The default configuration file format will be pretty similar to what we have now, but it can of course be replaced with SQL backends and whatever. Config process will listen in some unix socket waiting for requests from other processes, which connect to it just after they start up and maybe also sometimes later (eg. to get user/IP-specific settings when a new client connects). - The data that config process sends and what other processes parse is based on simple key=value pairs, so it's possible to still store and parse them from environment if needed. Also it makes it possible to override settings from command line (eg. imap -o mail_debug=yes) or give user-specific settings in userdb. Here's an example: # config process reads something like: mmap_disable = yes namespace foo { type = private separator = / mbox { dir = ~/mail mbox_read_locks = dotlock } } # sends out: mmap_disable = yes namespaces = foo namespaces_foo_type = private namespaces_foo_separator = / namespaces_foo_mbox = 1 namespaces_foo_mbox_1_dir = ~/mail namespaces_foo_mbox_1_mbox_read_locks = dotlock - Config process should verify that all given settings are valid so that typos are noticed. This may not be that easy because the setting definitions are distributed all around the source code. I think there needs to be at least a (Perl) script that finds out all the setting definitions and writes them to some all-settings.c for config process so it knows them. Settings for plugins are more difficult though.. Should they somehow be marked in config file that they're plugins (plugin foo { .. })? Or maybe config process could dynamically load all the plugins itself and request their setting definitions?

3 4

[Dovecot] Logwatch
by Rich West 16 Apr '05

16 Apr '05

Having FC3 systems and utilizing Dovecot, I have come to rely on some of the information I get from the daily logwatch run. Logwatch, by default, doesn't properly recognize dovecot's information in the messages file, but it is easy to tweak. The patch below will give you some statistics regarding POP3 and IMAP usage. I believe I grabbed all of the differences in the unified diff below. If I missed anything, please post. :) -Rich --- /etc/log.d/conf/services/imapd.conf 2005-04-15 14:24:52.000000000 -0400 +++ /etc/log.d/conf/services/imapd.conf 2005-04-15 14:51:52.073609243 -0400 @@ -20,7 +20,7 @@ # Only give lines pertaining to courier... # I'm not sure if this is complete, especially for the new webmail daemon in 0.44.1 #but you will get at least all currently supported logs -*OnlyService = (imapd|imapd-ssl) +*OnlyService = (imap-login|imapd|imapd-ssl) *RemoveHeaders = --- /etc/log.d/conf/services/pop3.conf 2005-04-15 14:51:22.795813799 -0400 +++ /etc/log.d/conf/services/pop3.conf 2005-04-15 15:07:00.837019392 -0400 @@ -15,12 +15,12 @@ Title = "POP-3" # Which logfile group... -LogFile = messages +LogFile = maillog # Only give lines pertaining to courier... # I'm not sure if this is complete, especially for the new webmail daemon in 0.44.1 #but you will get at least all currently supported logs -*OnlyService = (pop3d-ssl|pop3d|spop3d) +*OnlyService = (pop3-login|pop3d-ssl|pop3d|spop3d) *RemoveHeaders = --- /etc/log.d/scripts/services/pop3 2005-04-15 15:15:28.727123168 -0400 +++ /etc/log.d/scripts/services/pop3 2005-04-15 15:03:47.402606350 -0400 @@ -58,6 +58,8 @@ # Don't care about these... } elsif ( ($User, $Host) = ( $ThisLine =~ /^user (.*?) authenticated - (.*)$/ ) ) { $Login{$User}{$Host}++; + } elsif ( ($User, $Host) = ( $ThisLine =~ /Login: (.*?) (\[.*\])/ ) ) { + $Login{$User}{$Host}++; } elsif ( ($User,$Downloaded,$DownloadSize,$Left,$LeftSize) = ( $ThisLine =~ /^Stats: (.*?) (.*?) (.*?) (.*?) (.*?)$/) ) { $DownloadedMessages{$User} += $Downloaded; $DownloadedMessagesSize{$User} += $DownloadSize; --- /etc/log.d/scripts/services/imapd 2005-04-15 15:15:20.742084824 -0400 +++ /etc/log.d/scripts/services/imapd 2005-04-15 15:04:39.868698875 -0400 @@ -43,6 +43,9 @@ # Don't care about these... } elsif ( ($User, $Host) = ( $ThisLine =~ /^Login user=(.*?) host=(.* \[.*\])$/ ) ) { $Login{$User}{$Host}++; + } elsif ( ($User, $Host) = ( $ThisLine =~ /Login: (.*?) (\[.*\])/ ) ) { + $ConnectionNonSSL{$Host}++; + $Connection{$Host}++; } elsif ( ($User,$Host) = ( $ThisLine =~ /^Authenticated user=(.*) host=(.* \[.*\])$/ ) ) { $Login{$User}{$Host}++; } elsif ( ($User,$Host) = ( $ThisLine =~ /^Preauthenticated user=(.*) host=(.*)$/ ) ) {

3 3

Re: [Dovecot] maildir in NFS
by Lev Butyrev 15 Apr '05

15 Apr '05

>I think it might be replaced with "mmap_disable=yes" (see the >dovecot-example.conf). I guess the nfs.txt file is a little out of date. > >Chris I try to use option "mmap_disable=yes", but in dovecot logs same messages: "mail_index_lock_fd() failed with file /var/mail/sam-butyrev.lk/dovecot.index.log: No locks available\ ..." In syslog such messages: kernel: lockd: cannot monitor ip_address_nfs_server kernel: lockd: failed to monitor ip_address_nfs_server nfs server Solaris 9 nfs client linux slackware 10.1 with kernel 2.4.29 Sory for my english.

2 1

[Dovecot] comparision with other imap servers and folder sharing
by Emmett Lazich 15 Apr '05

15 Apr '05

Some praise: Timo, I must say I am super impressed with this product. I do free support a few good friends with small businesses running Linux servers (a mix of RH9, Suse9.2 and gentoo) that I dropped as email and samba servers. I found UW-IMAP to be terrible in every way except that using mbox is sometimes nice for old s/w tools. No need for details re UW-IMAP problems. In early 2004 I found courier-imap to be vastly better behaved than UW-IMAP and also much faster once large folders were open. DJB's Maildirs seemed cool, and postfix likes them too, and mb2md-3.20.pl worked great . So everything seemed good and certainly was better. But courier cannot cope with large imap accounts (eg. millions of messages and a few GBytes) where courier-imap grinds to a halt on the initial folder scanning. Cyrus just looks like a big evil beast that needs too much time to learn. Four days ago, this lead me to dovecot. The performance, reliability, configurability of dovecot is amazing. Timo, I think you could give yourself more credit on something important when selling Dovecot. You write that Dovecot "does not support shared folders". This might turn away some people considering Dovecot. People whom like me do not have/want a good understanding of the imap protocol. I believe you mean that Dovecot does not support shared folders in the technical imap sense of shared namespaces between separate imap accounts. Since Dovecot works beautifully when sharing folders via logging in with the same username more than once. Plus your documentation and config notes imply that this is no problem. IDLE and fcntl locking works great for me. Thank you and good luck! -Emmett

1 0

[Dovecot] maildir in NFS
by Lev Butyrev 14 Apr '05

14 Apr '05

How can I configure the dovecot for working with maildir's through NFS? If I add option "index_mmap_invalidate = yes" (as in doc/nfs.t) then in log I have this message: "Error: Error in configuration file /etc/dovecot.conf line 296: Unknown setting: index_mmap_invalidate" If I try to get the mail without option "index_mmap_invalidate" then I have in log this message: "Error: POP3(butyrev.lk): mail_index_lock_fd() failed with file /var/mail/sam-butyrev.lk/dovecot.index.log: No locks available dovecot: Error: POP3(butyrev.lk): mail_index_wait_lock_fd() failed with file /var/mail/sam-butyrev.lk/dovecot.index.log: No locks available dovecot: Error: POP3(butyrev.lk): Couldn't open INBOX: Internal error occured. Refer to server log for more information." dovecot-stable-20050414

2 1