May 2008 - dovecot - dovecot.org

[Dovecot] imap memory footprint rather large
by martin f krafft 12 May '08

12 May '08

Dear list, I am experimenting with a new mail handling setup and it involves a single IMAP folder with just under 70'000 messages. When OfflineIMAP connects to the server, the imap process starts to eat up a lot of memory: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 15607 madduck 35 19 283m 244m 239m D 16.9 49.3 0:09.96 imap On the contrary, when "online" client, such as Thunderbird connect, memory usage is around 10m, which is entirely acceptable. The way offlineimap reads may is by FETCHing metadata, then APPENDing new local mail, SEARCHing for the UIDs of each uploaded mail, and finally FETCHing new remote mail. Memory use seems to be O(n) in the size of the folder. On the folder with 70k messages, dovecot seems to allocate 280m of memory, which it then fills to about 70% during the metadata FETCH, and then keeps growing while APPEND/SEARCHing the new local messages. The 70k mailbox is just short of 600Mb in size on disk. Dovecot uses 280Mb to serve it. Is it possible that dovecot is reading too much into memory, or over-optimising? Can I somehow tweak this to lower the memory footprint? Cheers, -- martin; (greetings from the heart of the sun.) \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck "the unexamined life is not worth living" -- platon spamtraps: madduck.bogus(a)madduck.net

2 9

[Dovecot] Another numptie question - making maildirs for shared mailboxes.
by Stroller 12 May '08

12 May '08

Hi there, Sorry to have so much trouble with my Dovecot setup & to ask so many questions, but I'm having some problems with shared folders. I've read the thread at <http://dovecot.org/pipermail/dovecot/2005-January/006079.html> and added the following lines to /etc/dovecot.conf: namespace private { separator = / prefix = inbox = yes } namespace public { separator = / prefix = General/ location = maildir:/tmp/dovecot-test/ inbox = no hidden = no } So I create a directory /tmp/dovecot-test/ and an empty file /tmp/dovecot-test/dovecot-shared with liberal (777 & 666 respectively) permissions and then try to subscribe to the "General" folder in Outlook - I get a "failed to subscribe to the folder. The server responded `the mailbox doesn't exist: General`" error message, but cur, new & tmp are created in /tmp/dovecot-test. These subfolders are owned by the user that attempted to subscribe & have 700 permissions; chmod'ing them 777 makes no difference. Since "default_mail_env = maildir:%h/.maildir" in dovecot.conf I make a directory /tmp/dovecot-test/.maildir also containing cur, new, tmp & dovecot-shared. When I select "IMAP folders" in Outlook I am still unable to subscribe to "General" but am now able to subscribe to "General/maildir". The subscribed folder is shown in Outlook underneath the users' Inbox as a sub-folder of general, screenshot at <http://linux.stroller.uk.eu.org/DovecotSharedFolders.png> - see how I can't subscribe to "General" alone, but it is "brought in" by subscribing to "General/maildir"? If I try to copy a test message into General then, after a delay of a minute or so, I get "Can't move the items - this IMAP command could not be sent to the server before the connect was terminated"; if I try to copy the test message into General/maildir then I immediately get "Can't copy the items. The server responded: `Internal error occurred. Refer to server log for more information [2005-12-20 12:05:40]". I've tried tinkering with other directory structures under /tmp/dovecot-test/ with no success - can anyone suggest what I'm doing wrong, please? It's quite important to me that the "General" folder is a fairly high-level one - only one additional folder icon should show up for the users' shared mail. At one point I managed to create a folder General/General, but that's undesirable - the users wouldn't understand the concept of having a folder showing in Outlook that they can't write to, or which is only used as a container for another folder. (In any case, I believe I was still unable to write to it). I was about to say, "I know I could resolve this with the use of a symlink-type shared folder, but that's undesirable because I'd have to manually add the symlink for each user that's added to the system", but writing this it occurs to me that I could resolve that by adding the symlink to the the etc/skel/. Nevertheless, I see the namespaces approach as more elegant, as it allows for different user groups having different shared folders. I'd appreciate any advice - I suspect I'm just getting the directory hierarchies wrong, or the syntax of dovecot.conf wrong AGAIN! If anyone with a working shared folders setup could provide a copy of the appropriate section of dovecot.conf & a listing of the appropriate directories then I'd be EXTREMELY grateful. Stroller.

7 12

[Dovecot] how add size and vsize filed to a maildir filename ?
by Marco Giunta 12 May '08

12 May '08

Hi to all, I'm a sys admin in a college, and we're using Dovecot as IMAP/POP3 server and delivery; we're also patch dovecot to add managesieve capability. I've a question: I've read on dovecot's wiki, that is possible improve the performance on maildir file by adding ,S=<size>,W=<vsize> fields on filename. How is possible to do that ? I've search on all the wiki and on the mailing list, but I didn't find nothing about modify the maildir file name. Someone can help me ? Best regards, Marco

1 0

[Dovecot] IMAP, Shared folders, symlinks and permissions
by Nigel Allen 12 May '08

12 May '08

Greetings Using FC7, dovecot-1.0.3-6.fc6 and maildir I have a recurring problem with permissions. We have a (real) account on the system called "custs". Within the email account for custs there exists some 300 folders, one for each customer. This worked okay originally with mbox. Everyone had access to their own email folders and they all had access to the "custs" account. They only problem is that to find one folder (customers)'s emails, you had to scroll through 300+ folders. For the people who need access to all of the custs' folders, there was no problem (apart from a real estate one of scrolling which sometimes caused people to drag and drop into the wrong folders). The problem is that around 16 staff need only access to a subset of customers (i.e. the ones they manage - around 20 each) while the rest of the staff need access to the lot. What we did was to convert everything to Maildir so that we could use symlinks. So far so good. We created symlinks from /home/custs/Maildir/customer1 to /home/manager1/Maildir/customer1 and from /home/custs/Maildir/customer2 to /home/manager2/Maildir/customer2. Again, so far, so good. Manager 1 gets access to Customer 1 and Manager 2 gets access to Customer 2. The problem comes when someone who ONLY has access to the "custs" account drops an email into Customer 1. The file dovecot-uidlist gets created and suddenly Manager 1 cannot see the folder. We changed (in desperation) the group owner of custs/Maildir to "common" (which everyone is a member of) and did a chmod g+s from the custs/Maildir down. e even set the permissions of the custs/Maildir/* to 777 to watch what was happening. It appears that the permissions on the file dovecot-uidlist is set to 600 every time someone drops an email into the folder. Then when someone else tries to access the folder they get a "permission denied" error on dovecot-uidlist. Can anyone explain to me how to: Change the way dovecot creates the dovecot-uid files so that the permissions are set to 660 or 666 or Set this up better so that we can share all these folders. Any suggestions, questions, comments will be (very) gratefully accepted. Rgds Nigel. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

2 1

[Dovecot] BUG: Bad grep when building dovecot?
by Andrew Lentvorski 12 May '08

12 May '08

Is dovecot *really* that picky about grep, or did it just happen to pick up an autoconf that's being overly pedantic? I saw this message a couple of times when googling, but no particularly good solution presents itself. I'll probably give it a new GNU grep just to get it to shut up, but I'm a bit surprised. I tried to compile dovecot in an OpenSolaris 2008.5 system in a zone (which is quite a bit more minimal than base OpenSolaris). bash-3.2# uname -a SunOS postfix-s0 5.11 snv_86 i86pc i386 i86pc bash-3.2# ./configure --help `configure' configures dovecot 1.0.12 to adapt to many kinds of systems. bash-3.2# which grep /usr/bin/grep bash-3.2# grep -v Usage: grep -hblcnsviw pattern file . . . bash-3.2# which egrep /usr/bin/egrep Error in configure: checking for cc option to accept ISO C89... (cached) none needed checking dependency style of cc... (cached) none checking how to run the C preprocessor... cc -E checking for grep that handles long lines and -e... configure: error: no acceptable grep could be found in /opt/SunStudioExpress/bin:/usr/sbin:/usr/bin:/usr/xpg4/bin Thanks, -a

2 1

[Dovecot] dovecot mysql
by Evaggelos Balaskas 11 May '08

11 May '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, i am trying to setup dovecot with mysql support (mysql from lampp) but i am getting this : hg clone http://hg.dovecot.org/dovecot-1.1/ dovecot-1.1.hg cd dovecot-1.1.hg ./autogen.sh export -p LDFLAGS="-L/opt/lampp/lib/mysql/" ./configure --with-mysql ... checking mysql.h usability... no checking mysql.h presence... no checking for mysql.h... no configure: error: Can't build with MySQL support: mysql.h not found so i understand that i should use the mysql from mysql.com with the include dir of header files. My question is this : Can somehow use the mysql from lampp ? or dovecot can be installed without the includes header of mysql ? Evaggelos Balaskas - http://ebalaskas.gr Unix System Engineer Informatics Engineer Technological Education -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIJtFnWIK+Pe9twhoRArQeAJ9H3jAuJOg11HqpPNSY+EQErwt2RwCgkBTn 3bwLXjxa4nN8ljVKoaEe77Q= =zmeJ -----END PGP SIGNATURE-----

1 1

[Dovecot] Invalid transaction log size (32684 vs 32784)
by Adam McDougall 09 May '08

09 May '08

May 8 10:17:44 boomhauer dovecot: IMAP(username2): Invalid transaction log size (32684 vs 32784): /home/username2/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.log May 8 10:17:44 boomhauer dovecot: IMAP(username2): Invalid transaction log size (32684 vs 32784): /home/username2/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.log May 8 10:17:44 boomhauer dovecot: IMAP(username2): Invalid transaction log size (32684 vs 32784): /home/username2/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.log

1 0

[Dovecot] dovecot.index.cache: Corrupted virtual size for uid=956: 2182 != 4917
by Adam McDougall 09 May '08

09 May '08

This is from 1.1rc5. I have no idea where the 2182 even came from, all copies of the dovecot-uidlist and all copies of the email in snapshots going back at least 2 weeks show the info below: May 8 22:45:22 hill dovecot: IMAP(username): Corrupted index cache file /home/username/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.cache: Corrupted virtual size for uid=956: 2182 != 4917 dovecot-uidlist: 956 W4917 :1208807110.M854034P88191.hill:2,RS -rw------- 2 username egrstaff 4831 Mar 26 13:53 1208807110.M854034P88191.hill:2,RS

1 0

[Dovecot] LDA/Posfix error: bounce, user unknown
by Colin Brace 09 May '08

09 May '08

Hello, I am trying to get LDA working in conjuction with Postfix, but whenever I activate Dovecot LDA in main.cf and send myself a test messsage, I get an error message in maillog: [...] May 8 20:47:17 venus postfix/pipe[36077]: 4022B5C38: to=<cb(a)lim.nl>, orig_to=<colin(a)lim.nl>, relay=dovecot, delay=0.02, delays=0.01/0/0/0.01, dsn=5.1.1, status=bounced (user unknown) [...] Postfix's virtual LDA has no such problem: [...] May 8 20:49:39 venus postfix/virtual[36181]: F3DF05C37: to=<cb(a)lim.nl>, relay=virtual, delay=0.05, delays=0.03/0/0/0.01, dsn=2.0.0, status=sent (delivered to maildir) May 8 20:49:39 venus postfix/qmgr[36091]: F3DF05C37: removed [...] Here is the LDA section in dovecot.conf: protocol lda { postmaster_address = postmaster(a)lim.nl mail_plugin_dir = /usr/local/lib/dovecot/lda mail_plugins = cmusieve log_path = /var/log/dovecot-deliver info_log_path = /var/log/dovecot-deliver auth_socket_path = /var/run/dovecot/auth-master } My maildirs are here: mail_location = maildir:/home/colin/vmail/%u and Dovecot IMAP has no problem retrieving messages from that location. What might be going wrong with delivery though? Thanks. -- Colin Brace Amsterdam http://lim.nl

2 1

[Dovecot] ipv6 woes
by Jason Fesler 09 May '08

09 May '08

I'm trying to bring up dovecot on a dual stack machine. I tried search for the answers; the answers I found and tried failed :-(. If I try this: protocol imap { listen = 216.218.228.114:143, [2001:470:1:18::114]:143 ssl_listen = 216.218.228.114:993, [2001:470:1:18::114]:993 } (and similiar for pop3) I get: Fatal: listen: Can't resolve address 216.218.228.114:143, [2001:470:1:18::114]: hostname nor servname provided, or not known If I instead use: protocol imap { listen = *, [::] ssl_listen = *, [::] } I get this error: Fatal: listen: Invalid port in address This is while using dovecot 1.0.13 Build options: ioloop=kqueue notify=kqueue ipv6 openssl Any recommendations? -- Jason Fesler, email/jabber <jfesler(a)gigo.com> resume: http://jfesler.com "Give a man fire, and he'll be warm for a day; set a man on fire, and he'll be warm for the rest of his life."

2 4