March 2008 - dovecot - dovecot.org

[Dovecot] Another numptie question - making maildirs for shared mailboxes.
by Stroller 12 May '08

12 May '08

Hi there, Sorry to have so much trouble with my Dovecot setup & to ask so many questions, but I'm having some problems with shared folders. I've read the thread at <http://dovecot.org/pipermail/dovecot/2005-January/006079.html> and added the following lines to /etc/dovecot.conf: namespace private { separator = / prefix = inbox = yes } namespace public { separator = / prefix = General/ location = maildir:/tmp/dovecot-test/ inbox = no hidden = no } So I create a directory /tmp/dovecot-test/ and an empty file /tmp/dovecot-test/dovecot-shared with liberal (777 & 666 respectively) permissions and then try to subscribe to the "General" folder in Outlook - I get a "failed to subscribe to the folder. The server responded `the mailbox doesn't exist: General`" error message, but cur, new & tmp are created in /tmp/dovecot-test. These subfolders are owned by the user that attempted to subscribe & have 700 permissions; chmod'ing them 777 makes no difference. Since "default_mail_env = maildir:%h/.maildir" in dovecot.conf I make a directory /tmp/dovecot-test/.maildir also containing cur, new, tmp & dovecot-shared. When I select "IMAP folders" in Outlook I am still unable to subscribe to "General" but am now able to subscribe to "General/maildir". The subscribed folder is shown in Outlook underneath the users' Inbox as a sub-folder of general, screenshot at <http://linux.stroller.uk.eu.org/DovecotSharedFolders.png> - see how I can't subscribe to "General" alone, but it is "brought in" by subscribing to "General/maildir"? If I try to copy a test message into General then, after a delay of a minute or so, I get "Can't move the items - this IMAP command could not be sent to the server before the connect was terminated"; if I try to copy the test message into General/maildir then I immediately get "Can't copy the items. The server responded: `Internal error occurred. Refer to server log for more information [2005-12-20 12:05:40]". I've tried tinkering with other directory structures under /tmp/dovecot-test/ with no success - can anyone suggest what I'm doing wrong, please? It's quite important to me that the "General" folder is a fairly high-level one - only one additional folder icon should show up for the users' shared mail. At one point I managed to create a folder General/General, but that's undesirable - the users wouldn't understand the concept of having a folder showing in Outlook that they can't write to, or which is only used as a container for another folder. (In any case, I believe I was still unable to write to it). I was about to say, "I know I could resolve this with the use of a symlink-type shared folder, but that's undesirable because I'd have to manually add the symlink for each user that's added to the system", but writing this it occurs to me that I could resolve that by adding the symlink to the the etc/skel/. Nevertheless, I see the namespaces approach as more elegant, as it allows for different user groups having different shared folders. I'd appreciate any advice - I suspect I'm just getting the directory hierarchies wrong, or the syntax of dovecot.conf wrong AGAIN! If anyone with a working shared folders setup could provide a copy of the appropriate section of dovecot.conf & a listing of the appropriate directories then I'd be EXTREMELY grateful. Stroller.

7 12

[Dovecot] imap fs quota (rpc) won't work?
by Adam McDougall 06 May '08

06 May '08

In the past I dabbled with the imap quota plugin with the fs backend because I wanted to report usage to my users (not limit them). At the time, the quota plugin would make dovecot crash when trying to write to a folder (I can bring up this report if needed). However, in a later beta of dovecot 1.1 I tried quota again but I cannot get it to report any results. I've been wanting to use fs:noenforcing but also tried just quota = fs to simplify. When I right click a folder stored in my homedir, thunderbird tells me 'the server does not support quotas' in the folder properties. What else can I do? Thanks! IModule loaded: /usr/local/lib/dovecot/imap/lib10_quota_plugin.so IModule loaded: /usr/local/lib/dovecot/imap/lib11_imap_quota_plugin.so Feb 5 21:53:16 gribble dovecot: IMAP(mcdouga9): fs quota add storage dir = /home/mcdouga9/Maildir Feb 5 21:53:16 gribble dovecot: IMAP(mcdouga9): fs quota block device = cypher:/vol/mail/home Feb 5 21:53:16 gribble dovecot: IMAP(mcdouga9): fs quota mount point = /home gribble# quota -v mcdouga9 | grep home /home 10766672 20480000 20480000 963778 4294967295 4294967295 # 1.1.beta14: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/mail.pem ssl_key_file: /usr/local/etc/apache2/ssl/mail.pem login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login mail_max_userip_connections: 25 verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 1000 mail_extra_groups: postlocal mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_plugins: acl fts fts_squat quota imap_quota mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: Mail/ hidden: yes subscriptions: yes namespace: type: public separator: / prefix: #shared/decs/ location: maildir:/egr/mail/shared/decs:CONTROL=%h/Maildir/dovecot/public/control/decs:INDEX=%h/Maildir/dovecot/public/indexes/de list: yes subscriptions: yes auth default: passdb: driver: pam userdb: driver: passwd args: system_user= socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: quota: fs acl: vfile:/usr/local/etc/dovecot-acls:cache_secs=10 fts: squat

1 1

[Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
by Diego Liziero 05 May '08

05 May '08

To some users happens this assertion failure while deleting a message. dovecot: Mar 10 08:40:44 Panic: IMAP(user): file index-sync.c: line 39 (index_mailbox_set_recent_uid): assertion failed: (seq_range_exists (&ibox->recent_flags, uid)) dovecot: Mar 10 08:40:44 Error: IMAP(user): Raw backtrace: [see bleow] dovecot: Mar 10 08:40:44 Error: child 17683 (imap) killed with signal 6 And the message doesn't get deleted. Here one of the backtraces (gdb) bt #0 0x00ae9402 in __kernel_vsyscall () #1 0x00725ba0 in raise () from /lib/libc.so.6 #2 0x007274b1 in abort () from /lib/libc.so.6 #3 0x080ce02d in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfd8c6a4 "=???\016\b'") at failures.c:424 #4 0x080cdc3c in i_panic (format=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:187 #5 0x0809bb4a in index_mailbox_set_recent_uid (ibox=0x88ca768, uid=1041) at index-sync.c:39 #6 0x0808085f in mbox_sync_loop (sync_ctx=0xbfd8c8d0, mail_ctx=0xbfd8cac4, partial=true) at mbox-sync.c:457 #7 0x08081361 in mbox_sync (mbox=0x88ca768, flags=18) at mbox-sync.c:1504 #8 0x08079cf9 in mbox_transaction_commit (t=0x88c5550, log_file_seq_r=0xbfd8cbc8, log_file_offset_r=0xbfd8cbc0) at mbox-transaction.c:45 #9 0x0809c69e in index_transaction_commit (_t=0x88cb1c0, uid_validity_r=0xbfd8cc48, first_saved_uid_r=0xbfd8cc44, last_saved_uid_r=0xbfd8cc40) at index-transaction.c:105 #10 0x0805b0cf in cmd_copy (cmd=0x88b75c0) at cmd-copy.c:141 #11 0x0805f079 in cmd_uid (cmd=0x88b75c0) at cmd-uid.c:26 #12 0x0805f9a9 in client_command_input (cmd=0x88b75c0) at client.c:546 #13 0x0805fa3e in client_command_input (cmd=0x88b75c0) at client.c:595 #14 0x080601c5 in client_handle_input (client=0x88b7368) at client.c:636 #15 0x080603de in client_input (client=0x88b7368) at client.c:691 #16 0x080d5ba0 in io_loop_handler_run (ioloop=0x88b59b0) at ioloop-epoll.c:201 #17 0x080d4e28 in io_loop_run (ioloop=0x88b59b0) at ioloop.c:301 #18 0x08067dbc in main (argc=Cannot access memory at address 0x413a ) at main.c:293 (gdb) bt full #0 0x00ae9402 in __kernel_vsyscall () No symbol table info available. #1 0x00725ba0 in raise () from /lib/libc.so.6 No symbol table info available. #2 0x007274b1 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080ce02d in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfd8c6a4 "=???\016\b'") at failures.c:424 backtrace = 0x88add50 "/usr/libexec/dovecot/imap [0x80ce024] -> /usr/libexec/dovecot/imap [0x80cdc3c] -> /usr/libexec/dovecot/imap [0x809bb4a] -> /usr/libexec/dovecot/imap [0x808085f] -> /usr/libexec/dovecot/imap(mbox_sync+"... #4 0x080cdc3c in i_panic (format=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:187 args = 0xbfd8c6a4 "=???\016\b'" #5 0x0809bb4a in index_mailbox_set_recent_uid (ibox=0x88ca768, uid=1041) at index-sync.c:39 __PRETTY_FUNCTION__ = "index_mailbox_set_recent_uid" #6 0x0808085f in mbox_sync_loop (sync_ctx=0xbfd8c8d0, mail_ctx=0xbfd8cac4, partial=true) at mbox-sync.c:457 mail = {uid = 268, idx_seq = 133265, keywords = {arr = {buffer = 0x88d3770, element_size = 3218655176}, v = 0x88d3770, v_modifiable = 0x88d3770}, flags = 172 '???', uid_broken = 1, expunged = 0, pseudo = 1, from_offset = 143457960, body_size = 579466361970685888, offset = 143456608, space = 578778797692682360} rec = (const struct mail_index_record *) 0xb7e6f008 uid = 1041 messages_count = 3518 offset = 563 ret = <value optimized out> expunged = false skipped_mails = false uids_broken = false #7 0x08081361 in mbox_sync (mbox=0x88ca768, flags=18) at mbox-sync.c:1504 ret = <value optimized out> #8 0x08079cf9 in mbox_transaction_commit (t=0x88c5550, log_file_seq_r=0xbfd8cbc8, log_file_offset_r=0xbfd8cbc0) at mbox-transaction.c:45 mt = (struct mbox_transaction_context *) 0x88cb1c0 mbox = (struct mbox_mailbox *) 0x88ca768 lock_id = 3 ret = 0 #9 0x0809c69e in index_transaction_commit (_t=0x88cb1c0, uid_validity_r=0xbfd8cc48, first_saved_uid_r=0xbfd8cc44, last_saved_uid_r=0xbfd8cc40) at index-transaction.c:105 itrans = (struct mail_index_transaction *) 0x0 seq = 7 offset = 20544 #10 0x0805b0cf in cmd_copy (cmd=0x88b75c0) at cmd-copy.c:141 client = (struct client *) 0x88b7368 storage = (struct mail_storage *) 0x88b6c80 destbox = (struct mailbox *) 0x88ca768 t = (struct mailbox_transaction_context *) 0x0 search_arg = <value optimized out> messageset = 0x88bb6c0 "224" mailbox = 0x88bb6c8 "Trash" msg = <value optimized out> sync_flags = <value optimized out> imap_flags = <value optimized out> copy_count = 1 uid_validity = <value optimized out> uid1 = <value optimized out> uid2 = <value optimized out> ret = 1 __PRETTY_FUNCTION__ = "cmd_copy" #11 0x0805f079 in cmd_uid (cmd=0x88b75c0) at cmd-uid.c:26 command = (struct command *) 0x6 cmd_name = 0x88bb6b8 "copy" #12 0x0805f9a9 in client_command_input (cmd=0x88b75c0) at client.c:546 client = (struct client *) 0x88b7368 command = <value optimized out> __PRETTY_FUNCTION__ = "client_command_input" #13 0x0805fa3e in client_command_input (cmd=0x88b75c0) at client.c:595 client = (struct client *) 0x88b7368 command = (struct command *) 0x6 __PRETTY_FUNCTION__ = "client_command_input" #14 0x080601c5 in client_handle_input (client=0x88b7368) at client.c:636 _data_stack_cur_id = 3 ret = <value optimized out> remove_io = <value optimized out> handled_commands = false #15 0x080603de in client_input (client=0x88b7368) at client.c:691 cmd = <value optimized out> output = (struct ostream *) 0x88b74ec bytes = 24 __PRETTY_FUNCTION__ = "client_input" #16 0x080d5ba0 in io_loop_handler_run (ioloop=0x88b59b0) at ioloop-epoll.c:201 ctx = <value optimized out> event = (const struct epoll_event *) 0x88b5ae8 list = (struct io_list *) 0x88b7568 io = (struct io_file *) 0x88c0dd0 tv = {tv_sec = 1799, tv_usec = 999737} t_id = 2 msecs = <value optimized out> ret = 1 i = 0 j = 0 call = <value optimized out> #17 0x080d4e28 in io_loop_run (ioloop=0x88b59b0) at ioloop.c:301 No locals. #18 0x08067dbc in main (argc=Cannot access memory at address 0x413a ) at main.c:293 No locals. Regards, Diego.

2 6

[Dovecot] Users w/o acl access appear to be subscribed to public folders (1.1b8)
by Adam McDougall 05 May '08

05 May '08

I noticed this today, I had a user outside of our department test out dovecot. They were using squirrelmail and I noticed that dovecot thinks this user is subscribed to ALL public folders even though a dovecot ACL prevents all access. I'm pretty sure access is still denied. I was able to reproduce this with a guest account I added: l lsub "" "#shared/decs/%" * LSUB (\Noselect) "/" "#shared/decs/linuxadmin" * LSUB (\Noselect) "/" "#shared/decs/jbossadmin" * LSUB () "/" "#shared/decs/support" * LSUB () "/" "#shared/decs/receipts" * LSUB (\Noselect) "/" "#shared/decs/pcadmin" * LSUB () "/" "#shared/decs/network" * LSUB (\Noselect) "/" "#shared/decs/printmaster" * LSUB () "/" "#shared/decs/postmaster" * LSUB (\Noselect) "/" "#shared/decs/unixadmin" * LSUB () "/" "#shared/decs/security" * LSUB (\Noselect) "/" "#shared/decs/webmaster" l OK Lsub completed. This only seems to happen when the acl plugin is enabled. Without the acl plugin, these are not listed as subscriptions. After deleting /egr/mail/shared/decs/dovecot-acl-list and re-enabling the acl plugin, I get this: l lsub "" "#shared/decs/%" * LSUB () "/" "#shared/decs/unixadmin" * LSUB () "/" "#shared/decs/support" * LSUB () "/" "#shared/decs/security" * LSUB () "/" "#shared/decs/printmaster" * LSUB () "/" "#shared/decs/postmaster" * LSUB () "/" "#shared/decs/pcadmin" * LSUB () "/" "#shared/decs/network" * LSUB () "/" "#shared/decs/linuxadmin" * LSUB () "/" "#shared/decs/webmaster" * LSUB () "/" "#shared/decs/jbossadmin" l OK Lsub completed. Is it related, or is it different just because a new dovecot-acl-list got created by another user already (but is mode 700?)

2 3

[Dovecot] Segfault when opening a public folder, dovecot 1.1 beta4
by Adam McDougall 04 May '08

04 May '08

I was trying to debug this with gdb, but I'm not sure how to make env MAIL=maildir:~/Maildir MAIL_PLUGINS=acl ACL=vfile gdb /tmp/imap understand the #shared/decs namespace from below so I can SELECT it. I have a number of maildirs under #ahared/decs, I tried opening at least 2 of them but dovecot gives sig 11 without a direct error about the problem. I don't think this was exactly the same problem in 1.1b3, but unfortunately I deleted all the patches I was testing with so it would be a pain to test the same situation with 1.1b3 again (although if needed, I can). If dovecot-shared exists in the folder I try to open, dovecot says: Oct 27 12:57:38 gribble dovecot: IMAP(mcdouga9): fchown() failed with file /home/mcdouga9/Maildir/dovecot11testing/public/indexes/decs/.unixadmin/dovecot.index.log: Operation not permitted Oct 27 12:57:38 gribble dovecot: child 55470 (imap) killed with signal 11 I am used to getting the first message because dovecot tries to apply perms from dovecot-shared to the index files which are elsewhere. If I open a public folder without dovecot-shared, I see no contents but it does not sig11. I am NOT using the acl plugin right now, it didn't seem to make a difference if it was loaded. When I left off with the dovecot 1.1b3 + 8 patches, I believe I was getting an assertion. Please let me know if I goofed up something, I have a strange feeling about that here, or if you need more info. I'd like to go ahead and get a gdb trace. # 1.1.beta4: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/cert.pem ssl_key_file: /usr/local/etc/apache2/ssl/cert.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot11testing/private/control:INDEX=%h/Maildir/dovecot11testing/private/indexes mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep namespace: type: public separator: / prefix: #shared/decs/ location: maildir:/egr/mail/shared/decs:CONTROL=%h/Maildir/dovecot11testing/public/control/decs:INDEX=%h/Maildir/dovecot11testing/public/indexes/decs list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot11testing/private/control:INDEX=%h/Maildir/dovecot11testing/private/indexes hidden: yes subscriptions: yes namespace: type: private separator: / inbox: yes list: yes subscriptions: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: acl: vfile:/usr/local/etc/dovecot-acls

2 18

[Dovecot] UID inserted in the middle of mailbox errors
by Dean Brooks 04 May '08

04 May '08

Hi, We are still getting, on a daily basis, users who cannot move messages to Trash or expunge Trash due to these errors: Mar 28 10:43:57 tm2 dovecot: [ID 107833 mail.error] IMAP(kss021): mbox sync: UID inserted in the middle of mailbox /home/kss021/Mail/Trash (3137 > 536, seq=2, idx_msgs=76) Our users have no other access to their mailbox other than Dovecot, and we are using mbox format and 1.1rc3. Deleting the indexes fixes the problem, so it seems to be a problem with index corruption and not something inherently wrong with the mailbox. That is, once I delete the index files, the problem does not come back immediately. I've seen others on the list with the same problem, but I wasn't able to find a workaround. It's happening more and more frequently. I guess what I don't understand is why Dovecot cant just mark the index invalid and purge it instead of failing over and over. I thought the whole point behind Dovecot's indexing is that it was able to detect index inconsistencies and mark the index invalid. Any help is much appreciated. Even a quick stopgap fix would be a lifesaver, short of a cronjob that just purges everyone's index files every 2 hours (which sort of defeats the point of using Dovecot). -- Dean Brooks dean(a)iglou.com

3 4

[Dovecot] Embedded From_ lines breaking Content-Length (and Dovecot)
by Lennart Lövstrand 04 May '08

04 May '08

I feel like I'm going totally crazy. Is it just me, or have embedded From_ lines really been breaking mbox messages since (at least) dovecot 1.0? I found a whole lot of broken messages in an old mailbox of mine and when I looked closer at it, it seemed like Dovecot was ignoring the Content-Length header and truncating the messages at the first embedded From_ line instead. Figuring that this probably just was a fluke / old bug, I erased the index and upgraded Dovecot, first to 1.0.10, then to 1.1.beta14, but I'm still seeing the bug -- and it's even gotten worse! Earlier versions would just quietly split the message at the From_ line, but 1.1.beta14 is more diligent and will complain with: > dovecot[44795]: IMAP(lennart): FETCH for mailbox Drafts UID 196 got > too little data: 590 vs 719 before dropping the connection and exiting... which cause Thunderbird to reopen the connection and reissue the same request... which will cause Dovecot to complain and exit again... etc ad nauseam... It's trivial to reproduce too -- just mail yourself a message with a valid From_ line in it (assuming that your delivery system isn't doing >From-escaping), or put it in a draft plain text message and save it. Then go and look in your Drafts folder... (Like this one: "From someone Tue Jan 29 03:15:52 2008") Just be sure that you can stop your spinning mail client and/or server if you're using 1.1.beta14... --- OK, I am definitely not very familiar with the Dovecot source, but I spent some time trying to track this down today and it looks to me like i_stream_raw_mbox_read is being asked to find the end of the message (at the From_ line) before mbox_sync_parse_next_mail has had a chance to parse the Content-Length header and find out how big it is. Maybe someone who knows the code better can take a look at it and tell me if I'm barking up the totally wrong tree... Thanks, --Lennart

3 5

[Dovecot] Sieve 1.1.2 -> special chars
by mikkel＠euro123.dk 04 May '08

04 May '08

I'm unable to use local chars in vacation/auto replies with sieve. .dovecot.sieve.err gives the same error as when there's a parse error. Example of error: line 25: string 'æ ø å ö ä ü é á. ': My dovecot is v. 1.1b5 and sieve is v. 1.1.2. This issue was also present with dovecot 1.0.x Is there a way around this? - Mikkel

3 2

[Dovecot] Dovecot 1.1: Ignoring Trash/Junk for maildirsize?
by richs＠whidbey.net 04 May '08

04 May '08

Hi all, We're testing Dovecot 1.1rc3, but have run into a snag with IMAP Quota's and the way Trash and Junk folders can be handled. Our MTA (Exim) quota configuration uses this option: maildir_quota_directory_regex = ^(?:cur|new|\.(?!Junk$)(?!Trash$).*)$ This completely excludes "Junk" and "Trash" from the "maildirsize" file and quota calculations. Dovecot 1.0 could do this using "ignore=Trash" (although only on one folder). Is there any way to duplicate this behavior in Dovecot 1.1? We see that "quota_rule" can be used to set an unlimited quota (storage=0), but for some reason Dovecot still records their tallies in "maildirsize", which breaks the MTA. Thanks! -Rich

2 1

[Dovecot] Imap sent folder sometimes dosen't get updated in dovecot-1.1-rc3(+some current hg patches)
by Diego Liziero 29 Apr '08

29 Apr '08

Yes, again, some sent emails don't get copied to sent folder. This time I get the following error: Error: IMAP(username): UIDs broken with partial sync in mbox file /maildir/username/Sent This happened at least with thunderbird and with evolution. Dovecot is running on vith fcntl locking on Centos5, sendmail should use fcntl, too. Filesystem is ext3 on drbd partition. After a grep vith dovecot.log I see the same error message also with other imap folders as well (mainly Trash imap folder). Should I try to get a rawlog? Regards, Diego.

2 3