- dovecot - dovecot.org

[Dovecot] FETCH for mailbox INBOX UID 5003 got too little data: 5369 vs 38919
by Adam McDougall 07 May '08

07 May '08

This happened a while back, I was running 1.1rc3 up until today so I have no idea if it would have an effect. Sorry if its something that has been fixed. The error below was repeated many many times, I deleted the index so the user would stop getting disconnected (although I didn't hear any complaints). I have not seen this happen repeatedly in this manner to any other user yet. Apr 15 11:26:48 boomhauer dovecot: IMAP(username): FETCH for mailbox INBOX UID 5003 got too little data: 5369 vs 38919 Apr 15 11:26:48 boomhauer dovecot: IMAP(username): Corrupted index cache file /home/username/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.cache: Broken virtual size for mail UID 5003 Apr 15 11:26:48 boomhauer dovecot: IMAP(username): Disconnected: Disconnected bytes=139/145074 Apr 15 11:27:08 boomhauer dovecot: IMAP(username): FETCH for mailbox INBOX UID 5003 got too little data: 5369 vs 38919 Apr 15 11:27:08 boomhauer dovecot: IMAP(username): Corrupted index cache file /home/username/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.cache: Broken virtual size for mail UID 5003 Apr 15 11:27:08 boomhauer dovecot: IMAP(username): Disconnected: Disconnected bytes=139/145074 Apr 15 11:28:50 boomhauer dovecot: IMAP(username): FETCH for mailbox INBOX UID 5003 got too little data: 5369 vs 38919 Apr 15 11:28:50 boomhauer dovecot: IMAP(username): Corrupted index cache file /home/username/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.cache: Broken virtual size for mail UID 5003 Apr 15 11:28:50 boomhauer dovecot: IMAP(username): Disconnected: Disconnected bytes=139/145074 Apr 15 11:38:02 boomhauer dovecot: IMAP(username): FETCH for mailbox INBOX UID 5003 got too little data: 5369 vs 38919 Apr 15 11:38:02 boomhauer dovecot: IMAP(username): Corrupted index cache file /home/username/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.cache: Broken virtual size for mail UID 5003 Apr 15 11:38:02 boomhauer dovecot: IMAP(username): Disconnected: Disconnected bytes=139/145074 Apr 15 11:38:52 boomhauer dovecot: IMAP(username): FETCH for mailbox INBOX UID 5003 got too little data: 5369 vs 38919 Apr 15 11:38:52 boomhauer dovecot: IMAP(username): Corrupted index cache file /home/username/Maildir/dovecot/private/indexes/.INBOX/dovecot.index.cache: Broken virtual size for mail UID 5003 Apr 15 11:38:52 boomhauer dovecot: IMAP(username): Disconnected: Disconnected bytes=139/145074 >From dovecot-uidlist: 5003 W38919 S38270 :1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 1208208981.7303_0.hill:2,S # ls -l .snapshot/*/1208208981.7303_0.hill* -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.0/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.1/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.10/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.11/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.12/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.13/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.14/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.15/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.16/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.17/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.18/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.19/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.2/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.20/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.21/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.22/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.3/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.4/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.5/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.6/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.7/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.8/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/hourly.9/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/nightly.0/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/nightly.1/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/nightly.2/1208208981.7303_0.hill:2,S -rw------- 1 username faculty 5236 Apr 14 17:36 .snapshot/nightly.3/1208208981.7303_0.hill:2,S # mv dovecot.index.cache dovecot.index.cache.broken

2 1

[Dovecot] dovecot + exim + verify recipient / reject bad recipient at SMTP time
by Peter Hudec 07 May '08

07 May '08

hello, I'm quite new to dovecot. My instalation is working good Debian Etch - exim: 4.63-17 - dovecot: 1.0.rc15 I'm using virtual user setup, with LDA delivery method. There is no database backend at this time. Is there any way to configure exim to reject non-existing user /asking dovecot lda/ at SMTP time. The standard exim configuration in ACL section do not work: --- cut --- require message = e-mail address is not valid ($local_part@$domain / $recipient_verify_failure) domains = +virtual_domains verify = recipient --- cut --- Please, have any work working example ? This setup work good witch cyrus-imap and LMTP. best regards Peter Hudec

2 1

[Dovecot] Manual thread breaking from Thunderbird
by Christian Jaeger 07 May '08

07 May '08

Hello I'm looking for a way to manually break threads apart when independent threads are mixed up, mostly by users which aren't aware that they should not be replying to existing threads when starting a new one. Since Thunderbird afaik does the threading itself, it would require a solution that actually changes the In-Reply-To header (and maybe also that other (non-standard?) header which is being sent by the M$ clients). For me modifying the actual emails on disk would be fine, i.e. renaming the In-Reply-To header to Old-In-Reply-To and such. (I'd actually *prefer* a solution that modifies the server side store, as opposed to one which keeps that info on the client side only like mutt seems to be doing.) Is there a way to do this already? Seems like a task for a dovecot plugin--or for the xexec plugin and an external perl script or some such? For triggering the renaming, a hacky solution could be to misuse labels (e.g. marking a message starred and "Later" and "Important" at the same time would trigger the modification and remove the labels). When implementing such a thing: maybe also offer a way to modify subjects (for those cases where people are using short meaningless and recurring subjects--to both help breaking up the ensuing threading as well as giving them a useful replacement), maybe as an IMAP extension or such (to be used by Thunderbird plugins (?), or by web apps). (I can code in C and Perl, but don't generally have much time.) Christian.

6 21

[Dovecot] Problem finding inbox and pam_authenticate failed problem
by Chris Beech 07 May '08

07 May '08

Testing installation of Dovecot. Log is showing two errors. Any ideas how to fix this thing??? dovecot: May 07 05:19:06 Error: auth-worker(default): pam(Yoda,127.0.0.1): pam_authenticate() failed: User not known to the underlying authentication module dovecot: May 07 05:21:48 Error: IMAP(Yoda): open() failed with mbox file /var/mail/Yoda: Permission denied Here's the dovecot.conf file # 1.1.rc4: /usr/local/etc/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot-info.log protocols: imap imaps pop3 pop3s listen(default): *:143 listen(imap): *:143 listen(pop3): *:10100 ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login verbose_proctitle: yes first_valid_uid: 1000 last_valid_uid: 5000 mail_privileged_group: mail mail_location: mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_debug: yes mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 auth default: debug: yes debug_passwords: yes passdb: driver: pam passdb: driver: passwd passdb: driver: shadow passdb: driver: passwd-file args: /usr/local/etc/dovecot.passdb userdb: driver: passwd userdb: driver: passwd-file args: /usr/local/etc/dovecot.passdb socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: dovecot group: mail ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

1 0

[Dovecot] Question about Maildir automatic cleanup.
by José Ildefonso Camargo Tolosa 07 May '08

07 May '08

Hi! I want to automatically delete old messages from one user's Maildir, and I was thinking on running something like this: find /home/user/Maildir/cur/ /home/user/Maildir/new/ -daystart -mtime +15 -delete but I'm not sure if by deleting messages "manually" I will break the dovecot.index* files. What do you think? Thanks! Ildefonso. ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

3 2

[Dovecot] 1.07 to 1.0.13 mail_extra_groups
by Mark Sapiro 07 May '08

07 May '08

I have just upgraded from 1.0.7 to 1.0.13. I read the comments in the example .conf file and it seemed what I needed to enable dotlock access to mailboxes in /var/spool/mail (writable by 'mail' group) was "mail_privileged_group = mail" so I removed the "mail_extra_groups = mail" that I had in 1.0.7 and added "mail_privileged_group = mail", but I got errors like the following: May 6 12:48:54 sbh16 dovecot: POP3(xxx): file_lock_dotlock() failed with mbox file /var/spool/mail/xxx: Permission denied May 6 12:48:54 sbh16 dovecot: POP3(xxx): Couldn't open INBOX: Internal error occurred. Refer to server log for more information. [2008-05-06 12:48:54] May 6 12:48:54 sbh16 dovecot: POP3(xxx): Couldn't open INBOX top=0/0, retr=0/0, del=0/0, size=0 May 6 12:48:54 sbh16 dovecot: pop3-login: Login: user=<xxx>, method=PLAIN, rip=72.52.113.36, lip=72.52.113.36, secured I then put "mail_extra_groups = mail" back in the configuration. This seems to have the effect of setting both "mail_privileged_group = mail" and "mail_access_groups = mail" as well as issuing "Warning: mail_extra_groups setting was often used insecurely so it is now deprecated, use mail_access_groups or mail_privileged_group instead" when dovecot is started. So my question is what do I need. I thought it would be just "mail_privileged_group = mail", but that doesn't work. Do I need both "mail_privileged_group = mail" and "mail_access_groups = mail" or just "mail_access_groups = mail". Also, are the comments describing these in the example .conf file correct, or are they reversed. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

[Dovecot] dovecot shows inbox of different user wher logged from the same host twice
by Anselmo Rosa herlen 06 May '08

06 May '08

I have squirrenmail 1.4.13 configured on FC8 with sendmail,procmail as delivery agent, dovecot(imap) ,Using MAILDIR, enabled secure_login plugin, and using password/shadow and system user . When I login through squirrelmail every thing works fine . But when I login again from the same host as a different user with new browser session, the mailbox of the new user displays correctly but the first user session displays the inbox of the second user . ie if I have two https sessions from a single machine and login as two different users the inbox of the later user is displayed in both the sessions. I checked the logs of dovecot and the following is noticed dovecot: May 06 11:12:25 Info: imap-login: Login: user=<ath>, method=PLAIN, rip=::ffff:127.0.0.1, lip=::ffff:127.0.0.1, secured dovecot: May 06 11:12:25 Info: IMAP(ath): Disconnected: Logged out there are lots of messages for a single login . Also the USER shown above is the user of the second session and is displayed when i try to access the first htts session dovecot -n (reports the following) # 1.0.5: /etc/dovecot.conf log_path: /var/log/dovecot protocols: imap imaps ssl_disable: yes login_dir: /var/run/dovecot/login login_executable: /usr/libexec/dovecot/imap-login mail_location: maildir:~/Maildir auth default: passdb: driver: pam userdb: driver: passwd kindly help Best Jokes, Best Friends, Best Food and more. Go to http://in.promos.yahoo.com/groups/bestofyahoo/

2 1

[Dovecot] deliver custom maildir creation
by Steve Roemen 06 May '08

06 May '08

/ I'm testing a new setup of postfix using deliver to put the messages into the user's maildir, and am curious if there is a way to make deliver create custom maildirs when it delivers to a valid user without a maildir? Right now it only creates the inbox, I'd like it to create the .sent/ .Trash/ and .SPAM dirs. Is this possible? Thanks -------------------------------------- / /Steve Roemen// /

5 8

[Dovecot] Problem with quota.
by john.wall＠cky.se 06 May '08

06 May '08

Hi, Im having problem with quota with 1.0.rc17-1ubuntu2.3. Problem is that dovecot wont bounce when quota are exceeded. I have set my quota limit to 2MB and my test-box is 3MB. here is my config: dovecot.conf: #ssl_cert_file = /etc/pki/dovecot/certs/myserver.example.com.crt #ssl_key_file = /etc/pki/dovecot/private/myserver.example.com.key #ssl_ca_file = /etc/pki/dovecot/certs/ca-bundle.crt mail_location = maildir:/home/vmail/%d/%u first_valid_uid = 5000 last_valid_uid = 5000 maildir_copy_with_hardlinks = yes disable_plaintext_auth = no protocols = imap imaps pop3 pop3s protocol imap { mail_plugins = quota imap_quota imap_client_workarounds = outlook-idle delay-newmail } protocol pop3 { mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol lda { postmaster_address = postmaster(a)example.com mail_plugins = quota log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } auth default { # Having "login" also as a mechanism make sure outlook can use the auth smtpd as well # http://wiki.dovecot.org/Authentication/Mechanisms mechanisms = plain login passdb sql { args = /etc/dovecot/sql.conf } userdb sql { args = /etc/dovecot/sql.conf } userdb static { args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } userdb prefetch { } user = nobody socket listen { master { path = /var/run/dovecot/auth-master mode = 0660 user = vmail group = vmail } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } dict { } plugin { quota = maildir:storage=1024:messages=1000 acl = vfile:/etc/dovecot/acls trash = /etc/dovecot/trash.conf } sql.conf: driver = mysql connect = host=localhost dbname=mdbv3 user=xxxx password=xxxx user_query = SELECT concat('/home/vmail/', maildir) as home, concat('maildir:/home/vmail/', maildir) as mail, 5000 AS uid, 5000 AS gid, concat('maildir:storage=', quota) AS quota FROM mailbox WHERE username = '%u' AND active = '1' password_query = SELECT username as user, password, concat('/home/vmail/', maildir) as userdb_home, concat('maildir:/home/vmail/', maildir) as userdb_mail, 5000 as userdb_uid, 5000 as userdb_gid FROM mailbox WHERE username = '%u' AND active = '1' Is this file even nessesary ? (postfix stuff) mysql_virtual_mailbox_limit_maps.cf (not showing login inf etc): query = SELECT quota FROM mailbox WHERE username='%s' AND active = '1'

2 1

[Dovecot] [Solved] Re: Quota not showing correctly after conversion (mb2md.pl)
by k bah 06 May '08

06 May '08

On Fri, 2008-04-25 at 09:56 +0800, k bah wrote: >> I have a problem, when a user has mails that exceed the limit in mb's >> (I set just size limit, no messages limit), neither >> "getquotaroot" (via telnet to port 143), neither webmail software >> (horde) show the quota usage correctly (I'm not talking about new >> mail, talking about those messages copied, when they exceed the size >> limit). Just after trying to delete messages it is shown (gives me an >> error message, quota exceeded, since Trash is not excluded from quota >> limits, for now, I'll exclude it later). This server has some issues: >> many old users have lots of mails, in thousands (1000-15000). Some >> have 50mb to 150mb of mails. The migration is part of the cleanup >> process. > That's quite a confusing paragraph. Do you just mean that "quota is > counted wrong for the old converted messages"? If the user received new mail on the new system, say, around 200kb, and then he/she logs in, messages from the old system are copied. If he had 100Mb of mails (old system), and quota is 20Mb, only the 200kb are shown. Dovecot was not restarted. The problem was what you pointed out, maildirsize had to be deleted, then everything seems to be ok - getquotaroot indicates 100Mb + 200kb. > How badly wrong is the quota? Show me some example values of what they > are and what they're supposed to be? How do you calculate what the quota > is supposed to be? User has +100Mb, quota 20Mb, quota indicates 0 or just the new mail count (messages received on the new system). As you said, just delete maildirsize. >> Is it possible that the migration order (creating dirs in that way) >> is causing the quota usage to be shown wrong? > If all this is before the imap process is started, I don't see any > problems. >> Is it possible that the amount of messages and the size can be the >> cause? > No. >> How can I manually tell dovecot to recalculate the quota, after >> copying messages (when a user logs in, and didn't receive new mails on >> the new mail system, but had messages big enough to exceed the quota >> limit) from the old system? > Deleting maildirsize file recalculates the quota. That was the problem! >> Should I run any commands to rebuild indexes or any operation to keep >> information integrity about the mails/mailboxes to dovecot, after >> copying the converted messages? > No. Ok = -- Powered by Outblaze

1 0

[Dovecot] imap fs quota (rpc) won't work?
by Adam McDougall 06 May '08

06 May '08

In the past I dabbled with the imap quota plugin with the fs backend because I wanted to report usage to my users (not limit them). At the time, the quota plugin would make dovecot crash when trying to write to a folder (I can bring up this report if needed). However, in a later beta of dovecot 1.1 I tried quota again but I cannot get it to report any results. I've been wanting to use fs:noenforcing but also tried just quota = fs to simplify. When I right click a folder stored in my homedir, thunderbird tells me 'the server does not support quotas' in the folder properties. What else can I do? Thanks! IModule loaded: /usr/local/lib/dovecot/imap/lib10_quota_plugin.so IModule loaded: /usr/local/lib/dovecot/imap/lib11_imap_quota_plugin.so Feb 5 21:53:16 gribble dovecot: IMAP(mcdouga9): fs quota add storage dir = /home/mcdouga9/Maildir Feb 5 21:53:16 gribble dovecot: IMAP(mcdouga9): fs quota block device = cypher:/vol/mail/home Feb 5 21:53:16 gribble dovecot: IMAP(mcdouga9): fs quota mount point = /home gribble# quota -v mcdouga9 | grep home /home 10766672 20480000 20480000 963778 4294967295 4294967295 # 1.1.beta14: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/mail.pem ssl_key_file: /usr/local/etc/apache2/ssl/mail.pem login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login mail_max_userip_connections: 25 verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 1000 mail_extra_groups: postlocal mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_plugins: acl fts fts_squat quota imap_quota mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: Mail/ hidden: yes subscriptions: yes namespace: type: public separator: / prefix: #shared/decs/ location: maildir:/egr/mail/shared/decs:CONTROL=%h/Maildir/dovecot/public/control/decs:INDEX=%h/Maildir/dovecot/public/indexes/de list: yes subscriptions: yes auth default: passdb: driver: pam userdb: driver: passwd args: system_user= socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: quota: fs acl: vfile:/usr/local/etc/dovecot-acls:cache_secs=10 fts: squat

1 1

Re: [Dovecot] 1.1rc1: assertion failed: (output->offset > uid_list[0])
by Adam McDougall 06 May '08

06 May '08

Timo Sirainen wrote: > On Fri, 2008-03-07 at 00:17 -0500, Adam McDougall wrote: > >> $3 = {offset = 40896, stream_errno = 70, last_failed_errno = 70, overflow >> = 0, closed = 1, >> real_stream = 0x678500} >> > > Ah, 70 = Stale NFS handle in FreeBSD. Linux used something else. Squat > still needs some NFS fixes. > > Were these 'NFS fixes' done? I just put rc5 in production at my site an hour or two ago so I have just begun gathering log data. I may not see anything interesting for days but I will review them daily. (apologies if this gets double posted, I was sending mail and got disconnected)

1 0

[Dovecot] Couldn't init INBOX: Can't sync mailbox: Messages keep getting expunged
by David Jonas 06 May '08

06 May '08

Upon upgrading from 1.0.2 to 1.1rc4 I see this error for many of our users: Getting size of message UID=1 failed Couldn't init INBOX: Can't sync mailbox: Messages keep getting expunged Logging in with IMAP I would see a bunch of messages with no subject or time and blank bodies (usine horde/imp). Removing dovecot.index.log, dovecot.index.log.2, or dovecot-uidlist fixes the problem for the account (I chose dovecot.index.log). I imagine that forces a rebuild of the indexes. Removing other files in the directory didn't seem to make a difference, including dovecot.index or some arbitrary non-dovecot file. Is there a more conventional way around this problem?

3 9

[Dovecot] A previously unreported lsub/list discrepancy in 1.1rc5 and earlier
by Adam McDougall 06 May '08

06 May '08

Not sure about 1.0, I don't run it anymore :) But a few users discovered for a small issue that only affects a very small portion of my userbase where instructional correction will suffice. I guess I am reporting it in the interest of getting it out there, and I can live with it if not fixed but it might be an issue for some people. I'm running through my list of issues. Basically, if a user has a valid namespace prefix set such as mail/ (used to be required on old mail server but discouraged on new mail client setups with dovecot), lsub/list do not show public namespaces that the user has permission to via acl. If the user removes the namespace prefix, they can 'see' the public folders and subscribe. Additionally if the user subscribes to a public folder then puts mail/ back in the prefix, the folders are no longer listed. Not sure if public folders should be affected by the requested prefix in this manner, since they are not in the user's account. With prefix set: 3 lsub "" "mail/*" 4 list "" "mail/%" 5 list "" "mail/%/%" * OK [RAWLOG TIMESTAMP] 2008-05-05 21:07:16 * LSUB () "/" "mail/INBOX" * LSUB () "/" "mail/Sent Items" * LSUB () "/" "mail/Drafts" * LSUB () "/" "mail/Junk E-mail" * LSUB () "/" "mail/Sent" * LSUB () "/" "mail/Trash" 3 OK Lsub completed. * LIST (\HasNoChildren) "/" "mail/Drafts" * LIST (\HasNoChildren) "/" "mail/Sent Items" * LIST (\HasNoChildren) "/" "mail/Junk E-mail" * LIST (\HasNoChildren) "/" "mail/Sent" * LIST (\HasNoChildren) "/" "mail/Trash" 4 OK List completed. 5 OK List completed. Without prefix set: 3 lsub "" "*" 4 list "" "%" 5 list "" "%/%" * OK [RAWLOG TIMESTAMP] 2008-05-05 21:07:49 * LSUB () "/" "INBOX" * LSUB () "/" "Sent Items" * LSUB () "/" "Drafts" * LSUB () "/" "Junk E-mail" * LSUB () "/" "Sent" * LSUB () "/" "Trash" 3 OK Lsub completed. * LIST (\HasNoChildren) "/" "Drafts" * LIST (\HasNoChildren) "/" "Sent Items" * LIST (\HasNoChildren) "/" "Junk E-mail" * LIST (\HasNoChildren) "/" "Sent" * LIST (\HasNoChildren) "/" "Trash" * LIST (\HasNoChildren) "/" "INBOX" 4 OK List completed. * LIST (\Noselect \HasChildren) "/" "#shared/be" * LIST (\Noselect \HasChildren) "/" "#shared/me" * LIST (\Noselect \HasChildren) "/" "#shared/cee" 5 OK List completed.

2 1

[Dovecot] Replication milestone 1
by Timo Sirainen 06 May '08

06 May '08

5 milestones planned currently. I'll start with the always-on multi-master replication, because that's the most difficult one to get working correctly and efficiently. So if during its implementation I find some design problems, less code needs to be fixed. Milestone 0 will hopefully be done within a month. This includes reading and replying to the rest of the mails on this list and getting a v1.1.0 release out. :) There will most likely also be a v1.2.0 release this summer that has some new features on top of v1.1. Several companies have asked for replication within the past year. At least two are willing to contribute some money into its development this summer, but having more wouldn't hurt of course. :) Paying also guarantees (within reasonable limits) that I'll listen to what kind of a installation you're planning and make sure that the replication will work in that kind of a setup. So, below's the milestone 1 fully described with a couple of FIXMEs left. The rest of the milestones are described only briefly. 0. Prerequisites: - Change v1.2 CONDSTORE implementation to allocate modseqs differently. Currently log file sequence + offset determines the modseq. This can't work in a cluster since their log files are different. Instead add a new "UIDs [..] have modseq X" transaction log record that gets updated when needed. UID list can be 0 to update higest-modseq. To avoid adding modseq for all changes, we can be optimistic and assume that usually multiple servers aren't doing changes at the same time. So the next transaction after modseq=5 record will have modseq=6 and the following one modseq=7, etc. - Update v2.0 framework to include v1.2 changes and make it work. Base the replication code on top of v2.0. - Shared mailboxes making all users' all mailboxes available for read/write within the same (replication) process. 1. Incremental replication for existing mailboxes. IMAP, POP3 and deliver use a replication plugin, which connect to a replication process via UNIX socket (or TCP optionally for chroots). The plugin sends the SAVE, EXPUNGE and STORE changes to the replication process, which forwards them to other servers. The other servers receive the commands and store them to local filesystem. The sending and the receiving sides have quite distinct jobs, so they should probably be implemented as separate processes. They could still use the same TCP socket though, replication sending process writes to the socket and replication receiving process reads the socket. The processes still need to share some state though. Most importantly keeping track of the mailbox master servers. This is probably the most cleanly implemented by having a separate master tracking process which the sending/receiving processes talk to via UNIX socket. The rest of the data sharing is about forwarding command replies (SAVE, STORE with CONDSTORE flag) to replication sender, so it can pass them on to the originating mail process. This can be done easily using an extra pipe/socket between the processes. Examples use: - C1 = Client 1 - C2 = Client 2 - S = Replication sending process - R = Remote replication receiving process - Small letter before commands = IMAP-like command tag - * before commands = tag for a command that doesn't expect a reply 1.1. Master tracking process Master tracking process is connected to sending and receiving replication processes. LOCK: (mail process -> replication sender -> master tracker) - User name - Mailbox name Reply: - success LOCK is sent by replication plugin from mail processes when it wants to assign UIDs for new messages. Replication sender proxies this command to master tracker. If server is already the master, the command returns success immediately. Otherwise tracker looks up from its cache who the current master is and requests it using the REQUEST command. Once GIVE command is received, reply success to the mail process. FIXME: handle timeouts (and master disconnections) somehow? UNLOCK: (mail process -> replication sender -> master tracker) - User name - Mailbox name No reply. UNLOCK is sent by mail processes after they've finished assigning UIDs and sending replication SAVEs. If this was the last lock, pending TAKE requests for this mailbox are handled. GIVE: (replication receiver -> master tracker) - User name - Mailbox name No reply. Notification that this server is now the master for the specified mailbox. If there are any pending LOCK requests for this mailbox, they're replied to in here. TAKE ([mail process ->] replication receiver -> master tracker): - User name - Mailbox name - Server ID No reply. Request for giving master state for the specified server. When this mailbox contains no locks, the TAKE command is forwarded to replication sending process which passes it to the requesting server. While waiting for existing locks to be releases, handling new LOCK commands for the mailbox must be delayed to avoid lock starvation. When forwarding TAKE command, the destination server ID is cached as being the mailbox's current master. If we're not the current master for this mailbox, forward the request using REQUEST command. This also happens if there are multiple pending TAKE requests. The first one gets to become the master and the rest of them are handled as REQUEST commands. REQUEST (master tracker -> replication sender): - User name - Mailbox name - Server ID of the current assumed master - Server ID who wants to be the new master (e.g. ourself) Request master status to be moved to the specified server. If server ID isn't ours, cache it as being the mailbox's current master. If we don't have any idea who the master might be currently, use the root server ID. ROOT-SET (replication receiver -> master tracker): - Server ID This command causes all cached master states to be dropped. Any new LOCK requests are delayed. Once all mailboxes are unlocked, drop being a master from all mailboxes and send ROOT-CLEAR command to replication sender, which passes on the information that we've cleared the master states. The given server ID is set to be the new root. ROOT-CLEAR (replication receiver -> master tracker): - Server ID Given server ID has notified that it has cleared its master states. If we're not the current root, this command is forwarded to it. If we're the root and this was the last pending ROOT-CLEAR, start handling the pending REQUEST and LOCK commands. FIXME: How is initial root selection done? How is it decided who's the new one when previous one dies? 1.2. Replication sending 1.2.1. Saving (For now we ignore the possibility of saving as non-master.) When a mail process wants to give UIDs for newly saved messages, it first requests for master lock for the mailbox from the master tracker process. Once UIDs have been assigned for the new messages, the messages are sent to replication sender using SAVE commands. After the last one is sent, the master lock is dropped. Note that this lock duration should be as small as possible, so in Dovecot's code this means doing both locking and unlocking in mailbox_transaction_commit(). If multiple processes are sending SAVEs at the same time, the replication sender may receive the UIDs in wrong order. It must reorder them before forwarding the SAVEs to other servers. Since all master LOCK/UNLOCK commands pass through the replication sender, it can figure out when it has seen all the SAVEs based on them. For example: C1-C3 -> S: a LOCK [1] S -> C1-C3: a OK [Client 1 sends the first mail] C1 -> S: b SAVE uid=5 .. [SAVE uid=5 depends on dropping [1] locks (currently clients 1,2,3)] C1 -> S: * UNLOCK [1] (left 2) [Client 1 starts sending more mails in a separate transaction, lock counter is incremented because of the previous SAVE] C1 -> S: d LOCK [2] (remember lock[2].prev_uid=5) S -> C1: d OK [Client 2 sends its mail] C2 -> S: b SAVE uid=9 .. [SAVE uid=9 depends on dropping [1-2] locks (currently clients 1,2,3)] C2 -> S: * UNLOCK [1] (left 1) [Client 3 sends its mail] C3 -> S: b SAVE uid=3 .. [SAVE uid=3 depends on dropping [1] lock (currently client 3). It doesn't depend on [2] lock, because uid < lock[2].prev_uid (3 < 5)] C3 -> S: * UNLOCK [1] (left 0) [SAVEs depending on [1] locks are ordered and forwarded.] S -> R: x SAVE uid=3 .. S -> R: y SAVE uid=5 .. [Client 1 finishes its second transaction] C1 -> S: f SAVE uid=7 [SAVE uid=7 depends on dropping [2] lock (currently client 1)] C1 -> S: * UNLOCK [2] (left 0) [SAVEs depending on [2] locks are ordered and forwarded] S -> R: x SAVE uid=7 .. S -> R: y SAVE uid=9 .. The idea is that locks have a counter which is incremented if there was a SAVE command sent after the previous LOCK command. The previous SAVE's UID (prev_uid) is also stored to the lock. When the first SAVE after LOCK arrives, it's assigned a dependency on all existing locks which have uid > prev_uid. The lock counter order doesn't guarantee an ascending prev_uid order. (An actual lock counter probably isn't even needed, prev_uid should be enough as the lock ID. Lock counter simplifies the above example though.) All subsequent SAVEs from the client before the next UNLOCK will have the same dependency as the first SAVE. This is because by that time the transaction has been committed and all messages within the transaction have been given UIDs. So in fact it wouldn't really matter in which order the SAVEs are even sent to the replication sender. Although sending them in order allows an extra optimization: If only a single client is holding a lock, the SAVEs can be immediately forwarded as they're being received. After all lock dependencies have been dropped, it's guaranteed that all new SAVEs will have higher UIDs, so the SAVEs can be ordered and forwarded. A further optimization could use some kind of weak locks that are dropped after the client's first SAVE command. After the weak lock count has dropped to zero, the messages can started to be sent. For example assume two clients having the same lock counter. Both clients would be sending 10000 messages within the same transaction. Waiting for and buffering all of them would waste a lot of memory. Instead it would be possible to read the first SAVE from the first client and then block until the first SAVE is read from the second client. The SAVE with a smaller UID can be forwarded to remote servers. The next message is then read from that connection, and this is continued until all messages have been processed. A simpler solution would have been to order and forward SAVEs only when lock counter dropped to zero, but under very high load it's (at least theoretically) possible that the counter never drops to zero. In synchronous setups (recommended whenever possible) the SAVE commands use a tag (not "*"), which requests an ack reply from the remote. After replication sender has sent SAVE to a remote server, the remote server eventually replies back. This reply is forwarded from replication receiver to sender, which in turn forwards it to the mail process. After mail process has finished sending all SAVEs, it waits until it has received at least one ack for each SAVEd message. If the replication sender sent the SAVE to multiple servers, mail process may receive multiple acks for each message. It should just ignore the ones it doesn't care about. 1.2.2. Modseq updating If all mailbox modifications were done only while the server was the mailbox master, there wouldn't be any problems with modseq updates, because only one server could modify the mailbox at a time. This scales badly though, so it's preferrable to move the master only when absolutely required (for SAVEs) and handle modseqs another ways. Modseqs are incremented by each mailbox modification. All modification commands contain a modseq parameter. When applying the modification command make sure the local modseq is at least as high as the modification command's modseq. If the local modseq is higher, broadcast the new modseq to all other servers using MODSEQ command. When receiving a MODSEQ command, update the modseq only if its current value is lower than the received one. Remember that modseqs can also be updated for expunged messages. All this means that eventually everyone's modseq will be the highest assigned modseq for the change in the cluster. It doesn't matter if temporarily some clients saw lower modseqs. The worst that can happen is that some extra FLAG/MODSEQ updates are sent to the client. Although there is a problem if a client can jump from a server where it had higher modseqs to a server that hadn't yet received the incremented modseqs and performed a sync using the higher modseq. If the updated modseqs come after the sync, they're lost from the client. This could be avoided by issuing a cluster-wide sync wait before such syncing commands are run. 1.2.3. Expunging EXPUNGE commands can be replicated without waiting for anything. There are problems with messages getting expunged while COPY is being handled, but this will be resolved later when COPY is implemented. 1.2.4. Flag updates When STORE is received by a master, it checks if it has higher modseqs in messages than the modseq in STORE parameter. Then it applies all the changes and forwards the changes to other servers using possibly updated modseqs. For messages that had higher initial modseqs their current flags are sent back to the originating server to fix a potential desync. If STORE is initiated by the master, there are no flag desynchronization possibilities. The correctness of applying conflicting changes could be improved by including a "highest modseq of messages before this STORE" parameter. The applying server (master) could then find the flags from that point in time and apply changes on top of it. And finally on the top would be applied newer changes (that were already once applied). There could still be conflicting changes, but the result would be better for updates that had been delayed for a long time. Finding these flag changes and old flag values isn't simple though, so this won't be at least in initial implementation. STORE UNCHANGEDSINCE (CONDSTORE extension) is initially implemented by grabbing a master lock. If there are simultaneous STOREs in other servers, they'll temporarily have lower modseqs and different message flags. Eventually when the STOREs reach the master, they will be applied on top of the CONDSTORE modification. So even if normal STOREs were sent slightly earlier than CONDSTORE, they'd eventually be applied on top of the CONDSTORE change. This makes the STORE UNCHANGEDSINCE work correctly (even if with a slight delay), and our multi-master replication should be fully compatible with the CONDSTORE spec. 1.3. Replication receiving Replication receives 3 different types of data from remote sender: 1. Replies to commands this server's replication sender sent 2. Message texts (as part of SAVE) 3. Metadata updates (SAVEs, STORE, EXPUNGE, etc.) Command replies are handled by directly forwarding them to the replication sender process via a pipe/socket. 1.3.1. Receiving SAVEs The message text part of the SAVE is stored in a file. The file format depends on the server's configured default data format. For now we consider only maildir format where the data is written as-is to the file. This file is later hard linked to the destination maildir directory. If multiple partitions are used, the file should be written to the correct partition. The initial code could write the file to the destination mailbox's tmp/ directory to make sure the partition is correct. There may be some permission problems with this approach though, a separate replication drop directory would be nicer. The rest of the SAVE is processed similarly to other metadata updates. The path to the written file is added as new metadata to the SAVE command. 1.3.2. Metadata updates The primary replication receiver process's job is to read data from the network as fast as possible. Processing the actual updates can take a while, so the work is distributed to multiple worker processes. As discussed above, the primary replication receiver process still writes message texts to files. The workers could do this as well, but it would mean that the text needs to be sent via IPC to the workers, which adds extra overhead. If the message writing really becomes the bottleneck, this change could be done. The replication receiver tries to pass the same user's updates to the same workers so they can benefit from keeping mailboxes open longer. Each worker could be limited to e.g. 100 open mailboxes. The worker handles the SAVE/EXPUNGE/STORE commands using the normal mailbox handling functions. After the changes were successfully committed, commands that had tags are replied as having succeeded. If a change failed, a failure reply is sent instead for it. Replication receiver passes this reply to the replication sender (not to be confused by remote command replies which are also forwarded), which in turn passes it to the originating server. ------ 2. Mailbox synchronization for existing mailboxes. - As described in the replication protocol. - IMAP UID conflict resolution. 3. Mailbox list synchronization. - Track creates, deletes, renames. - Add unique global mailbox ID which is preserved across renames and allows replication to perform renaming (and symlinking?) 4. More features 4.1. Global UIDs - Save GUIDs to messages (dbox: metadata, maildir: dovecot-uidlist) - Tracking GUIDs globally - Implement COPY command - Implement SAVEs sending only global UIDs and FETCHing unknown ones 4.2. Support for untrusted user-run replication processes - Make SAVE and work without being a master - Make CONDSTORE's STORE UNCHANGEDSINCE work without being a master 4.3. Transactions - Wrap changes into transactions just like they're in the originating Dovecot's transaction (e.g. TX-BEGIN, TX-END commands) - On receiving side a transaction is handled in a single Dovecot transaction by a single worker. 4.4. Cluster-wide sync - Broadcast a ping packet and wait for a reply from all servers on the cluster. This ensures that all changes before the ping packet was sent have been received. - Then ping replication receiver process and have it ping worker processes to wait until all received changes have been committed. - CHECK command could do it. - CONDSTORE/QRESYNC commands that take modseq parameters could maybe do it? Or would it be too slow? It would at least increase reliability. We could probably also figure out some rules when the sync is needed and when not. 5. dbox support - Message text can't just be written to a file and renamed. At minimum some dbox metadata needs to be written to it. - Files can't be shared between mailboxes by hard linking like they can be with maildir

1 1

[Dovecot] Timeout when using icedove's searching on a large amount of email.
by guillaume.yziquel＠free.fr 05 May '08

05 May '08

Hello list. I'm using dovecot as a local server, where I'm the only user (or so), and I appreciate using icedove searching functionality to roam through the whole inbox in search of some specific string in the body of my emails. Unfortunately, recently, such a search has become impossible, since the search takes too much time, and the connection times out. I've looked throughout the .conf file and the documentation, but I haven't found where I could increase the delay until timeout. If someone knew the right option in the .conf file, it would be really great... Guillaume Yziquel.

2 1

[Dovecot] imapsync + gnupg?
by Neal Becker 05 May '08

05 May '08

I'd like to mirror my dovecot server (which only serves myself). I'm thinking to use gmail. Problem is, I don't want all my mail going to gmail in plain text. I'd like to gpg+mime encrypt each mail using my public key. Any thoughts on how to do this?

2 1

[Dovecot] Inotify instance limit for user exceeded, disabling.
by Anders 05 May '08

05 May '08

Hi. What does the warning "Inotify instance limit for user exceeded, disabling." mean, and how do I get rid of it? I assume that I have to change a limit somewhere? Regards, Anders.

2 4

[Dovecot] mail_chroot problems / dovecot 1.1rc4
by Michal Soltys 05 May '08

05 May '08

Hello I've noticed a little strange behaviour of that option. For example with following settings: system user: admin, with its home directory as /home/admin dovecot options: as reported by dovecot -n (in attachment) dovecot will try to chroot into /home/home/admin with the following message in logs, in my case: Fatal: chdir(/home/home/admin) failed with uid 1999: No such file or directory The same happens if I use per-user chroot= option in userdb, f.e. in passwd-file I've noticed it's just now, as I've always used explicit chroot dirs specified through passwd-file with /./ (which works perfectly fine, except harmless double slashes in logs), and didn't bother with single system user (I made some chroot/lock related bugreports in distant 1.0rcXX past). Cheers Michal # 1.1.rc4: /etc/dovecot.conf base_dir: /var/dovecot/ protocols: imap imaps pop3 pop3s ssl_listen: * ssl_ca_file: /etc/ssl/cert_bundle.pem ssl_cert_file: /etc/ssl/ca_ppgk/certs/fetch_crt.pem ssl_key_file: /etc/ssl/ca_ppgk/private/fetch_key.pem verbose_ssl: yes login_dir: /var/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting: Dovecot IMAP server ready. login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_process_size: 32 valid_chroot_dirs: /home mail_chroot: /home verbose_proctitle: yes first_valid_uid: 1999 first_valid_gid: 10 mail_location: maildir:~/mspace/maildir:INDEX=~/mspace/index:INBOX=~/mspace/inbox:CONTROL=~/mspace/control mail_debug: yes fsync_disable: yes lock_method: flock mbox_read_locks: flock mbox_write_locks: flock mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle delay-newmail tb-extra-mailbox-sep imap_client_workarounds(imap): outlook-idle delay-newmail tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh namespace: type: private separator: / inbox: yes list: yes subscriptions: yes auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: passwd-file args: /etc/dovecot.z1.passwd passdb: driver: passwd-file args: /etc/dovecot.passwd passdb: driver: passwd userdb: driver: passwd-file args: /etc/dovecot.z1.passwd userdb: driver: passwd-file args: /etc/dovecot.passwd userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: quota: maildir

2 4

[Dovecot] libsieve support
by Mateusz Kijowski 05 May '08

05 May '08

Hi, I was wondering - are there any plans on switching the sieve plug-in to use libsieve (http://libsieve.sourceforge.net/)? The rationale of using this library is simple - cmusieve codebase seems stagnant and libsieve promises to implement some new sieve rfc drafts. -- matik

2 1

[Dovecot] v1.1.rc5 released
by Timo Sirainen 05 May '08

05 May '08

http://dovecot.org/releases/1.1/rc/dovecot-1.1.rc5.tar.gz http://dovecot.org/releases/1.1/rc/dovecot-1.1.rc5.tar.gz.sig I've finally gone through all my unread mails and fixed most of the reported bugs. There are only a couple ones left that I can't reproduce. Please report if you still have any problems with rc5, even if you already have reported the same bug before. v1.1.rc6 will hopefully be the last RC before v1.1.0. + Support cross-realm Kerberos 5 authentication. Based on patch by Zachary Kotlarek. + Added dict_db_config setting to point to a Berkeley DB config file. + If mail_chroot ends with "/.", remove chroot prefix from home directory. - Fixed several bugs and memory leaks in ACL plugin. LIST and LSUB may have listed mailboxes where user had no 'l' access. STORE could have been used to update any flags without appropriate access. - mbox: Valid-looking From_-lines in message bodies caused the message to be split to two messages (broken since v1.0). - Plugin initialization hooks were called in wrong order, possibly causing problems when multiple plugins were used at the same time. - Expire plugin was broken - LIST-EXTENDED options were ignored. - LDAP: Static attribute names weren't working correctly - deliver: mail_uid and mail_gid settings weren't used. - pop3 + maildir++ quota: maildirsize file wasn't created if it didn't exist already. - dnotify: Waiting for dotlock to be deleted used 100% CPU

2 1

[Dovecot] dovecot 1.1.rc3 assertion failed at index_mailbox_set_recent_uid while deleting message with thunderbird.
by Diego Liziero 05 May '08

05 May '08

To some users happens this assertion failure while deleting a message. dovecot: Mar 10 08:40:44 Panic: IMAP(user): file index-sync.c: line 39 (index_mailbox_set_recent_uid): assertion failed: (seq_range_exists (&ibox->recent_flags, uid)) dovecot: Mar 10 08:40:44 Error: IMAP(user): Raw backtrace: [see bleow] dovecot: Mar 10 08:40:44 Error: child 17683 (imap) killed with signal 6 And the message doesn't get deleted. Here one of the backtraces (gdb) bt #0 0x00ae9402 in __kernel_vsyscall () #1 0x00725ba0 in raise () from /lib/libc.so.6 #2 0x007274b1 in abort () from /lib/libc.so.6 #3 0x080ce02d in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfd8c6a4 "=???\016\b'") at failures.c:424 #4 0x080cdc3c in i_panic (format=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:187 #5 0x0809bb4a in index_mailbox_set_recent_uid (ibox=0x88ca768, uid=1041) at index-sync.c:39 #6 0x0808085f in mbox_sync_loop (sync_ctx=0xbfd8c8d0, mail_ctx=0xbfd8cac4, partial=true) at mbox-sync.c:457 #7 0x08081361 in mbox_sync (mbox=0x88ca768, flags=18) at mbox-sync.c:1504 #8 0x08079cf9 in mbox_transaction_commit (t=0x88c5550, log_file_seq_r=0xbfd8cbc8, log_file_offset_r=0xbfd8cbc0) at mbox-transaction.c:45 #9 0x0809c69e in index_transaction_commit (_t=0x88cb1c0, uid_validity_r=0xbfd8cc48, first_saved_uid_r=0xbfd8cc44, last_saved_uid_r=0xbfd8cc40) at index-transaction.c:105 #10 0x0805b0cf in cmd_copy (cmd=0x88b75c0) at cmd-copy.c:141 #11 0x0805f079 in cmd_uid (cmd=0x88b75c0) at cmd-uid.c:26 #12 0x0805f9a9 in client_command_input (cmd=0x88b75c0) at client.c:546 #13 0x0805fa3e in client_command_input (cmd=0x88b75c0) at client.c:595 #14 0x080601c5 in client_handle_input (client=0x88b7368) at client.c:636 #15 0x080603de in client_input (client=0x88b7368) at client.c:691 #16 0x080d5ba0 in io_loop_handler_run (ioloop=0x88b59b0) at ioloop-epoll.c:201 #17 0x080d4e28 in io_loop_run (ioloop=0x88b59b0) at ioloop.c:301 #18 0x08067dbc in main (argc=Cannot access memory at address 0x413a ) at main.c:293 (gdb) bt full #0 0x00ae9402 in __kernel_vsyscall () No symbol table info available. #1 0x00725ba0 in raise () from /lib/libc.so.6 No symbol table info available. #2 0x007274b1 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080ce02d in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfd8c6a4 "=???\016\b'") at failures.c:424 backtrace = 0x88add50 "/usr/libexec/dovecot/imap [0x80ce024] -> /usr/libexec/dovecot/imap [0x80cdc3c] -> /usr/libexec/dovecot/imap [0x809bb4a] -> /usr/libexec/dovecot/imap [0x808085f] -> /usr/libexec/dovecot/imap(mbox_sync+"... #4 0x080cdc3c in i_panic (format=0x80e1750 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:187 args = 0xbfd8c6a4 "=???\016\b'" #5 0x0809bb4a in index_mailbox_set_recent_uid (ibox=0x88ca768, uid=1041) at index-sync.c:39 __PRETTY_FUNCTION__ = "index_mailbox_set_recent_uid" #6 0x0808085f in mbox_sync_loop (sync_ctx=0xbfd8c8d0, mail_ctx=0xbfd8cac4, partial=true) at mbox-sync.c:457 mail = {uid = 268, idx_seq = 133265, keywords = {arr = {buffer = 0x88d3770, element_size = 3218655176}, v = 0x88d3770, v_modifiable = 0x88d3770}, flags = 172 '???', uid_broken = 1, expunged = 0, pseudo = 1, from_offset = 143457960, body_size = 579466361970685888, offset = 143456608, space = 578778797692682360} rec = (const struct mail_index_record *) 0xb7e6f008 uid = 1041 messages_count = 3518 offset = 563 ret = <value optimized out> expunged = false skipped_mails = false uids_broken = false #7 0x08081361 in mbox_sync (mbox=0x88ca768, flags=18) at mbox-sync.c:1504 ret = <value optimized out> #8 0x08079cf9 in mbox_transaction_commit (t=0x88c5550, log_file_seq_r=0xbfd8cbc8, log_file_offset_r=0xbfd8cbc0) at mbox-transaction.c:45 mt = (struct mbox_transaction_context *) 0x88cb1c0 mbox = (struct mbox_mailbox *) 0x88ca768 lock_id = 3 ret = 0 #9 0x0809c69e in index_transaction_commit (_t=0x88cb1c0, uid_validity_r=0xbfd8cc48, first_saved_uid_r=0xbfd8cc44, last_saved_uid_r=0xbfd8cc40) at index-transaction.c:105 itrans = (struct mail_index_transaction *) 0x0 seq = 7 offset = 20544 #10 0x0805b0cf in cmd_copy (cmd=0x88b75c0) at cmd-copy.c:141 client = (struct client *) 0x88b7368 storage = (struct mail_storage *) 0x88b6c80 destbox = (struct mailbox *) 0x88ca768 t = (struct mailbox_transaction_context *) 0x0 search_arg = <value optimized out> messageset = 0x88bb6c0 "224" mailbox = 0x88bb6c8 "Trash" msg = <value optimized out> sync_flags = <value optimized out> imap_flags = <value optimized out> copy_count = 1 uid_validity = <value optimized out> uid1 = <value optimized out> uid2 = <value optimized out> ret = 1 __PRETTY_FUNCTION__ = "cmd_copy" #11 0x0805f079 in cmd_uid (cmd=0x88b75c0) at cmd-uid.c:26 command = (struct command *) 0x6 cmd_name = 0x88bb6b8 "copy" #12 0x0805f9a9 in client_command_input (cmd=0x88b75c0) at client.c:546 client = (struct client *) 0x88b7368 command = <value optimized out> __PRETTY_FUNCTION__ = "client_command_input" #13 0x0805fa3e in client_command_input (cmd=0x88b75c0) at client.c:595 client = (struct client *) 0x88b7368 command = (struct command *) 0x6 __PRETTY_FUNCTION__ = "client_command_input" #14 0x080601c5 in client_handle_input (client=0x88b7368) at client.c:636 _data_stack_cur_id = 3 ret = <value optimized out> remove_io = <value optimized out> handled_commands = false #15 0x080603de in client_input (client=0x88b7368) at client.c:691 cmd = <value optimized out> output = (struct ostream *) 0x88b74ec bytes = 24 __PRETTY_FUNCTION__ = "client_input" #16 0x080d5ba0 in io_loop_handler_run (ioloop=0x88b59b0) at ioloop-epoll.c:201 ctx = <value optimized out> event = (const struct epoll_event *) 0x88b5ae8 list = (struct io_list *) 0x88b7568 io = (struct io_file *) 0x88c0dd0 tv = {tv_sec = 1799, tv_usec = 999737} t_id = 2 msecs = <value optimized out> ret = 1 i = 0 j = 0 call = <value optimized out> #17 0x080d4e28 in io_loop_run (ioloop=0x88b59b0) at ioloop.c:301 No locals. #18 0x08067dbc in main (argc=Cannot access memory at address 0x413a ) at main.c:293 No locals. Regards, Diego.

2 6

[Dovecot] Users w/o acl access appear to be subscribed to public folders (1.1b8)
by Adam McDougall 05 May '08

05 May '08

I noticed this today, I had a user outside of our department test out dovecot. They were using squirrelmail and I noticed that dovecot thinks this user is subscribed to ALL public folders even though a dovecot ACL prevents all access. I'm pretty sure access is still denied. I was able to reproduce this with a guest account I added: l lsub "" "#shared/decs/%" * LSUB (\Noselect) "/" "#shared/decs/linuxadmin" * LSUB (\Noselect) "/" "#shared/decs/jbossadmin" * LSUB () "/" "#shared/decs/support" * LSUB () "/" "#shared/decs/receipts" * LSUB (\Noselect) "/" "#shared/decs/pcadmin" * LSUB () "/" "#shared/decs/network" * LSUB (\Noselect) "/" "#shared/decs/printmaster" * LSUB () "/" "#shared/decs/postmaster" * LSUB (\Noselect) "/" "#shared/decs/unixadmin" * LSUB () "/" "#shared/decs/security" * LSUB (\Noselect) "/" "#shared/decs/webmaster" l OK Lsub completed. This only seems to happen when the acl plugin is enabled. Without the acl plugin, these are not listed as subscriptions. After deleting /egr/mail/shared/decs/dovecot-acl-list and re-enabling the acl plugin, I get this: l lsub "" "#shared/decs/%" * LSUB () "/" "#shared/decs/unixadmin" * LSUB () "/" "#shared/decs/support" * LSUB () "/" "#shared/decs/security" * LSUB () "/" "#shared/decs/printmaster" * LSUB () "/" "#shared/decs/postmaster" * LSUB () "/" "#shared/decs/pcadmin" * LSUB () "/" "#shared/decs/network" * LSUB () "/" "#shared/decs/linuxadmin" * LSUB () "/" "#shared/decs/webmaster" * LSUB () "/" "#shared/decs/jbossadmin" l OK Lsub completed. Is it related, or is it different just because a new dovecot-acl-list got created by another user already (but is mode 700?)

2 3

[Dovecot] Compiling plugins
by Juan Asensio Sánchez 05 May '08

05 May '08

Hi I use Debian Etch 4.0r3. I am trying to compile antispam plugin, but i have no idea about how to do it cause i have never done so. I have compiled source packages for Debian with some instructions in some pages, but i can not compile this plugin separately. Can anyone help me or give some tips to do this (yes, i have google'd it yet, but no good results). This is the result trying to compile antispam plugin: ---------------------------------------------------- pdc-server:~/tmp/antispam# ls -l ../dovecot-1.1.rc5/ total 3508 -rw-r--r-- 1 root root 322488 2008-05-05 00:30 aclocal.m4 -rw-r--r-- 1 root root 551 2008-03-10 23:23 AUTHORS -rw-r--r-- 1 root root 804 2008-05-05 01:01 autogen.sh -rw-r--r-- 1 root root 1470323 2008-05-05 00:31 ChangeLog -rwxr-xr-x 1 root root 3707 2008-03-10 23:23 compile -rwxr-xr-x 1 root root 44892 2008-02-17 21:10 config.guess -rw-r--r-- 1 root root 14778 2008-05-05 00:31 config.h -rw-r--r-- 1 root root 13687 2008-05-05 00:31 config.h.in -rwxr-xr-x 1 root root 18343 2008-05-04 12:13 config.rpath -rwxr-xr-x 1 root root 33387 2008-02-17 21:10 config.sub -rwxr-xr-x 1 root root 1086089 2008-05-05 00:30 configure -rw-r--r-- 1 root root 56429 2008-05-05 00:30 configure.in -rw-r--r-- 1 root root 3980 2008-03-10 23:23 COPYING -rw-r--r-- 1 root root 26536 2008-03-10 23:23 COPYING.LGPL -rw-r--r-- 1 root root 1023 2008-03-10 23:23 COPYING.MIT drwxr-xr-x 4 root root 4096 2008-05-05 01:01 debian -rwxr-xr-x 1 root root 15936 2008-03-10 23:23 depcomp drwxrwxrwx 3 root root 4096 2008-05-05 01:01 doc -rw-r--r-- 1 root root 187 2008-03-10 23:23 dovecot-config.in.in -rw-r--r-- 1 root root 46655 2008-05-05 00:30 dovecot-example.conf drwxr-xr-x 3 root root 4096 2008-05-05 01:01 dovecot-managesieve drwxr-xr-x 3 root root 4096 2008-05-05 01:01 dovecot-sieve -rw-r--r-- 1 root root 1235 2008-03-10 23:23 INSTALL -rwxr-xr-x 1 root root 9233 2008-03-10 23:23 install-sh -rw-r--r-- 1 root root 199984 2008-04-05 19:05 ltmain.sh -rw-r--r-- 1 root root 656 2008-03-10 23:23 Makefile.am -rw-r--r-- 1 root root 24492 2008-05-05 00:30 Makefile.in -rwxr-xr-x 1 root root 11014 2008-03-10 23:23 missing -rw-r--r-- 1 root root 71910 2008-05-05 00:30 NEWS -rw-r--r-- 1 root root 1344 2008-03-16 10:58 README drwxrwxrwx 26 root root 4096 2008-05-05 00:32 src -rw-r--r-- 1 root root 0 2008-03-10 23:23 stamp.h.in -rw-r--r-- 1 root root 14980 2008-05-05 00:30 TODO pdc-server:~/tmp/antispam# ls -l total 144 -rw-r--r-- 1 root root 7359 2008-05-05 00:31 antispam.7 -rw-r--r-- 1 root root 4864 2008-05-05 00:31 antispam-plugin.c -rw-r--r-- 1 root root 2418 2008-05-05 00:31 antispam-plugin.h -rw-r--r-- 1 root root 13279 2008-05-05 00:31 antispam-storage-1.0.c -rwxr-xr-x 1 root root 14026 2008-05-05 00:58 antispam-storage-1.1 -rw-r--r-- 1 root root 14308 2008-05-05 00:31 antispam-storage-1.1.c -rw-r--r-- 1 root root 18101 2008-05-05 00:31 COPYING -rw-r--r-- 1 root root 4351 2008-05-05 00:31 crm114-exec.c -rw-r--r-- 1 root root 547 2008-05-05 00:31 debug.c -rw-r--r-- 1 root root 1854 2008-05-05 00:31 defconfig -rw-r--r-- 1 root root 5087 2008-05-05 00:31 dspam-exec.c -rw-r--r-- 1 root root 7631 2008-05-05 00:31 mailtrain.c -rw-r--r-- 1 root root 2244 2008-05-05 00:31 Makefile -rw-r--r-- 1 root root 372 2008-05-05 00:31 NOTES -rw-r--r-- 1 root root 23 2008-05-05 00:31 README -rw-r--r-- 1 root root 1855 2008-05-05 00:31 signature.c -rw-r--r-- 1 root root 617 2008-05-05 00:31 signature.h -rw-r--r-- 1 root root 3859 2008-05-05 00:31 signature-log.c pdc-server:~/tmp/antispam# grep -v '^ *$#.*$\?$' .config DOVECOT=/root/tmp/dovecot-1.1.rc5 DOVECOT_VERSION=1.1 # CURRENTLY BROKEN BACKEND=dspam-exec INSTALLDIR=/usr/lib/dovecot/modules/imap USER=root GROUP=root PLUGINNAME=antispam pdc-server:~/tmp/antispam# make install cc -o .o cc: no input files make: *** [.o] Error 1 pdc-server:~/tmp/antispam# ---------------------------------------------------- Thanks in advance.

1 1

Re: [Dovecot] Dovecot SASL doesn't seem to be working with Postfix
by Jeremy Gillick 05 May '08

05 May '08

w00t. Thanks everyone for all your input. It works now. The key was to set mydestination to "localhost, localhost.localdomain". From there Postfix started using Dovecot LDA (deliver) and I was able to track the rest of the problems down through the log files. Here's my latest config for anybody interested: ## DOVECOT ## # 1.0.13: /usr/local/etc/dovecot.conf log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot-info.log protocols: imap pop3 imaps pop3s disable_plaintext_auth: no verbose_ssl: yes login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login mail_location: maildir:~/Maildir dotlock_use_excl: yes maildir_copy_with_hardlinks: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 pop3_uidl_format: %08Xu%08Xv auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: static args: uid=postfix gid=postfix home=/home/vmail/%u socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /usr/local/var/run/dovecot/auth-master mode: 384 user: vmail group: vmail ## POSTFIX main.cf ## broken_sasl_auth_clients = yes command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 default_privs = nobody html_directory = no inet_interfaces = $myhostname, localhost mail_owner = postfix mail_spool_directory = /home/vmail/ mailq_path = /usr/bin/mailq manpage_directory = /usr/local/man mydestination = localhost, localhost.localdomain mydomain = $myhostname myhostname = mixermixer3.com newaliases_path = /usr/bin/newaliases queue_directory = /var/spool/postfix/ readme_directory = no sample_directory = /etc/postfix sendmail_path = /usr/sbin/sendmail setgid_group = postdrop smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_sasl_auth_enable = yes smtpd_sasl_exceptions_networks = $mynetworks smtpd_sasl_local_domain = $myhostname smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_tls_CAfile = /etc/postfix/ssl/smtpd.pem smtpd_tls_cert_file = /etc/postfix/ssl/smtpd.pem smtpd_tls_key_file = /etc/postfix/ssl/smtpd.pem smtpd_tls_loglevel = 1 smtpd_tls_session_cache_timeout = 3600s unknown_local_recipient_reject_code = 550 virtual_mailbox_domains = $myhostname virtual_transport = dovecot ## POSTFIX (a few lines from master.cf) ## smtp inet n - n - - smtpd smtps inet n - n - - smtpd -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject # Dovecot LDA dovecot unix - n n - - pipe flags=DRhu user=nobody:mail argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${recipient} Thanks Everyone! - Jeremy

1 0