- dovecot - dovecot.org

Re: [Dovecot] imap memory footprint rather large
by Timo Sirainen 21 Aug '07

21 Aug '07

[back to list] On Tue, 2007-08-14 at 14:50 +0300, Timo Sirainen wrote: > On Tue, 2007-08-14 at 13:47 +0200, martin f krafft wrote: > > also sprach Timo Sirainen <tss(a)iki.fi> [2007.08.14.1250 +0200]: > > > mmap()ing a 200MB file doesn't mean that it immediately uses 200MB > > > of memory. It just reserves that much virtual space. Even if it > > > happens to read the whole 200MB, it shouldn't be much different > > > from simply reading the file in small parts, in which case kernel > > > places the same 200MB to page cache. > > > > I am aware of that. I just wonder why dovecot has to read the entire > > file into memory, via read() or mmap(). > > It doesn't. It mmaps it entirely so it doesn't waste CPU mmaping and > unmapping parts of it. It's then accessed only when and where needed. It probably does access the whole file though. This is because all the cached data for messages are stored next to each others. So now that your client fetches only INTERNALDATE it doesn't really need to read much, but because mmaped pages are read 4kB at a time, it practically reads the whole file. Perhaps this could use some work in future too, so that fields that are accessed together are stored in cache file close together. But then if multiple clients are used, there can be conflicting access patterns, and what's good for one client would be bad for another. Data could also be duplicated to reduce disk reads for different access patterns, but that comes at the cost of disk space..

4 4

[Dovecot] Dovecot fails almost every 20 minutes exactly
by Patrick - South Valley Internet 21 Aug '07

21 Aug '07

Hi all, We just set up our new mail servers using Postfix and Dovecot. We have implemented a load balanced solution to where our two mail servers have a common NFS share. Everything is working great. Everything but Dovecot for the most part. In testing, Dovecot was great. Now that we're in the production environment, we've noticed that every 20 minutes, Dovecot will stop running. But it gets even weirder - it appears that Machine A will stop, then 20 mins later after we issue a 'service dovecot restart', Machine B will stop. Then the whole process starts over when we restart Dovecot on Machine B. I can't see anything out of the ordinary in the log files either. I'm at a loss as to what's going on. Here's my dovecot -n: ################################ # 1.0.3: /etc/dovecot.conf listen: * ssl_cert_file: /usr/share/ssl/hermes.garlic.com/hermes.garlic.com.cert.pem ssl_key_file: /usr/share/ssl/hermes.garlic.com/hermes.garlic.com.privatekey.pem login_dir: /var/run/dovecot-login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login first_valid_uid: 200 mail_location: maildir:/home/%u/Maildir:INDEX=MEMORY maildir_copy_with_hardlinks: yes maildir_copy_preserve_filename: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 auth default: user: dovecot-auth username_format: %Lu passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix ################################ Any ideas what could be wrong? I don't think that setting up a cron to restart both dovecot servers is an idea solution - I'm hoping this is an easy fix. Thanks in advance. Patrick

2 2

[Dovecot] Dovecot IDLE bandwidth usage
by Fredrik Tolf 20 Aug '07

20 Aug '07

Hi list, I just recently discovered the IMAP IDLE command and tried it out with my Dovecot server. It seems to work fine and all, but it seems to be using quite a deal of bandwidth, because of the "* OK Still here" messages sent by the server at two minute intervals. I would argue that is a bit of a problem when using the IDLE command for "Push mail" on a mobile phone. I don't know exactly what parts of the packets that mobile carriers collect payment for, but the Ethernet bandwidth consumed by one such message is 149 bytes (83 bytes for the message itself, and 66 bytes for the TCP ACK from the client). At two minute intervals, that becomes over 100 KB per day, and over 3 MB per month. At GRPS rates typical here in Sweden, that becomes about 5-7 USD per month. That might not seem tremendously much, but seeing how it is what takes up the vast majority of the IMAP bandwidth, it is still quite annoying. Are those messages really necessary? I mean, Dovecot still times out after half an hour if the client doesn't "finish" the IDLE command and re-issues it, right? (Coincidentally, I would like to argue that it would be better not to time out at all during an IDLE command and just turn on TCP keepalives instead, but that is the minor issue here) I'll apologize in advance if this is already a tunable option, but I haven't found anything about it in the configuration file. Cheers, Fredrik Tolf

5 5

[Dovecot] MANAGESIEVE patch v6 for dovecot 1.0.3
by Stephan Bosch 18 Aug '07

18 Aug '07

Hello dovecot users, I have updated the MANAGESIEVE patch to (hopefully) fix the compilation issues reported by Robin Breathe. This is a patch against the latest stable release 1.0.3. It currently won't compile with 1.1 due to significant changes in the master code. Change Log V6 ------------- - Corked the client output stream while producing the capability greeting and on other some other occasions as well. Some naive client implementations expect to receive this as a single tcp frame and it is a good practice to do so anyway. Using this change the Thunderbird sieve extension (v0.1.1) seemed to work. However, scripts larger than a tcp frame still caused failures. All these issues are fixed in the latest version of the sieve add-on (currently v0.1.4). - Cleaned up the new proxy source. My editor made the indentation a complete mess in terms of TABs vs spaces. - Added TRYLATER response codes to BYE and NO messages where appropriate. - Recopied the libsieve library into this patch to incorporate any changes that were made (only sieve-cmu.c still needs to be compared to the old cmu-sieve.c). This also solves the __attribute__((unused)) GCC dependencies. These were fixed long ago by Timo.... the code duplication beast strikes again. (Will have to do this again for dovecot 1.1) - Removed spurious return value from void function in src/lib-sieve/sieve-implementation.c as reported by Robin Breathe. GCC fails to report these issues. The function involved is currently not used and serves only as an example on how dovecot could support multiple sieve back-ends... This patch still includes (yet another) instance of the CMU Sieve source, as explained in one of my previous e-mails (http://dovecot.org/list/dovecot/2006-July/015016.html) The patch can be downloaded at: http://sinas.rename-it.nl/~sirius/dovecot-1.0.3-MANAGESIEVE-v6.diff.gz The README.managesieve is located in the root of the dovecot source tree after applying the patch. Have fun testing the patch. Don't hesitate to notify me when there are problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl IRC: Freenode, #dovecot, S[r]us

3 5

[Dovecot] Migrating from imapd to dovecot - how to keep user's Mail folders
by Patrick - South Valley Internet 17 Aug '07

17 Aug '07

Hi all, We're migrating off our current mail server with runs postfix and imapd on an AIX machine to Postfix and Dovecot running linux. Our customers have a lot of IMAP folders with folders they configured, which are currently located in their /home directory on the current mail server. The new mail server will have them at the same location. How do I configure Dovecot so that when a customer connects via IMAP using their favorite e-mail client, that the folders will be there? Thanks in advance. Patrick

1 0

[Dovecot] Dovecot 1.0.3: LDA + quota plugin
by Przemek Grądzki 17 Aug '07

17 Aug '07

I have problem with quota limit settings for "deliver" - global plugin settings overrides userdb quota setting !!! imap and pop3 works fine. Quota limit for user is set to "quota=maildir:ignore=Kosz:storage=512". And "maildirsize" file in user's mail_location contains "524288S" line - good. But when I set: ---------------- plugin { quota = maildir:ignore=Kosz:storage=102400 } ---------------- in "/etc/dovecot.conf", quota limit is set to "104857600S" in "maildirsize" file - BAD. Local delivery is configured as in http://wiki.dovecot.org/LDA, quota plugin is set in /etc/dovecot.conf: ---------------- protocol lda { ...... mail_plugins = cmusieve quota #mail_plugin_dir = /usr/lib/dovecot/lda ..... } ---------------- Per-user quota is configured like this (from LDAP): ---------------- user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mailQuotaSize=quota=maildir:ignore=Kosz:storage pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,mailQuotaSize =userdb_quota=maildir:ignore=Kosz:storage ---------------- "mailQuotaSize" attribute contains quota limit in kilobytes. /etc/dovecot.conf: ---------------------------- # 1.0.3: /etc/dovecot.conf log_path: /dev/stderr protocols: imap imaps pop3s managesieve login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_executable(managesieve): /usr/libexec/dovecot/managesieve-login first_valid_uid: 8 first_valid_gid: 12 mail_location: maildir:%h/Maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_executable(managesieve): /usr/libexec/dovecot/managesieve mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/managesieve pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_client_workarounds(managesieve): auth default: mechanisms: plain login gssapi passdb: driver: ldap args: /etc/dovecot-ldap.conf userdb: driver: prefetch userdb: driver: ldap args: /etc/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 438 plugin: quota: maildir:ignore=Kosz:storage=102400 ----------------- PS. Sorry for my english.

1 0

[Dovecot] High CPU utilization on Solaris 10
by Jon Alessandrello 17 Aug '07

17 Aug '07

I've read similar post about such an issue, but have not seen any posted solutions. I have a clean install of Solaris 10 running a Sunfire v440 w/postfix. I am using dovecote solely for SMTP authentication, and it is working just fine except while running top I constantly see: 548 root 1 30 0 5240K 3416K run 522:51 49.45% dovecot-auth The process never crashes, nor are there any errors. I ran a test devel box with an almost identical setup of postfix and dovecot on Solaris 9 (V240) and the functionality is identical, but with no high CPU usage. The following are my configs and system info: dovecot --version 1.0.3 OS Solaris 10 Sparc (Patched) No NFS all local ufs dovecot -n # 1.0.3: /usr/local/etc/dovecot.conf log_path: /var/log/dovecot protocols: none ssl_disable: yes login_dir: /usr/local/var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login auth default: mechanisms: plain login passdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf userdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf socket: type: listen client: path: /opt/pmx/postfix/var/private/auth mode: 432 user: postfix group: postfix cat dovecot-ldap.conf: hosts = localhost dn = cn=priv-account,ou=container,dc=example,dc=com dnpass = mydnpasswd ldap_version = 3 base = ou=people,dc=example,dc=com scope = subtree pass_attrs = uid=user,userPassword=password pass_filter = (&(objectClass=posixAccount)(uid=%u)) default_pass_scheme = CRYPT Any help would be great!! -Jon

5 6

[Dovecot] dovecot notifying a database about total number of messages.
by Erland Nylend 17 Aug '07

17 Aug '07

Hello, in addition to our mail system, we have some other systems which needs to know how many messages, and how many unread messages, a user has in his/her inbox (maildir). Currently, we have a mod_perl script which takes a mail address as input, and then traverses the maildir, and outputs the number of unread messages. Our challenge is that the maildirs are located on a SAN, and traversing a maildir is slow. We currently have more than 100 requests pr. second, which we handle with caching, and spreading the load across different servers. I'm looking into the possibility of either using dovecots index files, or having dovecot update a database with total number of messages/number of unread messages. Question: 1) if I'm using perl, is there any way I can read in the information in the index files? Is the data formatted in some standarized way, so that I can parse it, or use a module which read in the data? (if I can read the indexes in wrapper scripts to imap/pop/deliver, then I could probably update a database with the information) 2) how would you proceed, if the aim is to end up with dovecot notifying a database regarding the number of messages, and unread messages? -- Erland Nylend

2 1

[Dovecot] ssh tunneling and maildir
by Joel Becker 17 Aug '07

17 Aug '07

Hello everyone, please keep my CC'd, I'm not on the list. I've been using dovecot happily via an ssh tunnel. My mutt is configured to use IMAP, and the tunnel is as follows in my muttrc: set tunnel="ssh -q mymailhost 'MBOX_LOCKS=fcntl MAIL=/home/jlbec/mail /usr/libexec/dovecot/imap'" It should be pretty obvious that this logs into mymailhost as me (via SSH keyauth) and runs /usr/libexec/dovecot/imap directly as a foreground process. Setting MAIL, it points dovecot imap to my mbox folders. dovecot imap speaks imap over stdin/out, this goes over the ssh to mutt. It works great. Really great. I don't need root access at all, the server isn't running dovecot globally, and I get IMAP through firewalls if I can ssh through a firewall/vpn. I'm using mbox and fcntl, so my MTA has no trouble working in concert. However, this doesn't appear to be documented anywhere. A friend showed it to me, and I don't even remember who that was. I'm now considering changing to maildir folders. I figure dovecot imap might figure out a folder is maildir when stat says S_IFDIR, but without documentation, I can't figure out how to tell dovecot imap to use maildir when I create a folder. I'm also unsure how this will interact with my MTA. For the latter, I'm thinking I just need to configure my MTA to use dovecot's deliver program. For the former, is there some magic environment variable I need to add to my tunnel command line? Thanks Joel -- "The first requisite of a good citizen in this republic of ours is that he shall be able and willing to pull his weight." - Theodore Roosevelt http://www.jlbec.org/ jlbec(a)evilplan.org

4 8

[Dovecot] Recover mailboxes
by Jordi Espasa Clofent 17 Aug '07

17 Aug '07

HI all, Some time ago I've user(a)domain.com in maildir mailbox with their own mails, their folders, subfolders, indexes files.. etc My server was missconfigured and I've rebuilded all SMTP; I've recreated all the mailboxes with same names that before. Of course, the problem is the content of these mailboxes it's not the same. So, the idea is: add all the contento from user(a)domain.com (old) mailbox to user(a)domain.com (new) mailbox. because I have got a complete copy of (old) mailboxes and their content. ¿How can I do it?

1 0

[Dovecot] High CPU utilization on Solaris 10
by Jon Alessandrello 17 Aug '07

17 Aug '07

Some people had this problem when giving some superoptimization flags with Sun Studio. Did you do that? -Timo No I did not use any optimization. And Dave, I took what you said and just tried both dovecot-1.0.2 dovecot-1.0.rc29 and have the same exact results :(

3 2

[Dovecot] index and control files not being created
by Luis Pablo Michelena Scaffo 16 Aug '07

16 Aug '07

Hello, I'm configurating dovecot 1.0.3 on debian etch. While trying to set public folders, y decided to have the index and control files on the users homes. those homes are mounted by nfs, with disabled access to root. In that scenario, I'm getting the errors i transcribe next: Aug 16 11:21:17 server dovecot: IMAP(user): fchown(/home/user/Maildir/shared-settings/control/.folder/dovecot-uidlis t) failed: Operation not permitted Aug 16 11:21:17 server dovecot: IMAP(user): fchown(/home/user/Maildir/shared-settings/control/.folder/dovecot-uidlis t) failed: Operation not permitted Aug 16 11:21:17 server dovecot: IMAP(user): rename(/inbox/shared/.folder/cur/mailfile, /inbox/shared/.folder/cur/mailfile :2,S) failed: Operation not permitted Aug 16 11:21:17 server dovecot: IMAP(user): fchown(/home/user/Maildir/shared-settings/control/.folder/dovecot-uidlis t) failed: Operation not permitted Aug 16 11:21:17 server dovecot: IMAP(user): rename(/inbox/shared/.folder/cur/mailfile, /inbox/shared/.folder/cur/mailfile :2,S) failed: Operation not permitted Aug 16 11:21:26 server dovecot: IMAP(user): Disconnected: Logged out Aug 16 11:21:27 server dovecot: IMAP(user): fchown(/home/user/Maildir/shared-settings/control/.folder/dovecot-uidlis t) failed: Operation not permitted Aug 16 11:21:27 server dovecot: IMAP(user): rename(/inbox/shared/.folder/cur/mailfile, /inbox/shared/.folder/cur/mailfile :2,S) failed: Operation not permitted Then I touched those files and the errors faded away. Beside that, I was testing the setup, and while trying to delete some files on the shared folder, I got another error: Aug 16 16:38:04 server dovecot: IMAP(user): rename(/inbox/shared/.folder/cur/1183976419.755_0.server:2,Sa, /inbox/shared/.folder/cur/1183976419.755_0.server:2,STa) failed: Operation not permitted Aug 16 16:38:11 server dovecot: IMAP(user): fchown(/home/user/Maildir/shared-settings/control/.folder/dovecot-uidlist) failed: Operation not permitted Aug 16 16:38:11 server dovecot: IMAP(user): rename(/inbox/shared/.folder/cur/1183976419.755_0.server:2,Sa, /inbox/shared/.folder/cur/1183976419.755_0.server:2,STa) failed: Operation not permitted Which seems odd, because I though that as a flag, the deleted flag should be marked on the index files and not directly on the shared folder (where the user hasn't any permission other than reading), ant the same goes to the "read" flag Am I wrong? Is there any workaround to make dovecot behave the way I'd liked? The config follows: login_executable: /usr/lib/dovecot/imap-login mail_extra_groups: mail mail_location: maildir:/home/%u/Maildir:INBOX=/inbox/mdir/%u mmap_disable: yes namespace: type: private separator: / location: maildir:~/Maildir:INBOX=/inbox/mdir/%u inbox: yes namespace: type: public separator: / prefix: Shared folders/ location: maildir:/inbox/shared:INDEX=/home/%u/Maildir/shared-settings/index:CONTROL=/home/%u/Maildir/shared-settings/control auth default: passdb: driver: passwd args: blocking=yes passdb: driver: shadow userdb: driver: passwd Cheers, Luis

1 0

[Dovecot] ANY password option
by Master Abi 16 Aug '07

16 Aug '07

Hi We sometimes migrate mailboxes over to one of our mail servers which has a very handy feature called 'ANY password'. This will allow access to a mailbox using the given username and disregard checking for the password. This is a useful feature for many reasons. Is there a possibility Dovecot could have such a a feature or is there a workaround already. Thanks Master

3 3

Re: [Dovecot] use of deliver from procmail advisable?
by Jerry Yeager 16 Aug '07

16 Aug '07

On Aug 15, 2007, at 5:00 AM, dovecot-request(a)dovecot.org wrote: > Message: 7 > Date: Wed, 15 Aug 2007 00:58:40 +0200 > From: martin f krafft <madduck(a)madduck.net> > Subject: Re: [Dovecot] use of deliver from procmail advisable? > To: dovecot(a)dovecot.org > Message-ID: <20070814225840.GA23721(a)piper.oerlikon.madduck.net> > Content-Type: text/plain; charset="us-ascii" > > also sprach Charles Marcus <CMarcus(a)Media-Brokers.com> > [2007.08.14.2028 +0200]: >>> Well, the whole point of sieve, I believe, is to make it >>> something that an >>> admin would want to let arbitrary users modify on their own >>> recognizance, >>> and the ability to specify arbitrary programs to run would be >>> just *asking* >>> to be hacked. >> >> Wouldn't a decent, secure alternative to procmail be sieve+amavisd- >> new? > > Except it's not really possible to make amavisd-new do per-user spam > filtering. And it's even more of a performance hog. > > -- > martin; (greetings from the heart of the sun.) > \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck > > "all language designers are arrogant. goes with the territory..." > -- larry wall > > spamtraps: madduck.bogus(a)madduck.net >

3 2

[Dovecot] Moving a mail between folders and post-processing ?
by Jerry Nicholls 16 Aug '07

16 Aug '07

Hi, Within Dovecot is there a way of spotting a change to a folder and running a post-processing script on the email that was moved ? I'm specifically after a way of seeing either an email entering or leaving a folder - preferably running a different post-process script depending on the direction. Essentially if an email enters or is moved out of a 'spam' folder I'd like to mark it as 'spam'/'ham' in the Bayesian filters respectively. Is there any such mechanism, or should I start looking at how to write a plugin to achieve this ? If it's the plugin route is there any good documentation on the plugin APIs etc ? Thanks, Jerry. -- Jerry Nicholls E: Jerry.Nicholls(a)zeninternet.co.uk Core Systems & Networks Manager W: http://www.zen.co.uk/ Zen Internet Ltd. T: 0845 058 9000 F: 0845 058 9005

3 2

Re: [Dovecot] deliver assertion failure
by Jerry Yeager 16 Aug '07

16 Aug '07

On Aug 15, 2007, at 5:00 PM, dovecot-request(a)dovecot.org wrote: > > > Message: 7 > Date: Thu, 16 Aug 2007 08:59:25 +1200 > From: Jasper Bryant-Greene <jbg(a)indy.net.nz> > Subject: [Dovecot] deliver assertion failure > To: dovecot(a)dovecot.org > Message-ID: <20070815205925.GA2808(a)port1.indy.net.nz> > Content-Type: text/plain; charset=us-ascii > > Deliver from latest 1.1 HG with sieve also from latest 1.1 HG is dying > with an assertion failure when it delivers to my mailbox. Other > mailboxes on the same system are unaffected, but I can read my mailbox > fine through IMAP. > > Dovecot and sieve were compiled with -march=nocona -pipe -ggdb > > I've commented out my entire sieve script and the problem still > occurs. > > file index-mail-headers.c: line 590 (index_mail_get_raw_headers): > assertion failed: (ret != -1) > - Last output repeated twice - > > Raw backtrace: > /usr/libexec/dovecot/deliver(i_syslog_panic_handler+0x2b) > [0x493125] > -> /usr/libexec/dovecot/deliver(i_fatal+0) [0x492b17] > -> /usr/libexec/dovecot/deliver [0x44f7cf] > -> /usr/libexec/dovecot/deliver(index_mail_get_first_header+0x30) > [0x44fb9b] > -> /usr/libexec/dovecot/deliver(mail_get_first_header+0x3a) > [0x47ca3d] > -> /usr/libexec/dovecot/deliver(deliver_get_return_address+0x21) > [0x41453b] > -> /usr/lib/dovecot/lda/lib90_cmusieve_plugin.so(cmu_sieve_run > +0x14b) [0x2b3687f194a1] > -> /usr/lib/dovecot/lda/lib90_cmusieve_plugin.so [0x2b3687f169ba] > -> /usr/libexec/dovecot/deliver(main+0x8dc) [0x415c4d] > -> /lib/libc.so.6(__libc_start_main+0xe3) [0x2b3687cf6533] > -> /usr/libexec/dovecot/deliver [0x4137a9] > > Jasper > > > ------------------------------ > > _______________________________________________ > dovecot mailing list > dovecot(a)dovecot.org > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot > > End of dovecot Digest, Vol 52, Issue 56 > *************************************** I am getting the same error, the mail is still being delivered, but the system is generating an error message sent to the original sender The mail system <EMAIL ADDRESS HIDDEN>: Command died with signal 11: "/usr/local/libexec/dovecot/deliver" Reporting-MTA: dns; HIDDEN X-Postfix-Queue-ID: 7F344239D31 X-Postfix-Sender: rfc822; EMAIL ADDRESS HIDDEN Arrival-Date: Wed, 15 Aug 2007 15:53:32 -0400 (EDT) Final-Recipient: rfc822; EMAIL ADDRESS HIDDEN Action: failed Status: 5.3.0 Diagnostic-Code: x-unix; internal software error the mail logs show: to=<EMAIL ADDRESS HIDDEN>, relay=dovecot, delay=1.5, delays=0.01/0.02/0/1.5, dsn=5.3.0, status=bounced (Command died with signal 11: "/usr/local/libexec/dovecot/deliver") This did not happenr in v1.1 alpha 2, but is now appearing in 1.1 alpha 1.3 after a day or so of using the new version. Jerry Yeager

1 0

Re: [Dovecot] use of deliver from procmail advisable?
by Jerry Yeager 16 Aug '07

16 Aug '07

On Aug 15, 2007, at 5:00 PM, dovecot-request(a)dovecot.org wrote: > ------------------------------ > > Message: 3 > Date: Wed, 15 Aug 2007 18:08:58 +0200 > From: martin f krafft <madduck(a)madduck.net> > Subject: Re: [Dovecot] use of deliver from procmail advisable? > To: dovecot(a)dovecot.org > Message-ID: <20070815160858.GA22429(a)piper.oerlikon.madduck.net> > Content-Type: text/plain; charset="utf-8" > > also sprach Jerry Yeager <jerry(a)scene-naturally.dyndns.org> > [2007.08.15.1758 +0200]: >> a) Postfix milter to run ClamAv, eh something like this (for Linux >> fans) >> b) then use the regular Postfix <--> SpamAssassin <--> LDA (with >> sieve) >> setup (message routing via Postfix master.cf) so that individual >> users can >> set their own SA rules and vacation stuff. > > This is exactly how I used to have it but then the need for > a vacation autoresponse to the From: address (as opposed to > Return-Path) arose and I had to switch to procmail: > > http://dovecot.org/list/dovecot/2007-August/024766.html > > Before that, I was using spamc with --pipe-to, but always had a bad > feeling about that, since the manpage says: > > Note that there is a very slight chance mail will be lost here, > because if the fork-and-exec fails there?s no place to put the > mail message. > > and my message to SA-users on this was never answered[0]. > > 0. http://marc.info/?l=spamassassin-users&m=115185095923772&w=2 > > Now I am using procmail and at least now that failure will cause > postfix to defer a message. > > -- > martin; (greetings from the heart of the sun.) > \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck > > half a bee, philosophically, must ipso facto half not be. > but half the bee has got to be, vis-a-vis its entity. you see? > but can a bee be said to be or not to be an entire bee, > when half the bee is not a bee, due to some ancient injury? > -- monty python > > spamtraps: madduck.bogus(a)madduck.net >

1 0

[Dovecot] Errors from dovecot 1.1
by wenjie zheng 15 Aug '07

15 Aug '07

I downloaded and installed dovecot 1.1 nightly build yesterday. Here is a the errors I collected from the log file. dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=24 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 2 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=52 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 2 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=80 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension record updated without intro prefix dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=416 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 0 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=444 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension header update without intro prefix dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=528 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 0 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=556 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension header update without intro prefix dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=776 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 2 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=804 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 3 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=832 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 2 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=876 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 3 dovecot: Aug 15 14:19:29 Error: IMAP(use@doamin): Sending log messages too fast, throttling.. dovecot: Aug 15 14:19:29 Error: child 8898 (imap) killed with signal 6 dovecot: Aug 15 14:19:29 Info: IMAP(use@doamin): Disconnected: Logged out bytes=36/238 dovecot: Aug 15 14:19:30 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=920 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 2 dovecot: Aug 15 14:19:30 Error: IMAP(use@doamin): Log synchronization error at seq=7,offset=948 for /data2/mail/we/webmail.us/ji/jing/dovecot.index: Extension introduction for unknown id 2 dovecot: Aug 15 14:19:30 Error: IMAP(use@doamin): file mail-index-sync-keywords.c: line 285 (mail_index_sync_keywords): assertion failed: (rec->modify_type == MODIFY_REMOVE) dovecot: Aug 15 14:19:30 Error: IMAP(use@doamin): Raw backtrace: imap [0x80badb6] -> imap [0x80ba903] -> imap(mail_index_sync_keywords+0xee) [0x80ac3e6] -> imap(mail_index_sync_record+0x393) [0x80a0437] -> imap(mail_index_sync_map+0x208) [0x80a075c] -> imap(mail_index_map+0x89) [0x809b059] -> imap [0x8098907] -> imap [0x8098b39] -> imap(mail_index_open+0x133) [0x8098c73] -> imap(index_storage_mailbox_open+0x75) [0x8092e71] -> imap [0x806de18] -> imap [0x806dfb6] -> /usr/lib/dovecot/imap/lib10_quota_plugin.so [0x119de8] -> imap(mailbox_open+0x50) [0x80ae1ec] -> imap(_cmd_select_full+0x76) [0x805da4e] -> imap(cmd_select+0x10) [0x805dcbc] -> imap [0x805fa22] -> imap [0x805f941] -> imap [0x805fab1] -> imap(_client_input+0x56) [0x805fb6e] -> imap(io_loop_handler_run+0x100) [0x80bff0c] -> imap(io_loop_run+0x2d) [0x8066b21] -> imap(main+0x64) [0x8066518] -> /lib/tls/libc.so.6(__libc_start_main+0xda) [0x6db78a] -> imap(__fxstat64+0x71) [0x8059605] Thanks, Wenjie

2 1

[Dovecot] MANAGESIEVE patch v5 for dovecot 1.0.2
by Stephan Bosch 15 Aug '07

15 Aug '07

Hello dovecot users, I have finally updated the MANAGESIEVE patch to fix the currently known small problems with the protocol implementation. Also, I included a proxy implementation based on imap-proxy.c. This patch is designed for dovecot release 1.0.2 and it will not apply cleanly to the 1.1 versions yet. Change Log V5 ------------- - Applied patch by Uldis Pakuls to fix master_dump_settings bug - Added some compilation/installation info to the README - Moved README to source tree's root as README.managesieve - Fixed minor error handling bug in sieve_storage.c with respect to a missing root directory. - Now sieve capabilities are reported as they are specified by the implementing library and not in forced upper case. The sieve RFC now explicitly states that sieve capability identifiers are case-sensitive. This broke compatibility with SquirrelMail/Avelsieve. - Disabled ANONYMOUS login entirely until proper support is implemented. V4 claimed to do so as well, but in fact it only stopped announcing it. - Implemented managesieve-proxy. It is not so much a clean copy of imap-proxy, since the managesieve greeting is much more complex and requires parsing. Configuration is identical to imap-proxy. This seems to be a little under-documented however (http://wiki.dovecot.org/PasswordDatabase/ExtraFields) This patch still includes (yet another) instance of the CMU Sieve source, as explained in one of my previous e-mails (http://dovecot.org/list/dovecot/2006-July/015016.html) Current plans include merging src/lib-managesieve with src/lib-imap, updating the patch for dovecot-1.1 and fixing any problems you might find. The patch can be downloaded at: http://sinas.rename-it.nl/~sirius/dovecot-1.0.2-MANAGESIEVE-v5.diff.gz The README.managesieve is located in the root of the dovecot source tree after applying the patch. Have fun testing the patch. Notify me when there are problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl IRC: Freenode, #dovecot, S[r]us

5 13

[Dovecot] Dovecot deliver + sieve + postfix virtual + mysql ?
by Radek Slavicinsky 15 Aug '07

15 Aug '07

Hey there guys. I have successful postfix + mysql + sagator + clamav + spamassassin + dovecot setup (it was a really crazy month full of tweaking :]) and the only thing i miss is some per maildir specific filtering. i decided to use sieve. everything is fine except that dovecot's deliver process doesn't see the actual .dovecot.sieve file with filtering rules in the root of a maildir. description of the situation follows: from /var/log/dovecot.log: deliver(support{the_at_symbol}support.etnetera.cz): "2007-08-15 13:32:11 "Error: userdb(support{the_at_symbol}support.etnetera.cz) didn't return a home directory or sieve script location, can't find it deliver(support{the_at_symbol}support.etnetera.cz): "2007-08-15 13:32:11 "Info: msgid=<200708151330.22021.radek.slavicinsky{the_at_symbol}etnetera.cz>: saved mail to INBOX /etc/dovecot/dovecot.conf: protocols = imap imaps pop3 pop3s listen = * disable_plaintext_auth = no shutdown_clients = yes log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " ssl_cert_file = /etc/certs/sponka.etnetera.cz.crt ssl_key_file = /etc/certs/sponka.etnetera.cz.key ssl_ca_file = /etc/certs/CA.crt login_process_size = 64 login_processes_count = 32 login_max_processes_count = 256 login_max_connections = 512 mail_location = maildir:/srv/vmail/domains/%d/%n/ #mail_location = maildir:/ namespace private { separator = . prefix = INBOX. inbox = yes } namespace private { separator = . prefix = inbox = yes hidden = yes } mail_extra_groups = vmail verbose_proctitle = yes mbox_read_locks = fcntl mbox_write_locks = fcntl protocol imap { } protocol pop3 { pop3_uidl_format = UID%u-%v } protocol lda { hostname = sponka.etnetera.cz postmaster_address = postmaster{the_at_symbol}etnetera.cz auth_socket_path = /var/run/dovecot/auth-master mail_plugins = cmusieve } auth default { mechanisms = plain login digest-md5 cram-md5 passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb sql { args = /etc/dovecot/dovecot-sql.conf } user = dovecot-auth socket listen { client { path = /var/spool/postfix/private/auth mode = 0666 user = postfix group = postfix } master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail group = vmail } } } dict { } plugin { } /etc/dovecot/dovecot-sql.conf: driver = mysql default_pass_scheme = PLAIN connect = host=127.0.0.1 dbname=postfix user=postfix password=averysecretpass password_query = SELECT password FROM mailbox WHERE username = '%u' AND active = '1' user_query = SELECT maildir, 65500 AS uid, 65500 AS gid FROM mailbox WHERE username = '%u' AND active = '1' the user_query gives: mysql> SELECT maildir, 65500 AS uid, 65500 AS gid FROM mailbox WHERE username = 'radek.slavicinsky(a)etnetera.cz' AND active = '1'; +--------------------------------+-------+-------+ | maildir | uid | gid | +--------------------------------+-------+-------+ | etnetera.cz/radek.slavicinsky/ | 65500 | 65500 | +--------------------------------+-------+-------+ 1 row in set (0.00 sec) the sieve rules file # ls /srv/vmail/domains/support.etnetera.cz/support/.dovecot.sieve -lha -rw-r--r-- 1 vmail vmail 957 Aug 15 13:30 /srv/vmail/domains/support.etnetera.cz/support/.dovecot.sieve i think there should be some specific settings for the cmusieve plugin, but i didnt find much docs about that thx for any help ;] blue4 :p

3 4

[Dovecot] Dovecot on OpenBSD stalls
by Bert Koelewijn 15 Aug '07

15 Aug '07

Hello list, I'm using dovecot 1.0.2 on OpenBSD 4.1. About every month, dovecot will just stall, without any error messages. The processes are still running, the network ports are open, but the login just hangs. The following error message does occur on a very regular basis: Aug 15 00:52:37 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 01:08:08 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 01:23:38 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 01:38:39 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 01:54:09 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 02:09:10 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 02:24:40 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 02:39:41 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server Aug 15 02:55:12 mail dovecot: auth(default): LDAP: ldap_result() failed: Can't contact LDAP server (stalling from here) The LDAP directory is Active Directory and is not used as userdb, only as passdb. Thanks, Bert # 1.0.2: /etc/dovecot.conf base_dir: /var/dovecot/ ssl_cert_file: /etc/ssl/server.pem ssl_key_file: /etc/ssl/private/server.key login_dir: /var/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login login_user: _dovecot mail_location: maildir:~/Maildir mbox_write_locks: fcntl imap_client_workarounds: delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep auth default: mechanisms: plain login passdb: driver: ldap args: /etc/dovecot-ldap.conf userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: _postfix group: _postfix /etc/dovecot-ldap.conf hosts = xxx.xxx.xxx.xxx:389 #uris = #dn = #dnpass = #sasl_bind = no #sasl_mech = #sasl_realm = #sasl_authz_id = #tls = no auth_bind = yes auth_bind_userdn = cn=%u,cn=Users,dc=koelewijn,dc=bz #ldap_version = 2 base = dc=bz #deref = never #scope = subtree #user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid #user_filter = (&(objectClass=posixAccount)(uid=%u)) #pass_attrs = uid=user,userPassword=password #pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid #pass_filter = (&(objectClass=posixAccount)(uid=%u)) #default_pass_scheme = CRYPT #user_global_uid = #user_global_gid =

1 0

[Dovecot] Sieve Vacation cause deliver to die
by M1 15 Aug '07

15 Aug '07

When a user enable vacation in sieve deliver die and the mail lost. I am using 1.0.3 Thanks ------------------------------------------------------------------------------------------------------------------------------------------- in syslog postfix/pipe[16060]: 8E33017D4: to=<vivi(a)abc.com.hk>, relay=dovecot, delay=0.17, delays=0.13/0.01/0/0.03, dsn=5.3.0, status=bounced (Command died with signal 11: "/usr/libexec/dovecot/deliver") ------------------------------------------------------------------------------------------------------------------------------------------- .dovecot.sieve # Sieve Filter # Generated by Ingo (http://www.horde.org/ingo/) (August 6, 2007, 9:56 am) require ["vacation", "fileinto"]; # Whitelisted Addresses if address :all :comparator "i;ascii-casemap" :is ["From", "Sender", "Resent-From"] "juscotsushin(a)juscocityhk.com" { keep; stop; } # Vacation if allof ( not exists ["list-help", "list-unsubscribe", "list-subscribe", "list-owner", "list-post", "list-archive", "list-id"], not header :comparator "i;ascii-casemap" :is "Precedence" "list,bulk" ) { vacation :days 7 :addresses "vivi(a)abc.com.hk" :subject "Autoreply : OUT OF OFFICE" "I will not be in Hong Kong from August 6-11. While I will check my mailbox from time to time, I may respond with some delay. Regards, Vivi"; } # Blacklisted Addresses if address :all :comparator "i;ascii-casemap" :is ["From", "Sender", "Resent-From"] ["store-news(a)amazon.com", "uschinaasset(a)aol.com", "janvqvogel(a)bible.org", "info(a)bichk.net", "anmhk(a)biznetvigator.com"] { discard; stop; } # dspam if header :comparator "i;ascii-casemap" :contains "X-DSPAM-Result" "Spam" { fileinto "INBOX.Spam"; } ------------------------------------------------------------------------------------------------------------------------------------------- mail / # dovecot -n # 1.0.3: /etc/dovecot/dovecot.conf protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/dovecot/server.pem ssl_key_file: /etc/ssl/dovecot/server.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login last_valid_uid: 500 first_valid_gid: 500 last_valid_gid: 500 mail_location: maildir:/home/mail/%d/%n mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota dspam mail_plugins(imap): quota imap_quota dspam mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: maildir trash: /etc/dovecot/dovecot-trash.conf

3 2

[Dovecot] use of deliver from procmail advisable?
by martin f krafft 15 Aug '07

15 Aug '07

Hi list, I understand that dovecot's deliver does a little more than deliver: it also updates the dovecot metadata stored with each Maildir. Thus, if I use deliver as opposed to procmail's internal Maildir delivery, it seems that the IMAP server later has less work to do since the metadata is can use are up to date. Doing this, however, incurs an extra process for each mail delivered. I thus wonder whether the two balance each other out, or whether there is a strong difference. What do you think will be less resource-heavy: calling deliver for every mail received *in addition to* procmail, or letting the IMAP server update the metadata on access? -- martin; (greetings from the heart of the sun.) \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck EARTH smog | bricks AIR -- mud -- FIRE soda water | tequila WATER spamtraps: madduck.bogus(a)madduck.net

4 8

[Dovecot] namespaces and creation of folders that contain folders...
by bhayden＠umn.edu 14 Aug '07

14 Aug '07

Hi Timo and all. imap/cmd-create.c contains the following bit of code: len = strlen(full_mailbox); if (len == 0 || full_mailbox[len-1] != ns->sep) directory = FALSE; else { /* name ends with hierarchy separator - client is just informing us that it wants to create children under this mailbox. */ directory = TRUE; mailbox = t_strndup(mailbox, len-1); full_mailbox = t_strndup(full_mailbox, strlen(full_mailbox)-1); i_info("Z CREATE NAME: YYY%sYYY", mailbox); } We've been having trouble with people not being able to create a mail folder that will contain other folders. Tried it both with users using mbox (where this would be a filesystem directory) and maildir. Failed everytime because of a test which rejects folder names that end with MAILDIR_FS_SEP. Looks like the above bit of code is supposed to catch these, make a note of the fact that the client wants a directory, and strip off that trailing separator. In fact, it does this in the case of 'full_mailbox'. However, for 'mailbox' (it seems to me) the code is incorrect as its using the len-1 which is the length of 'full_mailbox', not 'mailbox'. This means that when 'mailbox' and 'full_mailbox' are the same length--that is, for the default namespace (no prefix), it happens to work. However, in any other namespace, it won't, as 'full_mailbox' will always be longer then 'mailbox' if there is a ns prefix, and hence that trailing separator won't be trimmed. (As a matter of fact, in the unlikely instance that the prefix is a single character this will trim the null character from 'mailbox', which probably isn't a good thing.) So. Am I missing something, or crazy, or is this really a bug? Thanks, -Brian

2 2

[Dovecot] about index cache
by Joe Wong 14 Aug '07

14 Aug '07

Hi Timo, Is the dovecot index cache stored only standard header fields (from, to, cc, subject, content-type... )? What happen if there is non standard header, will it be cached? - Joe May the force be with you..

3 9