- dovecot - dovecot.org

[Dovecot] dovecot.index only gets updated in Root maildir Folder
by Florian Franzeck 13 Dec '07

13 Dec '07

Debian uses dnotify by default.

3 5

Re: [Dovecot] antispam integration (dovecot antispam plugin): chroot & no way to pass `--user=' to dspam?
by Johannes Berg 13 Dec '07

13 Dec '07

> I want have dspam statistics to be per-user, not global. And I like > idea to learn filter by moving messages from/to SPAM folder, it seems to > be natural and much better, than Web UI (I don't want to use dspam Web > UI at all). > > Is here any way to resolve two problems: > > (1) Run dspam in chrooted environment > (2) pass `--user=${USER}' parameter to dspam. No idea. Passing --user is easy, there's an example in the readme file for the antispam plugin. The former I have no idea about, I guess all it needs is access to the mysql db. > Oh, and even more: I think, that it will be useful to have `socket' > backend for spam-plugin, which allows to send messages to two different > configured sockets (TCP or UNIX domain) -- one for `learn as spam' and > other for `learn as ham'. It allows to run in chrooted environment > easily, and solve problem with exec-dspam-on-every-message. Dunno. Why are you running it chrooted anyway? If you write such a backend I can add it to my tree :) johannes

1 0

[Dovecot] SYN flooding / pipe() failed: Too many open files
by Frank Elsner 13 Dec '07

13 Dec '07

Hello *, I've the following problem with dovecot 1.0.7: /var/log/messages shows | Dec 13 13:48:27 mailbox kernel: possible SYN flooding on port 143. Sending cookies. and /var/log/maillog shows (Please note: nearly the same time): | Dec 13 13:48:28 mailbox dovecot: pipe() failed: Too many open files This leads to an unresponsive mail service and requires a dovecot restart. What can be done to prevent this? BTW, this is my dovecot -n: # 1.0.7: /usr/local/dovecot/etc/dovecot.conf base_dir: /var/run/dovecot/ protocols: imap imaps pop3 pop3s ssl_cert_file: /usr/local/dovecot/etc/mailbox.pem ssl_key_file: /usr/local/dovecot/etc/mailbox.key disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): /usr/local/dovecot/libexec/dovecot/imap-login login_executable(imap): /usr/local/dovecot/libexec/dovecot/imap-login login_executable(pop3): /usr/local/dovecot/libexec/dovecot/pop3-login login_greeting: TU-Berlin.DE POP3-/IMAP Service ready login_log_format_elements: %u %r %c login_processes_count: 24 login_max_processes_count: 512 verbose_proctitle: yes mail_location: maildir:/var/Mail/Boxes/%1u/%u mailbox_idle_check_interval: 15 maildir_copy_with_hardlinks: yes maildir_copy_preserve_filename: yes mail_executable(default): /usr/local/dovecot/libexec/dovecot/imap mail_executable(imap): /usr/local/dovecot/libexec/dovecot/imap mail_executable(pop3): /usr/local/dovecot/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/dovecot/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/dovecot/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/dovecot/lib/dovecot/pop3 mail_log_prefix: %Ls(%u): imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login anonymous count: 32 passdb: driver: ldap args: /usr/local/dovecot/etc/dovecot-ldap.conf userdb: driver: static args: uid=777 gid=12 plugin: quota: maildir:ignore=Trash Kind regards ______________________________________________________________ Frank Elsner / c/o Technische Universitaet Berlin | ____________/ tubIT [BD], Sekr. E-N 50 | | Einsteinufer 17 | | Phone: +49 30 314 23897 D-10587 Berlin | | eMail: Frank.Elsner(a)TU-Berlin.DE Germany __________________| |_________________________________________________________| in dubio pro exim

3 2

[Dovecot] How to copy Spammail for learning out of Postfix and Dovecot (vmail)
by Sven Schmidt 13 Dec '07

13 Dec '07

Hi! I am using Postfix 2.3.8 and Dovecot 1.0.rc15. Virtual Domains and Mailboxes are configured. I tried to copy spammy Mail with Procmail, but Procmail only sees local mails. How to configure it to get a copy of spammy mails in a mbox or mail-dir?

3 4

[Dovecot] Dovecot Antispam Problem (Using mail sending backend)
by Justin Refice 13 Dec '07

13 Dec '07

I'm trying to setup DSPAM training using the mail-sending backend of the Antispam filter, unfortunately, things don't appear to be working. I've compiled it against the debian etch dovecot libraries, and the plugin itself builds, and doesn't report any errors when running. As a matter of fact, it doesn't report ANYTHING when running, except for the initialization code (DEBUG=syslog): imap: antispam: plugin initialising imap: antispam: "trash" is trash folder imap: antispam: "Trash" is trash folder imap: antispam: "Deleted Items" is trash folder imap: antispam: "spam" is spam folder imap: antispam: "Spam" is spam folder imap: antispam: "SPAM" is spam folder imap: antispam: "junk" is spam folder imap: antispam: "Junk" is spam folder imap: antispam: "Junk Mail" is spam folder imap: antispam: no unsure folders imap: antispam: mail backend spam address dspam-add(a)xxx.yyy imap: antispam: mail backend not-spam address dspam-fp(a)xxx.yyy imap: antispam: mail backend sendmail /usr/sbin/sendmail imap: antispam: mail backend tmpdir /tmp .... And that's it. I never hear from the plugin again, regardless of how many times I create/delete the SPAM folder, or how many times I move/copy into/out of the folder. The plugin is basically silent. I don't see the mails being sent/recieved, so I'm assuming that's it isn't doing anything at all. I can manually use the training addresses, so I know they're working. Any help would be appreciated. -Justin R.

5 9

[Dovecot] got too little data ??
by Benoit Branciard 12 Dec '07

12 Dec '07

I just switched in a hurry one of our mailbox servers from UW-imapd to Dovecot in an attempt to address performance and stability problems. This server hosts ~5500 accounts, ~2000 are daily used, ~600 simultaneous IMAP connexions in rush hours, and lots of POP ones. Some accounts may be accessed simultaneously with IMAP and POP. The system is : - Debian Sarge (x86 with amd64 kernel) - MBOX files mailboxes - indexes migrated out of filesystem quotas - Sendmail MTA - Procmail local delivery agent (not easily replaceable due to extensive use of procmailrc filters) - Both Procmail and Dovecot use [dotlock, fcntl] as mailbox locking scheme - No other processes have access to mailboxes in normal use - Dovecot 1.0.8 hand-compiled from tarball The good new is that it seems to perform better than UW-imapd. The bad new is that we get lots or errors like this in log: IMAP(username): FETCH for mailbox INBOX UID 23862 got too little data: 3186 vs 3206 Such "accidents" happen about 4-5 times a day, and does not correct themselves, even with a mailbox expunge or reopen. However manual deletion of the faulty mailbox index seems to clear the problem, until another one occurs. On such errors some mail clients (Thunderbird) retry operation in infinite loop, so log gets really cluttered. In imap-fetch-body.c it is stated : "We shouldn't really ever get here. One reason is if mail was deleted from NFS server while we were reading it. Another is some temporary disk error." Both reasons do not seem to apply in our case: mail partition is not NFS-mounted, and disks (a RAID5 array) seem healthy. Furthermore, affected accounts are not over quota. So may we have found a bug ? Or may we have a locking problem ? Or an index handling one ? I tried to disable mbox_lazy_writes without success. I will try to disable disk indexes totally, which should help since problems seem index-related; but performance may suffer. Any clues appreciated... Dovecot -n says : # 1.0.8: /usr/local/etc/dovecot.conf protocols: imap imaps pop3 pop3s listen: [::] ssl_cert_file: /usr/local/etc/ssl/private/our-certificate.pem ssl_key_file: /usr/local/etc/ssl/private/our-certificate.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login first_valid_uid: 200 first_valid_gid: 200 mail_location: mbox:~/mail:INBOX=~/mail/INBOX:INDEX=/espace/dovecot/indexes/%u fsync_disable: yes mbox_lazy_writes: no mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep delay-newmail imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep delay-newmail imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xv%08Xu pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: passdb: driver: pam args: * userdb: driver: passwd plugin: quota: fs -- Ce message a ete verifie par MailScanner pour des virus ou des polluriels et rien de suspect n'a ete trouve.

4 6

[Dovecot] Dovecot 1.08 seems very slow
by Marc Perkel 12 Dec '07

12 Dec '07

I know I'm late reporting this but 1.08 seems really slow. It seems like it is as slow as some of the early RCs were. Wish I had more info. Using Maildir format.

2 1

[Dovecot] 1.0.8 with quota 1.0.8 patch - undefined symbol: quota_set
by adam_jh＠hotmail.com 12 Dec '07

12 Dec '07

I get the following error after patching the 1.0.8 src.rpm with the quota-rewrite-1.0.8.diff [root@test dovecot]# /etc/rc.d/init.d/dovecot start Starting Dovecot Imap: Edlopen(/usr/lib/dovecot/imap/lib02_imap_quota_plugin.so) failed: /usr/lib/dovecot/imap/lib02_imap_quota_plugin.so: undefined symbol: quota_set FCouldn't load required plugins Error: imap dump-capability process returned 89 Fatal: Invalid configuration in /etc/dovecot.conf [FAILED] I have checked the new quota config changes. The src rpm is taken from http://atrpms.net/dist/el4/dovecot/. Using the same process without the patch, the rpm that's built and installed works (or at least starts). A process of elimination revealed it doesn't understand the imap_quota aspect of the quota imap_quota configuration line. Removing that it works, but I kind of need it! Can anyone help? Thanks, adam To patch it I did: Patch103: dovecot-1.0.beta2-mkcert-permissions.patch Patch105: dovecot-1.0.rc7-mkcert-paths.patch # quota patch Patch106: quota-rewrite-1.0.8.diff and %patch103 -p1 -b .mkcert-permissions %patch105 -p1 -b .mkcert-paths %patch106 -p1 -b .quota Since I'm using centos to match RHEL 4 [rpmbuild@test SPECS]$ cat /etc/redhat-release Red Hat Enterprise Linux WS release 4 (Nahant) I also added to my .rpmmacros for the rpmbuild user I have %bcond_with() %{expand:%%{?_with_%{1}:%%global with_%{1} 1}} %bcond_without() %{expand:%%{!?_without_%{1}:%%global without_%{1} 1}} %with() %{expand:%%{?with_%{1}:1}%%{!?with_%{1}:0}} %without() %{expand:%%{?without_%{1}:0}%%{!?without_%{1}:1}} then rpm -e dovecot su rpmbuild ... rpmbuild -ba postfix.spec rpm -Uvh ...dovecot.rpm [root@test dovecot]# /usr/sbin/postfix -n # 1.0.8: /etc/dovecot.conf login_dir: /var/run/dovecot-login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_processes_count: 1 first_valid_uid: 505 last_valid_uid: 505 mail_extra_groups: mail mail_location: maildir:/var/vmail/%d/%n maildir_copy_with_hardlinks: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): outlook-idle pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: user: nobody passdb: driver: sql args: /etc/dovecot-mysql.conf userdb: driver: sql args: /etc/dovecot-mysql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: mail master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: mail plugin: quota: maildir

2 3

[Dovecot] Fishing attempt locking up dovecot
by Jerry Yeager 12 Dec '07

12 Dec '07

On Dec 11, 2007, at 5:58 PM, dovecot-request(a)dovecot.org wrote: > > > Message: 10 > Date: Tue, 11 Dec 2007 15:58:16 -0700 > From: Patrick Milvich <patrick(a)milvich.com> > Subject: [Dovecot] Fishing attempt locking up dovecot > To: dovecot(a)dovecot.org > Message-ID: <8C5CE5FE-BD2F-40C4-8A36-A4CD8BD533DB(a)milvich.com> > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes > > I've mentioned this before but only heard from one other person who > has experienced this, but it's becoming a pretty serious issue. > > The situation: > A spammer sets a bot on a fishing attempt to gain email addresses, > causing numerous login processes to spawn and suck up all available > resources. > > The problem: > Obviously this can act like a dos attack, but the real issue is after > the spammer stops (by virtue of being added to our firewall blacklist, > being caught and shut down by their isp, or otherwise), dovecot > doesn't seem to relinquish the resources, causing "too many files > open" errors for normal usage. > > stuff cut out > > End of dovecot Digest, Vol 56, Issue 33 > *************************************** Will the following be of any help to you? (it is a patch for Postfix 2.4.nn) It would seem that the type of fishing expedition you mention would fall into the bit described below (lots of errors). While it will not directly solve the "out of resources" Dovecot problem, it may limit the up-front damage, followed with a CRON script running every twenty minutes or so that scans the last line of the mail log for the 'too many files open' error and upon finding it runs a version of the killall imap-login processes. ftp://postfix.mirrors.pair.com/index.html Postfix 2.4 patch (PGP signature ) to add stress-adaptive behavior to the SMTP server. When some mail flood keeps all server ports busy, this feature can be used to quickly drop connections from clients that make errors, and to reduce the time that Postfix waits for a client command. This may delay some legitimate deliveries, but it will allow you to still keep some mail flowing. After the mail flood ends, Postfix reverts to its normal behavior.

1 0

[Dovecot] dovecot deliver errors
by Andrew Garner 12 Dec '07

12 Dec '07

I think someone else mentioned this problem earlier on the list, and I've recently started seeing the following error show up: file index-transaction.c: line 54 (index_transaction_finish_rollback): assertion failed: (t->mail_ref_count == 0) Raw backtrace: /usr/lib/dovecot/deliver(i_syslog_fatal_handler+0x2b) [0x80bff0b] -> /usr/lib/dovecot/deliver [0x80bfd1a] -> /usr/lib/dovecot/deliver [0x808f783] -> /usr/lib/dovecot/deliver(index_transaction_rollback+0x1a) [0x808f5aa] -> /usr/lib/dovecot/modules/lda//lib10_quota_plugin.so [0xb7e6d013] -> /usr/lib/dovecot/deliver(deliver_save+0x202) [0x8059812] -> /usr/lib/dovecot/modules/lda//lib90_cmusieve_plugin.so [0xb7e4bb1d] -> /usr/lib/dovecot/modules/lda//lib90_cmusieve_plugin.so [0xb7e5bdbc] -> /usr/lib/dovecot/modules/lda//lib90_cmusieve_plugin.so(sieve_execute_bytecode+0x1c4) [0xb7e5c7d4] -> /usr/lib/dovecot/modules/lda//lib90_cmusieve_plugin.so(cmu_sieve_run+0x30c) [0xb7e4c7bc] -> /usr/lib/dovecot/modules/lda//lib90_cmusieve_plugin.so [0xb7e4a81f] -> /usr/lib/dovecot/deliver(main+0x10c0) [0x805a990] -> /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xc8) [0xb7e86ea8] -> /usr/lib/dovecot/deliver [0x80588e1] With beta11 I started running into this problem (using postfix's smtp-source messages): postfix/pipe[19560]: 85372294: to=<example(a)example.com>, relay=dovecot, delay=20385, delays=20384/0.01/0/0.19, dsn=5.3.0, status=bounced (Command died with signal 11: "/usr/ lib/dovecot/deliver") Program received signal SIGSEGV, Segmentation fault. 0x080d017a in str_sanitize_append (dest=0x80ee2f8, src=0x0, max_len=80) at str-sanitize.c:11 11 str-sanitize.c: No such file or directory. in str-sanitize.c (gdb) #0 0x080d017a in str_sanitize_append (dest=0x80ee2f8, src=0x0, max_len=80) at str-sanitize.c:11 p = <value optimized out> #1 0x080d02ce in str_sanitize (src=0x0, max_len=80) at str-sanitize.c:35 str = (string_t *) 0x80ee2f8 #2 0x080594f4 in deliver_log (mail=0x8107a48, fmt=0x80d349b "saved mail to %s") at deliver.c:120 args = 0xbfb087d8 "\200<DA>\016\b" str = (string_t *) 0x80ee120 #3 0x0805979f in deliver_save (namespaces=0x80f7a90, storage_r=0xbfb088e8, mailbox=0x80ded98 "INBOX", mail=0x8107a48, flags=0, keywords=0x0) at deliver.c:222 box = (struct mailbox *) 0x810be58 t = (struct mailbox_transaction_context *) 0x0 kw = (struct mail_keywords *) 0x0 error = MAIL_ERROR_NONE mailbox_name = 0x80eda80 "INBOX" ret = 0 #4 0x0805ac15 in main (argc=5, argv=Cannot access memory at address 0x4 ) at deliver.c:954 error_string = <value optimized out> error = <value optimized out> ret = <value optimized out> config_path = 0x80d34ba "/etc/dovecot/dovecot.conf" envelope_sender = 0x80ed030 "example(a)foo.net" mailbox = 0x80ded98 "INBOX" auth_socket = <value optimized out> home = <value optimized out> destaddr = 0xbfb09f9c "example(a)example.com" user = 0xbfb09f9c "example(a)example.com" error = <value optimized out> extra_fields = {arr = {buffer = 0x80ed080, element_size = 4}, v = 0x80ed080, v_modifiable = 0x80ed080} ns = (struct mail_namespace *) 0x80f7a90 raw_ns = (struct mail_namespace *) 0x80f7ab8 storage = (struct mail_storage *) 0x80f7e98 box = (struct mailbox *) 0x8102768 input = (struct istream *) 0x80f8670 t = (struct mailbox_transaction_context *) 0x8106be8 headers_ctx = (struct mailbox_header_lookup_ctx *) 0x8106508 mail = (struct mail *) 0x8107a48 process_euid = 89 namespace_pool = <value optimized out> stderr_rejection = false keep_environment = false user_auth = true i = 63 ret = <value optimized out> # dovecot -n # 1.1.beta11: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ protocols: imap imaps pop3 pop3s login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login verbose_proctitle: yes mail_uid: 89 mail_gid: 89 mail_location: maildir:~/Maildir mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): fts fts_squat mail_plugins(imap): fts fts_squat mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap/ mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap/ mail_plugin_dir(pop3): /usr/lib/dovecot/modules/ imap_client_workarounds(default): delay-newmail imap_client_workarounds(imap): delay-newmail imap_client_workarounds(pop3): pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %f pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: login plain cram-md5 digest-md5 username_format: %Lu verbose: yes debug: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth user: postfix group: postfix master: path: /var/run/dovecot/auth-master user: vmail group: vmail plugin: quota: maildir quota_rule: *:storage=102400:messages=30000 fts: squat

2 1

[Dovecot] dovecot.index only gets updated in Root maildir Folder
by Florian Franzeck 11 Dec '07

11 Dec '07

version 1.0.rc15 (Newest Version available in Debian 4.0, newer version in testing tree requires new version libc...) I got a procmail rule that moves mail from some senders to another folder, e.g. .archive.facebook/ . This works fine, the message appear in .archive.facebook/new if i "ls" that directory by shell. But dovecot.index files don't get updated, hence IDLE doesn't tell the client new mail arrived. this only happens if i click on the folder in thunderbird. like this, not even clicking on Thunderbird "Check mail" button shows me new Mail arrived. Is this a common error in this version? would there any benefit if i self compiled a newer version regarding this issue? Thanx Config: log_timestamp: %Y-%m-%d %H:%M:%S ssl_listen: * login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_extra_groups: mail mail_location: maildir:~/Maildir auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:

1 0

[Dovecot] v1.0.9 released
by Timo Sirainen 11 Dec '07

11 Dec '07

http://dovecot.org/releases/1.0/dovecot-1.0.9.tar.gz http://dovecot.org/releases/1.0/dovecot-1.0.9.tar.gz.sig + Maildir: Don't wait on dovecot-uidlist.lock when we just want to find out a new filename for the message. - mbox: v1.0.8 changes sometimes caused FETCH to fail with "got too little data", disconnecting the client. - Fixed a memory leak when FETCHing message header/body multiple times within a command (e.g. BODY[1] BODY[2]) - IMAP: Partial body fetching was still slow with mboxes

1 0

[Dovecot] Using global und per-user sieve scripts at the same time
by Matthias Kellermann 11 Dec '07

11 Dec '07

Hi list, I'm trying to get a global sieve mailfilter with per-user sieve scripts to work. At the "lda"-section in the dovecot.conf file I have: global_script_path = /usr/local/etc/sieve/spam /usr/local/etc/sieve/spam: require "fileinto"; if header :contains ["Subject"] ["***Spam***"] { fileinto "Spam"; stop; } which will work fine delivering Spam Mails in a Spam dir for every user. If a user creates his own .dovecot.sieve file the global script will not work anymore. How can I enable global and local per-user sieve scripts at once? Is this possible? Do you know any workaround so all users will have the same (Spam-)sieve filters beside there own (global templates or so)? Matthias

5 11

[Dovecot] squat also not working in beta11
by Joe Wong 10 Dec '07

10 Dec '07

Hi Timo, Just tried squat in beta11. I also get the same error in the log: dovecot: Dec 10 14:21:17 Error: IMAP(joewong99:joew.outblaze.com): Corrupted squat uidlist file /mailfs/4/22/3/joewon g99:joew.outblaze.com@joew_outblaze_com/Maildir/dovecot.index.search.uids: Broken uidlists There is not change in my configuration file. # 1.1.beta11: /usr/local/etc/dovecot.conf log_path: /var/log/dovecot-1.1.log info_log_path: /var/log/dovecot-1.1.log protocols: imap ssl_disable: yes login_dir: /usr/local/var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login login_user: nobody verbose_proctitle: yes mail_location: maildir:~/Maildir mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_drop_priv_before_exec: yes mail_plugins: fts fts_squat auth default: user: mdrop username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@: username_translation: @: verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf plugin: fts: squat --

2 5

Re: [Dovecot] New Mailbox Format Plugin Help
by Sacha Bernstein 10 Dec '07

10 Dec '07

On 12/8/07, Timo Sirainen <tss(a)iki.fi> wrote: > On Sat, 2007-12-08 at 10:11 -0500, Sacha Bernstein wrote: > > I would like to develop a new mailbox format for a very specialized > > application. Can anyone point me in the appropriate direction to get > > started writing a plugin? > > > > More detail for anyone interested: > > I need to store mail in a format that's quickly accessible and > > modifiable to an external process. This is for a very low-volume mail > > system (averaging less than 50 messages stored at any one time) that > > needs very good indexing. I need to create new mailboxes on the fly > > and perform fast searches across all the mailboxes. I intend to store > > all the messages in an SQL database. Any help getting started > > interfacing with dovecot (other than reading through _all_ the source > > code) would be greatly appreciated. > > v1.1 makes this a lot easier, so start with it instead of v1.0. You can > probably get started with my existing SQL backend code: > http://dovecot.org/list/dovecot/2007-November/026632.html > > The main idea is to implement APIs in lib-storage/mail-storage-private.h > and lib-storage/mailbox-list-private.h. Most of these calls can be > implemented by calling generic index_*() functions in > lib-storage/index/. Thanks, I'll get started with this. I'm sure I'll have more questions once I get further into it. If anyone's interested (especially any amateur radio operators that may be on this list), you can see what my plans are for the SQL storage backend for dovecot at w2sjb.org. Basically, the short version is a linux gateway to a radio-based email system for use in emergencies. Of course it has many other uses, but that's what I'm going to be using it for :-) Thanks, Sacha

1 0

Re: [Dovecot] Automatic Index Generation
by Maciej Poszywak 10 Dec '07

10 Dec '07

>> Timo Sirainen wrote: >> >>> I guess you mean logging in with POP3? There the slowdown comes from >>> getting all messages' virtual sizes. So your preindexer could probably >>> do this by running pop3 for all users. If you use only a single UID for >>> users, you could do something like: >>> >>> for user in `cat users`; do >>> export MAIL=maildir:/home/$user/Maildir >>> echo "quit" | /usr/local/libexec/dovecot/pop3 >>> done >>> >>> >>> >> vmu1@test:~$ env | grep MAIL >> MAIL=maildir:/var/mail/testuser >> vmu1@test:~$ echo "quit" | /usr/lib/dovecot/pop3 >> pop3(vmu1): Fatal: pop3_uidl_format setting is missing from config file >> >> I have pop3_uidl_format correctly defined in the config file. I've also >> tried to strace pop3, to find where it does look for it, but with no >> success. >> > > It gets it from environment: > > POP3_UIDL_FORMAT=... pop3 > > In general all settings are passed in environment using uppercased > names. > > Are you sure that all variables can be read from environment ? vmu1@mail:~$ env | grep -E "(MMAP|LOCK)" MMAP_DISABLED=yes MAIL_READ_MMAPED=no LOCK_METHOD=dotlock POP3_LOCK_SESSION=yes vmu1@mail:~$ echo "quit" | /usr/lib/dovecot/pop3 pop3(vmu1): Fatal: lock_method=dotlock and mmap_disable=no combination isn't supported. You don't _really_ want it anyway. vmu1@mail:~$ /usr/sbin/dovecot --version 1.0.rc15 -- Maciej Poszywak

2 1

[Dovecot] v1.1.beta11 released (Problems)
by Jerry Yeager 10 Dec '07

10 Dec '07

On Dec 9, 2007, at 1:48 PM, dovecot-request(a)dovecot.org wrote: > > > Message: 7 > Date: Sun, 09 Dec 2007 20:29:49 +0200 > From: Timo Sirainen <tss(a)iki.fi> > Subject: [Dovecot] v1.1.beta11 released > To: dovecot(a)dovecot.org > Message-ID: <1197224989.922.405.camel@hurina> > Content-Type: text/plain; charset="us-ascii" > > http://dovecot.org/releases/1.1/beta/dovecot-1.1.beta11.tar.gz > http://dovecot.org/releases/1.1/beta/dovecot-1.1.beta11.tar.gz.sig > > This one should be the last beta release before the first v1.1 release > candidate. > > I'll try to stay away from this list and Dovecot in general for the > next > 1,5 weeks. I've several exams coming up and I should have started > studying for them days ago already.. :) > stuff cut out > End of dovecot Digest, Vol 56, Issue 26 > *************************************** I ran into a problem moving from v 1.1 beta 10 to v 1.1 beta 11, the logs show: Dec 9 16:03:49 ns1 dovecot[331]: Dovecot v1.1.beta11 starting up Dec 9 16:03:50 ns1 dovecot[331]: imap-login: We couldn't drop root group privileges (wanted=150, gid=150, egid=0) Dec 9 16:03:50 ns1 dovecot[331]: Temporary failure in creating login processes, slowing down for now The last two messages then begin repeating until Dovecot is shut down. obviously, no one can log in. I tried changing the following # System user and group used to access mails. If you use multiple, userdb # can override these by returning uid or gid fields. You can use either numbers # or names. <doc/wiki/UserIds> #mail_uid = #mail_gid = to explicitly specifying # System user and group used to access mails. If you use multiple, userdb # can override these by returning uid or gid fields. You can use either numbers # or names. <doc/wiki/UserIds> mail_uid = Dovecot mail_gid = Dovecot As well as setting the setuid bit in login_executable = /usr/local/libexec/dovecot/imap-login to no avail. Dropping back to v 1.1 beta 10 fixed things for the moment. I did a quick scan of the example conf file and the wiki docs but could not find anything relevant. dovecot -n shows (essentially the same for both versions) # 1.1.beta11: /usr/local/etc/dovecot.conf base_dir: /var/run/dovecot/ protocols: imaps ssl_listen: *:993 ssl_cipher_list: ALL:!LOW:!SSLv2 verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login login_user: Dovecot login_greeting: Hello, Dovecot is ready. valid_chroot_dirs: /var/mail/vmail/ mail_extra_groups: mail mail_location: maildir:/var/spool/vmail/%d/%n:INDEX=/var/spool/vmail/ %d/%n:INBOX=/var/spool/vmail/%d/%n mail_debug: yes lock_method: flock auth default: mechanisms: plain login passdb: driver: passwd-file args: scheme=plain-md5 username_format=%u /etc/dovecot/passwd userdb: driver: static args: uid=virtual gid=virtual home=/var/spool/vmail/%d/%n socket: type: listen client: path: /var/spool/postfix/private/auth user: postfix group: postfix master: path: /usr/local/var/run/dovecot/auth-master user: virtual group: virtual This is running on OS X 10.5.1 ??? Jerry

2 2

[Dovecot] Dovceot 1.0.8 compile error after rewrite patch
by Lars Johansen 10 Dec '07

10 Dec '07

Added the rewrite patch for 1.0.8 without any problems, but when I tried building with: configure --prefix=/usr/local/dovecot, make gives this output: gcc -shared .libs/mail-log-plugin.o -Wl,-soname -Wl,lib20_mail_log_plugin.so -o .libs/lib20_mail_log_plugin.so ar cru .libs/lib20_mail_log_plugin.a mail-log-plugin.o ranlib .libs/lib20_mail_log_plugin.a creating lib20_mail_log_plugin.la (cd .libs && rm -f lib20_mail_log_plugin.la && ln -s ../lib20_mail_log_plugin.la lib20_mail_log_plugin.la) make[4]: Leaving directory `/usr/local/src/dovecot-1.0.8/src/plugins/mail-log' Making all in trash make[4]: Entering directory `/usr/local/src/dovecot-1.0.8/src/plugins/trash' if /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-storage -I../../../src/plugins/quota -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -MT trash-plugin.lo -MD -MP -MF ".deps/trash-plugin.Tpo" -c -o trash-plugin.lo trash-plugin.c; \ then mv -f ".deps/trash-plugin.Tpo" ".deps/trash-plugin.Plo"; else rm -f ".deps/trash-plugin.Tpo"; exit 1; fi mkdir .libs gcc -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-storage -I../../../src/plugins/quota -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -MT trash-plugin.lo -MD -MP -MF .deps/trash-plugin.Tpo -c trash-plugin.c -fPIC -DPIC -o .libs/trash-plugin.o trash-plugin.c:110: warning: 'struct quota_root_transaction_context' declared inside parameter list trash-plugin.c:110: warning: its scope is only this definition or declaration, which is probably not what you want trash-plugin.c: In function 'trash_try_clean_mails': trash-plugin.c:134: error: dereferencing pointer to incomplete type trash-plugin.c:205: error: dereferencing pointer to incomplete type trash-plugin.c:205: error: dereferencing pointer to incomplete type trash-plugin.c:206: error: dereferencing pointer to incomplete type trash-plugin.c:207: error: dereferencing pointer to incomplete type trash-plugin.c:207: error: dereferencing pointer to incomplete type trash-plugin.c:208: error: dereferencing pointer to incomplete type trash-plugin.c: At top level: trash-plugin.c:214: warning: 'struct quota_root_transaction_context' declared inside parameter list trash-plugin.c: In function 'trash_quota_root_try_alloc': trash-plugin.c:216: error: dereferencing pointer to incomplete type trash-plugin.c:220: error: 'struct quota_backend_vfuncs' has no member named 'try_alloc' trash-plugin.c:238: warning: passing argument 1 of 'trash_try_clean_mails' from incompatible pointer type trash-plugin.c: At top level: trash-plugin.c:248: warning: 'struct quota_root_transaction_context' declared inside parameter list trash-plugin.c: In function 'trash_quota_root_try_alloc_bytes': trash-plugin.c:250: error: dereferencing pointer to incomplete type trash-plugin.c:254: error: 'struct quota_backend_vfuncs' has no member named 'try_alloc_bytes' trash-plugin.c:262: warning: passing argument 1 of 'trash_try_clean_mails' from incompatible pointer type trash-plugin.c: At top level: trash-plugin.c:272: warning: 'struct quota_root_transaction_context' declared inside parameter list trash-plugin.c: In function 'trash_quota_root_test_alloc_bytes': trash-plugin.c:274: error: dereferencing pointer to incomplete type trash-plugin.c:278: error: 'struct quota_backend_vfuncs' has no member named 'test_alloc_bytes' trash-plugin.c:286: warning: passing argument 1 of 'trash_try_clean_mails' from incompatible pointer type trash-plugin.c: In function 'trash_quota_root_created': trash-plugin.c:313: error: 'struct quota_root' has no member named 'v' trash-plugin.c:314: error: 'struct quota_root' has no member named 'v' trash-plugin.c:315: error: 'struct quota_root' has no member named 'v' trash-plugin.c:316: error: 'struct quota_root' has no member named 'v' trash-plugin.c:317: error: 'struct quota_root' has no member named 'v' make[4]: *** [trash-plugin.lo] Error 1 make[4]: Leaving directory `/usr/local/src/dovecot-1.0.8/src/plugins/trash' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/usr/local/src/dovecot-1.0.8/src/plugins' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/dovecot-1.0.8/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/dovecot-1.0.8' make: *** [all] Error 2

2 1

[Dovecot] Automatic Index Generation
by Maciej Poszywak 10 Dec '07

10 Dec '07

Hello, I'm looking for an application/script/etc. which will allow me to recreate dovecot indexes by hand from shell. I'm preparing myself for migration to dovecot 1.0.rc15 (debian stable) and would like to create indexes before users start to login into new server to ease the load. My other option is to write a script which will log as each user and create indexes, however this seems a bit of painful way to do it. Anybody can point me in right direction? -- Maciej Poszywak

5 10

[Dovecot] dict: Leaked a t_pop() call in I/O handler
by Sergio Lopez 10 Dec '07

10 Dec '07

Hello, I've just installed another postfix+dovecot based mail service (I love dovecot! :-), and I found this message "dict: Leaked a t_pop() call in I/O handler (nil)" being logged each time I authenticate against it via POP3 or IMAP. Searching through the code, I've found this (this is Dovecot v1.0.8): ---- src/dict/dict-cache.c ---- void dict_cache_unref(struct dict_cache *cache, const char *uri, const char *username) { struct dict_entry *entry; t_push(); entry = hash_lookup(cache->dicts, t_strdup_printf("%s\t%s", username, uri)); i_assert(entry != NULL && entry->refcount > 0); if (--entry->refcount == 0) { hash_remove(cache->dicts, entry->user_uri); dict_deinit(&entry->dict); i_free(entry->user_uri); i_free(entry); } } ---- src/dict/dict-cache.c ---- Notice the "t_push()" call there. So calling t_pop() somewhere before the end of the function, makes dovecot work like a charm (again). P.D: I'm really surprised that nobody noticed this before. Am I the only man in the world who uses Dovecot storing quota information into a SQL Database? ;-) -- Sergio López - <sergio.lopez(a)nologin.es> Nologin Consulting S.L. - http://nologin.es

2 1

[Dovecot] Internal Error - Cannot select inbox
by Eric Beda 10 Dec '07

10 Dec '07

Hi, I am new to this list although i have been using dovecot for some time now, i've come accross this error when a user tries to log in * OK Dovecot ready. 1 login username password 1 OK Logged in. 1 select inbox 1 NO Internal error occurred. Refer to server log for more information. [2007-12-07 12:51:56] there are no errors reported in the log files related to that issue for both dovecot.log and maillog what i do receive in the log files is dovecot: Dec 07 12:55:46 Info: imap-login: Login: user=<username@domain>, method=PLAIN, rip=::ffff:127.0.0.1, lip=::ffff:127.0.0.1, secured dovecot: Dec 07 12:55:46 Info: IMAP(username@domain): Effective uid=500, gid=500 dovecot: Dec 07 12:55:46 Info: IMAP(aadu(a)uccmail.co.tz): mbox: data=/var/spool/mail/vmail/domain/username:INBOX=/var/spool/mail/vmail/domain/username/Inbox:INDEX=/var/spool/mail/vmail/domain/username:home=/var/spool/mail/vmail/domain/username/.imap/.imap dovecot: Dec 07 12:55:46 Info: IMAP(aadu(a)uccmail.co.tz): mbox: root=/var/spool/mail/vmail/domain/username, index=/var/spool/mail/vmail/domain/username, inbox=/var/spool/mail/vmail/domain/username/Inbox dovecot: Dec 07 12:55:46 Info: IMAP(username@domain): Disconnected i can't seem to pinpoint what is going wrong, some things to note i have existing users who can read email succesfully, its when i create a new user via postfixadmin, one should note that all directories are created and owned by vmail whose gid and uid is 500 and for the new users the directories and mboxes are created succesfully, but even if i copy over a working inbox of previous user to new user i still get an error so i don't think it is an mbox formatting issue another thing is that i can successfully check mail using mutt facility, even sending using mail clients (outlook express) is possible other configurations that might be helpful # dovecot --version 1.0.rc15 #cat /etc/group |grep vmail dovecot:x:97:vmail vmail:x:500:dovecot # /etc/dovecot.conf log_path: /var/log/dovecot.log protocols: pop3 pop3s imap imaps ssl_disable: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_location: mbox:/var/spool/mail/vmail/%d/%n:INBOX=/var/spool/mail/vmail/%d/%n/Inbox:INDEX=/var/spool/mail/vmail/%d/%n:home=/var/spool/mail/vmail/%d/%n/.imap/.imap mail_debug: yes maildir_copy_with_hardlinks: yes mbox_min_index_size: 200 mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 auth default: mechanisms: plain login passdb: driver: pam passdb: driver: sql args: /etc/dovecot/dovecot-mysql.conf userdb: driver: sql args: /etc/dovecot/dovecot-mysql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:

3 5

[Dovecot] Roadmap to future
by Timo Sirainen 10 Dec '07

10 Dec '07

v1.1.0 is finally getting closer, so it's time to start planning what happens after it. Below is a list of some of the larger features I'm planning on implementing. I'm not yet sure in which order, so I think it's time to ask again what features companies would be willing to pay for? Sponsoring Dovecot development gets you: - listed in Credits in www.dovecot.org - listed in AUTHORS file - you can tell me how you want to use the feature and I'll make sure that it's possible (within reasonable limits) - allows you to change the order in which features are implemented (more or less) As you can see below, some features are easier to implement when other features are implemented first. Several of them depend on v2.0 master rewrite, so I'm thinking that maybe it's time to finally finish that first. I know several companies have requested replication support, but since it kind of depends on several other features, it might be better to leave that for v2.1. But of course sponsoring development of those features that replication depends on gets you the replication support faster. :) I'm still trying to go to school (and this year I started taking CS classes as well as biotech classes..), so it might take a while for some features to get implemented. I'm anyway hoping for a v1.2 or v2.0 release before next summer. So, the list.. -------------- Implemented, but slightly buggy so not included in v1.1 (yet): - Mailbox list indexes - THREAD REFERENCES indexes - THREAD X-REFERENCES2 algorithm where threads are sorted by their latest message instead of the thread root message Could be implemented for v1.2: - Shared mailboxes and ACLs http://www.dovecot.org/list/dovecot/2007-April/021624.html - Virtual mailboxes: http://www.dovecot.org/list/dovecot/2007-May/022828.html - Depends on mailbox list indexes dbox plans, could be implemented for v1.2: - Support for single instance attachment storage - Global dbox storage so copying messages from mailbox to another involves only small metadata updates - Support some kind of hashed directories for storing message files, instead of storing everything in a single directory. - Finish support for storing multiple messages in a file v2.0 master/config rewrite -------------------------- It's mostly working already. One of the larger problems left is how to handle plugin settings. Currently Dovecot just passes everything in plugin {} section to mail processes without doing any checks. I think plugin settings should be checked just as well as the main settings. Also plugins should be able to create new sections instead of having to use plain key=value settings for everything. v2.0 currently runs a perl script on Dovecot's source tree to find settiongs from different places then then creates a single all-settings.c for config process. This obviously doesn't work for out-of-tree plugins, so there needs to be some other way for them to pass the allowed settings. There are two possibilities: a) Read them from separate files. This is a bit annoying, because it's not anymore possible to distribute simple plugin.c files and have them compiled into plugin.so files. b) Read them from the plugin itself. This would work by loading the plugin and then reading the variable containing the configuration. I'm a bit worried about security implications though, because the plugins could execute some code while they're loaded. But I guess a new unprivileged process could be created for doing this. Of course normally admins won't run any user-requested plugins anyway so this might not be a real issue anyway. :) Index file optimizations ------------------------ - Since these are not backwards compatible changes, the major version number should be increased whenever these get implemented. So I think these should be combined with v2.0 master/config rewrite. I'd rather not release v3.0 soon after v2.0 just for these.. - dovecot.index.log: Make it take less space! Get rid of the current "extension introduction" records that are large and written over and over again. Compress UID ranges using similar methods than Squat uses now. Try all kinds of other methods to reduce space usage. Write transaction boundaries, so the reader can process entire transactions at a time. This will make some things easier, such as avoiding NFS data cache flushes, replication support and also allows lockless writes using O_APPEND. - dovecot.index: Instead of keeping a single message record array, split it into two: A compressed list of message UIDs and array containing the rest of the data. This makes the file smaller and keeping UIDs separate allows better CPU cache (and maybe disk I/O) utilization for UID -> sequence lookups. At least this is the theory, I should benchmark this before committing to this design. :) There could also be a separate expunged UIDs list. UIDs existing in there would be removed from existing UIDs list, but it wouldn't affect calculating offsets to record data in the file. So while currently updating dovecot.index file after expunges requires recreating the whole file, with this method it would be possible to just write 4 bytes to the expunged UIDs list. The space reserved for expunged UIDs list would be pretty small though, so when it gets full the file would be recreated. - dovecot.index.cache: Better disk I/O and CPU cache utilization by keeping data close together in the file when it's commonly accessed toghether. For example message sizes and dates for all messages could be close to each others to optimize SORTing by message size/date. If clients use conflicting access patterns, the data could be either duplicated to optimize both patterns or just let it be slower for the other pattern. Deliver / LMTP server --------------------- Currently deliver parses dovecot.conf using its own parser. This has caused all kinds of problems. v2.0's master rewrite helps with this, because deliver can then just ask the configuration from config process the same way as other processes are. Another problem is that people who use multiple user IDs have had to make deliver setuid-root. I really hate this, but currently there's no better way. Better fix for this would be to use LMTP server instead. There could be either a separate LMTP client or deliver could support LMTP protocol as well. I'm not sure which one is better. How would LMTP server then get around the multiple user IDs issue? There are two possibilities: a) Run as root and temporarily set effective UID to the user whose mails we are currently delivering. This has the problem that if users have direct access to their mailboxes and index files, the mailbox handling code has to be bulletproof to avoid regular users gaining root privileges by doing special modifications to index files -- possibly at the same time as the mail is being delivered to exploit some race condition with mmaped index files.. b) Create a child process for each different UID and drop its privileges completely. If all users have different UIDs, this usually means forking a new process for each mail delivery, making the performance much worse than with a) method. It would anyway be possible to delay destroying the child processes in case a new message comes in for the same UID. This would probably mostly help only when UIDs are shared by multiple users, such as each domain having their own UID. I think I'll implement both and let admin choose which method to use. This same problem exists for several other things besides deliver/LMTP server. For example a) method is already implemented for expire plugin's expire-tool in v1.1. So this code needs to be somewhat generic so it won't have to be duplicated in multiple places. All of this pretty much requires the new v2.0 master rewrite. Adding these to v1.x would be ugly. Proxying -------- - These could be implemented to v1.2. - Log in normally (no proxying) if destination IP is the server itself. - Support for per-namespace proxying: namespace public { prefix = Public/ location = proxy:public.example.org } There are two choices for this: Dummy IMAP proxying or handling it with a mail storage backend. Dummy IMAP proxying would just let the remote server whose mailbox is currently selected handle most of the commands and their replies. I'm just afraid that this will become problematic in future. For example if the proxying server wants to send the client an event, such as "new message was added to this other mailbox", it would have to be sure that the remote server isn't currently in the middle of sending something. And the only way to be sure would be to parse its input, which in turn would make the proxying more complex. Then there would be problems if the servers support different IMAP extensions. And maybe some extensions will require knowing the state of multiple mailboxes, which will make handling them even more complex if the mailboxes are on different servers. Implementing this as a mail storage backend would mean that the proxying server parses all the IMAP commands internally and uses Dovecot's mail storage API to process them. The backend would then request data from the remote server when needed and cache it internally to avoid asking the same data over and over again. Besides caching, this would mean that there are no problems with extensions, since Dovecot handles the the exact same way as for other backends, requiring no additional code. To get better performance the storage backend would have to be smarter than most other Dovecot backends. For example while the current backends handle searches by reading through all the messages, the proxy backend should just send the search request to the remote server. The same for SORT. And for THREAD, which the current storage API doesn't handle.. And how would proxying server and remote server then communicate? Again two possibilities: Either a standard IMAP protocol, with possibly some optional Dovecot-extensions to improve performance, or a bandwidth-efficient protocol. The main advantage with IMAP protocol is that it could be used for proxying to other servers. Bandwidth-efficient protocol could be based on dovecot.index.log format, making it faster/easier to parse and generate. Once we're that far with proxying, we're close to having..: Replication ----------- The plans haven't changed much from: http://www.dovecot.org/list/dovecot/2007-May/022792.html The roadmap to complete replication support would be basically: 1. Implement proxying with mail storage API. 2. Implement asynchronous replication: - Replication master process collecting changes from other Dovecot processes via UNIX sockets. It would forward these changes to slaves. Note that this requires that only Dovecot is modifying mailboxes, mails must be delivered with Dovecot's deliver! - If some slaves are down, save the changes to files in case they come back up. In case they don't and we reach some specific max. log file size, keep also track of mailboxes that have changed and once the slaves do come back, make sure those mailboxes are synced. - If replication master is down and there have been changes to mailboxes.. Well, I haven't really thought about this. Maybe refuse to do any changes to mailboxes? Maybe have mail processes write to some "mailboxes changed" log file directly? Probably not that big of a deal since replication master just shouldn't be down. :) - Support for manual resync of all mailboxes, in case something was done outside Dovecot. - Replication slave processes would be reading incoming changes. I think this process should just make sure that the changes are read as fast as possible, so with sudden bursts of activity the slave would start saving the changes to a log instead of the master. - Replication mailbox writer processes would receive changes from the main slave process via UNIX sockets. They'd be responsible for actually saving the data to mailboxes. If multiple user IDs are used, this has the same problems as LMTP server. - Conflict resolution. If mailbox writer process notices that the UID already exists in the mailbox, both the old and the new message must be given new UIDs. This involves telling the replication master about the problem, so both servers can figure out together the new UIDs. I haven't thought this through yet. - Conflict resolution for message flags/keywords. The protocol would normally send flag changes incrementally, such as "add flag1, remove flag2". Besides that there could be a 8bit checksom of all the resulting flags/keywords. If the checksum doesn't match, request all the current flags. It would be of course possible to send the current flags always instead of incremental updates, but this would waste space when there are lots of keywords, and multi-master replication will work better when the updates are incremental. 3. Synchronous replication: - When saving messages, don't report success until at least one replication slave has acknowledged that it has also saved the message. Requires adding bidirectional communication with the replication master process. The message could sent to slaves in smaller parts, so if the message is large the wait at the end would basically consist of sending "finished saving UID 1234, please ack". - Synchronous expunges also? Maybe optional. The main reason for this is that it wouldn't be nice to break IMAP state by having expunged messages come back. 4. Multi-master replication: - Almost everything is already in place. The main problem here is IMAP UID allocation. They must be allocated incrementally, so each mailbox must have a globally increasing UID counter. The usage goes like: 1. Send the entire message body using a global unique ID to all servers. No need to wait for their acknowledgement. 2. Increase the global IMAP UID counter for this mailbox. 3. Send global UID => IMAP UID message to all servers. 4. Wait for one of them to reply with "OK". Because messages sent by different servers can end up being visible in different times, the servers must wait that they've seen all the previous UIDs before notifying client of a new UID. If the sending server crashed between steps 2 and 3, this never happens. So the replication processes must be able to handle that, and also handle requesting missing UIDs in case they had lost connections to other servers. - I just had the idea of global UID counters instead of global locks, so I haven't yet thought how the counters would be best implemented. It should be easier to implement than global locks though. - Non-incremental changes are somewhat problematic and could actually require global locks. The base IMAP protocol supports replacing flags, so if there is no locking and conflicting flag replacement commands are sent to two servers, which one of those commands should be effective? Timestamps would probably be enough for flag changes, but it may not be enough for some IMAP extensions (like CONDSTORE?). - If global locks are required, they would work by passing the lock from one server to another when requested. So as long as only a single server is modifying the mailbox, it would hold the global lock and there would be no need to keep requesting it for each operation. 5. Automation: - Automatically move/replicate users between servers when needed. This will probably be implemented separately long after 4. The replication processes would be ugly to implement for v1.x master, so this pretty much depends on v2.0 master rewrite.

4 6

[Dovecot] disconnect after too many errors?
by Peter Hessler 09 Dec '07

09 Dec '07

There are a couple of jerks that are tying to dictionary attack my email server, and one of the vectors is pop3/imap logins. Something I would like to do in dovecot, but can't seem to find, is the ability to disconnect after a certain number of errors. The vast majority of my users (i.e. me) don't hand-type POP3 or IMAP transactions, but when we do, we know how to spell things properly. Does dovecot have this? A simiple look shows no. -- I wouldn't mind dying -- it's that business of having to stay dead that scares the shit out of me. -- R. Geis

4 6

[Dovecot] MANAGESIEVE patch v7 for dovecot 1.0.3 and TLS
by Andreas Schneider 09 Dec '07

09 Dec '07

Hi, I allow only TLS/SSL connection to dovecot. The managesieve doesn't work with TLS but it works without it. gladiac@magrathea:~> gnutls-cli -p 2000 -s --insecure -V milliways.cynapses.org Resolving 'milliways.cynapses.org'... Connecting to '213.239.217.124:2000'... - Simple Client Mode: - Received[194]: "IMPLEMENTATION" "dovecot" "SASL" "PLAIN" "SIEVE" "fileinto reject envelope vacation imapflags notify subaddress relational comparator-i;ascii-numeric regex" "STARTTLS" OK "Dovecot ready." STARTTLS - Sent: 9 bytes - Received[33]: OK "Begin TLS negotiation now." Let me know if you need more information. Thanks, -- andreas -- http://www.cynapses.org/ - cybernetic synapses

3 4

[Dovecot] Compiling with SQL support
by Mark Smith 09 Dec '07

09 Dec '07

Hello all, I am currently having a slight problem (ok well more likely me just not doing it right), with compiling Dovecot with MySQL support. The version is 1.0.8. I have tried running configure with --with-sql and --with-sql-drivers, but this however still does not compile MySQL support, I do no wish to install MySQL on this server, as I am using a network SQL server, but I do have the compiled source available on this server. I have tried running: gcc -shared -fPIC -DHAVE_CONFIG_H -DBUILD_MYSQL \ -I../.. -I../lib -I../lib-settings -I/data/mysql/include/mysql \ driver-mysql.c -o driver_mysql.so -lmysqlclient inside the /src/lib-sql, but this just returns an error that mysqlclient cannot be found. I have tried running dovecot --build-options, but MySQL is not listed here, the operating system is FreeBSD 6.2. The documentation does not seem to be giving any hints as to why this is not working, nor do How Tos. Anyrhing remotely helpful would be appreciated. Mark Smith

2 1

[Dovecot] Is dovecot the right choice for this problem?
by Darren McLaughlin 09 Dec '07

09 Dec '07

Is dovecot the right choice for this problem? I have a shared server with verio. I have one particular client that sends and receives numerous large emails, and those emails were causing me to run out of processes on my verio server, and basically slow things down a great deal due to the shared mail queue. Verio suggested Dovecot to reduce server load and allow this one client to have less problems with sending and receiving email. Does this sound logical to anyone in the know? Thanks! Darren

3 6

[Dovecot] Are these events normal?
by Jost Krieger 08 Dec '07

08 Dec '07

We have dropped dovecot in place of UW-IMAP for about 40000 users and have solved our immediate problem: disk I/O overload, mainly from people doing POP3 and "leave mail on server". Thank you! Now i got loads of dovecot error messages in the logs and don't know if there is any significance. Here are a few examples: The most common event is: 2007-12-06 16:00:17.837654500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1547 got too little data: 5176 vs 5184 2007-12-06 16:00:18.575403500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1548 got too little data: 4749 vs 4757 2007-12-06 16:00:19.276147500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1548 got too little data: 4749 vs 4757 2007-12-06 16:00:20.371238500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1540 got too little data: 5162 vs 5170 2007-12-06 16:00:20.996483500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1540 got too little data: 5162 vs 5170 2007-12-06 16:00:21.650218500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1541 got too little data: 4086 vs 4094 2007-12-06 16:00:22.273782500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1541 got too little data: 4086 vs 4094 2007-12-06 16:00:22.920507500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1542 got too little data: 3556 vs 3562 2007-12-06 16:00:23.490787500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1542 got too little data: 3556 vs 3562 2007-12-06 16:00:24.052849500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1543 got too little data: 4098 vs 4106 2007-12-06 16:00:24.741072500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1543 got too little data: 4098 vs 4106 2007-12-06 16:00:25.448977500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1544 got too little data: 4627 vs 4635 2007-12-06 16:00:26.130131500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1544 got too little data: 4627 vs 4635 2007-12-06 16:00:26.819376500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1545 got too little data: 7582 vs 7590 2007-12-06 16:00:27.445497500 dovecot: Error: IMAP(xxxxxxxx): FETCH for mailbox UCE-TMP UID 1545 got too little data: 7582 vs 7590 Somewhat rarer are events like 2007-12-06 19:36:44.114294500 dovecot: Info: POP3(yyyyyyyy): Disconnected: Storage error during logout. top=1/1723, retr=1/4665462, del=27/61, size=17280696 and rarely we got 2007-11-24 12:10:01.086620500 dovecot: Error: IMAP(zzzzzzzz): file istream-header-filter.c: line 194 (read_header): assertion failed: (hdr == NULL) 2007-11-24 12:10:01.086718500 dovecot: Error: IMAP(zzzzzzzz): Raw backtrace: 2007-11-24 12:10:01.089552500 dovecot: Error: child 13083 (imap) killed with signal 6 2007-11-24 12:10:01.655875500 dovecot: Error: IMAP(zzzzzzzz): file istream-header-filter.c: line 194 (read_header): assertion failed: (hdr == NULL) 2007-11-24 12:10:01.656106500 dovecot: Error: IMAP(zzzzzzzz): Raw backtrace: 2007-11-24 12:10:01.659853500 dovecot: Error: child 13098 (imap) killed with signal 6 2007-11-24 12:10:02.022553500 dovecot: Error: IMAP(zzzzzzzz): file istream-header-filter.c: line 194 (read_header): assertion failed: (hdr == NULL) 2007-11-24 12:10:02.022761500 dovecot: Error: IMAP(zzzzzzzz): Raw backtrace: 2007-11-24 12:10:02.025866500 dovecot: Error: child 13121 (imap) killed with signal 6 Our dovecot is # 1.0.5: /usr/local/dovecot/etc/dovecot.conf log_path: /dev/stderr log_timestamp: protocols: pop3 imap listen(default): 134.147.64.16:143 listen(imap): 134.147.64.16:143 listen(pop3): 134.147.64.16:110 ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/dovecot/libexec/dovecot/imap-login login_executable(imap): /usr/local/dovecot/libexec/dovecot/imap-login login_executable(pop3): /usr/local/dovecot/libexec/dovecot/pop3-login login_process_per_connection: no valid_chroot_dirs: /net/sunu007/disc3/home:/net/sunu007/disc4/home max_mail_processes: 4096 first_valid_uid: 50 mail_location: mbox:~/Mail:INBOX=~/Mailbox:INDEX=~/ mail_executable(default): /usr/local/dovecot/libexec/dovecot/imap mail_executable(imap): /usr/local/dovecot/libexec/dovecot/imap mail_executable(pop3): /usr/local/dovecot/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/dovecot/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/dovecot/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/dovecot/lib/dovecot/pop3 mail_log_max_lines_per_sec: 0 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: passdb: driver: shadow args: blocking=yes userdb: driver: passwd Is there anything we should do? The crashes have stopped in the meantime, it seems. Jost -- | Helft Spam ausrotten! HTML in Mail ist unhöflich. | | Postmaster, JAPH, manchmal Wahrsager am RZ der RUB | | Wahre Worte sind nicht gefällig, gefällige Worte sind nicht wahr.| | Lao Tse, Tao Te King 81 |

3 3

[Dovecot] New Mailbox Format Plugin Help
by Sacha Bernstein 08 Dec '07

08 Dec '07

I would like to develop a new mailbox format for a very specialized application. Can anyone point me in the appropriate direction to get started writing a plugin? More detail for anyone interested: I need to store mail in a format that's quickly accessible and modifiable to an external process. This is for a very low-volume mail system (averaging less than 50 messages stored at any one time) that needs very good indexing. I need to create new mailboxes on the fly and perform fast searches across all the mailboxes. I intend to store all the messages in an SQL database. Any help getting started interfacing with dovecot (other than reading through _all_ the source code) would be greatly appreciated. Thanks, Sacha

2 1

[Dovecot] Questions about deliver log files
by frank.kintrup＠fkware.de 08 Dec '07

08 Dec '07

Hi, I upgraded from Courier to dovecot, and now I try to rebuild some of the features I used to have. One was a monthly per user log of maildrop's work. I used this code in maildroprc: DATE=`date +%m` logfile "$MAILDIR/maildrop-$DATE.log" This created a monthly log in the each user's Maildir. I find it easier to have a log per user than having to filter the log entries from a large log file for all users. I tried to recreate it with deliver by using the variables %h and %d/%n, but they are not interpreted when the log file is accessed ("Can't open log file /var/vmail/%d/%n/deliver.log: No such file or directory"). And there seems to be no way to include a date in the file name (but maybe I could use logrotate for that). Another problem: Currently deliver only logs the message id. A user would never ask for this (especially if he asks where certain mails were stored/discarded), he would ask for the sender's name/email or subject. Is the contents of the log file configurable? Frank

2 2

[Dovecot] Stale NFS file handle and duplicated messages
by Adrian Barker 08 Dec '07

08 Dec '07

We recently moved all our users from the Washington IMAP server to Dovecot. There has been a substantial improvement in performance, but some users are reporting duplicate messages, and we see 'Stale NFS file handle' errors in the syslog. Does anyone know what would cause this ? The errors relate to the index file, eg. Nov 29 00:49:47 imap-e.ucl.ac.uk dovecot: [ID 107833 mail.error] IMAP(ccaazzz): close() failed with index file /home/ccaazzz/imap-mail/.imap/mbox/dovecot.index: Stale NFS file handle Our IMAP servers (5 of them) are a mixture of Sparc Solaris 8 and Solaris 10 machines, using NFS to a central mailspool running on AIX. We are using a mixture of Dovecot 1.0.3 and 1.0.5. dovecot -n gives: # 1.0.3: /usr/local/etc/dovecot.conf base_dir: /var/run/dovecot/ listen: *:143 ssl_listen: *:993 ssl_ca_file: /usr/local/ssl/certs/sureserveredu.pem ssl_cert_file: /usr/local/ssl/certs/imap-server_ucl_ac_uk.crt ssl_key_file: /usr/local/ssl/certs/imap.key ssl_cipher_list: MEDIUM:!LOW disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable: /usr/local/libexec/dovecot/imap-login login_user: pp login_process_per_connection: no login_processes_count: 30 max_mail_processes: 1500 first_valid_uid: 200 mail_location: mbox:%h/imap-mail:INBOX=/nfs/rcs/mail-spool/mail/%u mmap_disable: yes mbox_lock_timeout: 55 mbox_dotlock_change_timeout: 50 mail_drop_priv_before_exec: yes mail_executable: /usr/local/sbin/dovecot-login.csh mail_plugins: mbox_snarf mail_plugin_dir: /usr/local/lib/dovecot namespace: type: private separator: / inbox: yes auth default: passdb: driver: pam passdb: driver: passwd userdb: driver: passwd plugin: mbox_snarf: ~/imap-mail/mbox -- Adrian Barker, Internet Technology Section Information Systems University College London, Gower Street, London WC1E 6BT External phone: +44 20 7679 5140, Fax (+44) 20 7388 5406 Internal phone: x 25140 Email: A.Barker(a)ucl.ac.uk

2 3

[Dovecot] Sponsorship of Dovecot
by Ed W 08 Dec '07

08 Dec '07

Timo Sirainen wrote: > v1.1.0 is finally getting closer, so it's time to start planning what > happens after it. Below is a list of some of the larger features I'm > planning on implementing. I'm not yet sure in which order, so I think > it's time to ask again what features companies would be willing to pay > for? > OK, I would like to sponsor Dovecot for three months for $1,000/month I am just a small business and this is basically coming straight out of my own paycheck, but we use the product internally and I like it and would like to see it continue to develop at the rate it has done. Although this is just a small token of my appreciation, I hope that other businesses who are using dovecot will step up and at least match this contribution. It's less than the price of a small Exchange Server setup and has oh so much more potential I have some minimal requests for what I would like to be developed during that period, but largely the pledge is without strings and you can develop as you wish. I will add a note on the Roadmap thread with my thoughts Thanks for writing Dovecot. Please contact me offlist and let me know how you want the sponsorship delivered (Paypal is fine but sometimes expensive for the receiver). I hope others will step up and increase this substantially Regards Ed W http://www.mailasail.com

1 0

Re: [Dovecot] imap crash with fts squat enabled.
by Joe Wong 08 Dec '07

08 Dec '07

Hi Timo, Dovecot was running on FC7 while the file server is running on FC4. - Joe May the force be with you..

1 0

[Dovecot] "pipe" plugin - is anyone interested (or using it)?
by Nicolas Boullis 08 Dec '07

08 Dec '07

Hi, A few months ago, I sent a message to this list asking if people would be interested by a "pipe" plugin (see http://dovecot.org/pipermail/dovecot/2007-May/023005.html ). I received a few answers of people who told they were intersted. A few months later, I sent a new message announcing the availability of my work (see http://dovecot.org/pipermail/dovecot/2007-August/024805.html ). Since then, I have received no feedback... So, is anyone using this plugin? Is anyone using it? Timo, on my first message, you seemed to be interested to merge such a feature (see http://dovecot.org/pipermail/dovecot/2007-May/023055.html ). Are you still interested? All comments are welcome. I think the feature (or a more generic one) would be a worthy addition for dovecot, be it for ham/spam learning, to implement an "outbox" or anything else I can't think about. All comments about my work are welcome. (Please note that I have not (yet) tried to port it to dovecot 1.1...) Cheers, Nicolas Boullis Ecole Centrale Paris

7 13

[Dovecot] Corrupted Transaction Log File: v1.0.7
by Daniel Watts 07 Dec '07

07 Dec '07

Following errors have happened quite a bit: Dec 7 14:19:26 locallog@tg3/tg3 dovecot: IMAP(peter.day(a)mydomain.com): Transaction log file /home/virtual/mydomain.com/home/peter.day/Maildir/.Trash/dovecot.index.log: marked corrupted Dec 7 14:19:26 locallog@tg3/tg3 dovecot: IMAP(peter.day(a)mydomain.com): Transaction log file /home/virtual/mydomain.com/home/peter.day/Maildir/.Trash/dovecot.index.log.2: marked corrupted Dec 7 14:19:26 locallog@tg3/tg3 dovecot: IMAP(peter.day(a)mydomain.com): Corrupted transaction log file /home/virtual/mydomain.com/home/peter.day/Maildir/.Trash/dovecot.index.log: Append with UID 50, but next_uid = 15870 Causing Peter Day not to be able to delete emails. Seems mostly to happen in the Trash I think. Shouldn't Dovecot be able to recover from this? And then have some rule that if it cannot recover, delete the index file and rebuild? When I manually remove the file it works fine. Thanks, Daniel

3 4

[Dovecot] open files
by Steve Heaven 07 Dec '07

07 Dec '07

This morning dovecot stopped allowing pop3 logins. In the logs I found the messages below Is there a config value for the max open files ? Dec 7 10:00:22 frigga dovecot: execv(/usr/local/libexec/dovecot/pop3) failed: Too many open files Dec 7 10:01:14 frigga dovecot: pipe() failed: Too many open files Dec 7 10:03:40 frigga dovecot: pipe() failed: Too many open files Dec 7 10:03:40 frigga dovecot: pipe() failed: Too many open files Dec 7 10:03:40 frigga dovecot: pipe() failed: Too many open files Dec 7 10:03:40 frigga dovecot: pipe() failed: Too many open files Dec 7 10:03:43 frigga dovecot: pipe() failed: Too many open files Dec 7 10:03:45 frigga dovecot: pipe() failed: Too many open files Dec 7 10:04:39 frigga dovecot: pipe() failed: Too many open files Dec 7 10:04:41 frigga dovecot: pipe() failed: Too many open files Dec 7 10:04:50 frigga dovecot: pipe() failed: Too many open files Dec 7 10:05:00 frigga dovecot: pipe() failed: Too many open files Dec 7 10:05:02 frigga dovecot: pipe() failed: Too many open files Dec 7 10:06:05 frigga dovecot: pipe() failed: Too many open files Dec 7 10:06:05 frigga dovecot: pipe() failed: Too many open files Dec 7 10:06:05 frigga dovecot: pipe() failed: Too many open files Dec 7 10:06:06 frigga dovecot: pipe() failed: Too many open files Dec 7 10:07:48 frigga dovecot: pipe() failed: Too many open files Dec 7 10:08:05 frigga dovecot: pipe() failed: Too many open files

3 5

[Dovecot] imap crash with fts squat enabled.
by Joe Wong 07 Dec '07

07 Dec '07

Hi Timo, Here is the backtrace for your reference. I have already applied dovecot-97702c9c4111 changes to my 1.1 beta10 code base. I am running dovecot on FC7, gcc 4.1.2, mailbox is on NFS. Whenever I did UID SEARCH body "xxxx" imap crashes. Program terminated with signal 6, Aborted. #0 0x00110402 in __kernel_vsyscall () (gdb) bt full #0 0x00110402 in __kernel_vsyscall () No symbol table info available. #1 0x00790fa0 in raise () from /lib/libc.so.6 No symbol table info available. #2 0x007928b1 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080ca029 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80ebb5c "Leaked a t_pop() call in timeout handler %p", args=0xbf8538b4 "\005\b\226\225~") at failures.c:424 backtrace = 0xa0e1168 "imap [0x80ca020] -> imap [0x80c9f2c] -> imap(io_loop_handle_timeouts+0x1b7) [0x80d0bc7] -> imap(io_loop_handler_run+0x69) [0x80d1309] -> imap(io_loop_run+0x28) [0x80d09f8] -> imap(main+0x4ac) [0x8066c"... #4 0x080c9f2c in i_panic (format=0x80ebb5c "Leaked a t_pop() call in timeout handler %p") at failures.c:197 args = 0xbf8538b4 "\005\b\226\225~" #5 0x080d0bc7 in io_loop_handle_timeouts (ioloop=0xa0e99b0, update_run_now=true) at ioloop.c:268 diff = <value optimized out> called_timeouts = (struct timeout *) 0x0 tv = {tv_sec = 8579595, tv_usec = 323093} t_id = 2 #6 0x080d1309 in io_loop_handler_run (ioloop=0xa0e99b0) at ioloop-poll.c:159 ctx = (struct ioloop_handler_context *) 0xa0e99f0 pollfd = <value optimized out> tv = {tv_sec = 0, tv_usec = 0} io = <value optimized out> t_id = <value optimized out> msecs = <value optimized out> ret = 0 call = <value optimized out> #7 0x080d09f8 in io_loop_run (ioloop=0xa0e99b0) at ioloop.c:305 No locals. #8 0x08066c2c in main (argc=Cannot access memory at address 0x5dd2 ) at main.c:293 -- And dovecot.conf: # 1.1.beta10: /usr/local/etc/dovecot.conf log_path: /var/log/dovecot-1.1.log info_log_path: /var/log/dovecot-1.1.log protocols: imap ssl_disable: yes login_dir: /usr/local/var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login login_user: nobody verbose_proctitle: yes mail_location: maildir:~/Maildir mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_drop_priv_before_exec: yes mail_plugins: fts fts_squat mail_log_max_lines_per_sec: 100 auth default: user: mdrop username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@: username_translation: @: verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf plugin: fts: squat

2 4

[Dovecot] Mails "get lost" - RHEL 5.1 - dovecot-1.0-1.2.rc15.el5
by Götz Reinicke 07 Dec '07

07 Dec '07

Hi, we do have a Voiceboxserver which forward recorded avi messages to the mailserver account of users. I'v tested it for my account 10 minutes ago and everything works. Some users don't get the messages shown up in there mailclient while the logs show a delivery of that mail. May be some mail caches are messed up? We use Thunderbird 2 and the users concerned use imap. Where may I look or what may I check? Thanks for your help! Regards Götz -- Götz Reinicke IT Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke(a)filmakademie.de Filmakademie Baden-Württemberg GmbH Mathildenstr. 20 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: Dr. Christoph Palmer, MdL, Minister a.D. Geschäftsführer: Prof. Thomas Schadt

2 2

[Dovecot] Sieve MySQL integration
by Steven Murphy 07 Dec '07

07 Dec '07

I have dovecot running with sieve performing vacation autoreply. Can sieve do sql lookups for vacation messages instead of having to do flat files in the users home directory? Thank you in advance. Steve

4 3

[Dovecot] exim/kmail vs. dovecot
by Kristian Koehntopp 07 Dec '07

07 Dec '07

I am using exim via dovecot_deliver to store messages in Maildir in my $HOME. I am using kmail to retrieve stuff. Unfortunately, something in my data crashes dovecot. I was using 1.0.rc14 from opensuse, but downloaded and installed 1.0.8 from the site. Here is the crash: Dec 5 18:05:09 h743107 dovecot: IMAP(kris): file mail-index-transaction.c: line 629 (mail_index_update_flags_range): assertion failed: (seq1 <= seq2 && seq1 > 0) Dec 5 18:05:09 h743107 dovecot: child 13896 (imap) killed with signal 6 Dec 5 18:05:09 h743107 dovecot: IMAP(kris): Raw backtrace: imap [0x8006cf7d] -> imap [0x8006ce58] -> imap(mail_index_update_flags_range+0x194) [0x8004d034] -> imap(mail_index_sync_begin+0x5c9) [0x8004fb79] -> imap(maildir_sync_index_begin+0x61) [0x80021c71] -> imap [0x8002350a] -> imap(maildir_storage_sync_init+0x56) [0x80023646] -> imap(mailbox_sync_init+0x16) [0x8005d566] -> imap(imap_sync_nonselected+0x28) [0x8001b838] -> imap(_cmd_select_full+0xd7) [0x80012c47] -> imap(cmd_select+0x25) [0x80012e25] -> imap [0x800143ea] -> imap[0x8001446c] -> imap(_client_input+0x7d) [0x80014bcd] ->imap(io_loop_handler_run+0x127) [0x80073987]-> imap(io_loop_run+0x28) [0x80072ac8] -> imap(main+0x518) [0x8001d898] -> /lib/libc.so.6(__libc_start_main+0xdc) [0xb7e8d02c] -> imap[0x8000f0c1] I took the liberty to syslog() the seq numbers before the assert and indeed h743107:/etc/dovecot # grep "imap: seq1" /var/log/messages | head -3 Dec 5 18:30:18 h743107 imap: seq1 1 seq2 1 Dec 5 18:30:18 h743107 imap: seq1 1 seq2 1 Dec 5 18:30:18 h743107 imap: seq1 1 seq2 1 h743107:/etc/dovecot # grep "imap: seq1" /var/log/messages | awk '($9 < $7) { print }' Dec 5 18:30:42 h743107 imap: seq1 28 seq2 16 Dec 5 18:32:43 h743107 imap: seq1 28 seq2 16 What exactly is broken here and how can I fix that? Kris -- Kristian =?iso-8859-15?q?K=F6hntopp?= <kris(a)xn--khntopp-90a.de>

3 3

[Dovecot] Compiling with SQL support
by Mark Smith 07 Dec '07

07 Dec '07

Hello all, I am currently having a slight problem (ok well more likely me just not doing it right), with compiling Dovecot with MySQL support. The version is 1.0.8. I have tried running configure with --with-sql and --with-sql-drivers, but this however still does not compile MySQL support, I do no wish to install MySQL on this server, as I am using a network SQL server, but I do have the compiled source available on this server. I have tried running: gcc -shared -fPIC -DHAVE_CONFIG_H -DBUILD_MYSQL \ -I../.. -I../lib -I../lib-settings -I/data/mysql/include/mysql \ driver-mysql.c -o driver_mysql.so -lmysqlclient inside the /src/lib-sql, but this just returns an error that mysqlclient cannot be found. I have tried running dovecot --build-options, but MySQL is not listed here, the operating system is FreeBSD 6.2. The documentation does not seem to be giving any hints as to why this is not working, nor do How Tos. Any help would be appreciated. Mark Smith

1 0

Re: [Dovecot] Automatic Index Generation
by Maciej Poszywak 06 Dec '07

06 Dec '07

Gabriel Millerd wrote: > On Dec 6, 2007 6:35 AM, Maciej Poszywak <jet(a)surreal.pl> wrote: > >> I'm looking for an application/script/etc. which will allow me to recreate dovecot indexes by hand from shell. I'm preparing myself for migration to dovecot 1.0.rc15 (debian stable) and would like to create indexes before users start to login into new server to ease the load. My other option is to write a script which will log as each user and create indexes, however this seems a bit of painful way to do it. Anybody can point me in right direction? >> >> > > I have a perl script that will rip through all the users using imap > and touch every folder (or every subscribed folder). if you dont have > the authentication for a proper login you can always setup a temporary > method that returns "username, 'static password', and homedir" before > going live. > > Dunno if this was what you were asking. > > That's what I'm looking for. It would be nice if you sent me that script. -- Maciej Poszywak

1 1

[Dovecot] Worse now: Cores
by Kristian Koehntopp 06 Dec '07

06 Dec '07

Worse now: I get stuff such as http://paste.debian.net/44248 in my log. What is dovecot trying to do and why? Kris -- Kristian =?iso-8859-15?q?K=F6hntopp?= <kris(a)xn--khntopp-90a.de>

4 7

[Dovecot] Dovecot + SASL + allow_nets
by Marc Cuypers 05 Dec '07

05 Dec '07

Hi, When using dovecot for authentication of an SASL (postfix) request, i cannot use the allow_nets parameter. The IP-address of the requester is not known in dovecot. I would like to allow sasl for certain users, others are not allowed to access via SASL. Some users can have access to imap and pop3 from certain IP-addresses. How could i combine this in then dovecot configuration? -- Best regards, Marc

3 9

[Dovecot] Please configure dovecot
by bhagwat swarup 05 Dec '07

05 Dec '07

hi I am new to linux and dovecot .It is my request that please write the steps to configure 1] dovecot as POP server giving access to a user and controll accesss from to different domains eg block example.com allow nh.com Thanks And Regards Bhagwat Swarup NH India Ltd

2 1

[Dovecot] Imap and quota problem
by Najib Abi Fadel 05 Dec '07

05 Dec '07

Hi everyone, i am using dovecot 1.0 as IMAP and i am facing the following problem: when a user is overquota he is unable to access his account and thus he can't delete emails ! Is there any workaround for this problem ? thanks for any help. ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs

3 4

[Dovecot] SIGSEGV login process
by Maxim 05 Dec '07

05 Dec '07

Hi all. I recently upgraded from 1.0.0 to 1.0.7 version and found that process imap-login begins randomly crash: pid 29997 (imap-login), uid 143: exited on signal 11 (core dumped) pid 32189 (imap-login), uid 143: exited on signal 11 (core dumped) pid 32121 (imap-login), uid 143: exited on signal 11 (core dumped) pid 31309 (imap-login), uid 143: exited on signal 11 (core dumped) i guess that it happens during high load on CPU (but just guess) no anything upon upgrade changed - traffic, config are the same. can anyone help me? running FreeBSD 6.2, compiled from port with mysql, lda, kqueue, pop3 support. mail# dovecot -n # 1.0.7: /usr/local/etc/dovecot.conf protocols: imap pop3 pop3s imaps listen(default): *:143 listen(imap): *:143 listen(pop3): *:110 ssl_listen(default): *:993 ssl_listen(imap): *:993 ssl_listen(pop3): *:995 ssl_cert_file: /usr/local/etc/ssl_mail/dovecot.pem ssl_key_file: /usr/local/etc/ssl_mail/dovecot.pem ssl_cipher_list: ALL:!LOW disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login verbose_proctitle: yes first_valid_uid: 26 last_valid_uid: 26 mail_extra_groups: mail mail_location: maildir:~/Maildir mmap_disable: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain cram-md5 digest-md5 apop login default_realm: proc.ru passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf mail# grep -v '^ *$#.*$\?$' /usr/local/etc/dovecot-sql.conf driver = mysql connect = host=/tmp/mysql.sock dbname=exim_users user=exim password=** default_pass_scheme = PLAIN password_query = SELECT concat(pw_name, '@', domain) as user, pw_clear_passwd as password FROM vusers WHERE pw_name = '%n' AND domain = '%d' user_query = SELECT pw_dir AS home, 26 AS uid, 26 AS gid, concat('maildir:storage=', quota*1024) AS quota FROM vusers WHERE pw_name = '%n' AND domain = '%d' backtrace from core file: mail# gdb ./work/dovecot-1.0.7/src/imap-login/imap-login /var/run/dovecot/login/tmp/imap-login.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"...(no debugging symbols found)... warning: exec file is newer than core file. Core was generated by `imap-login'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/libssl.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.4 Reading symbols from /lib/libcrypto.so.4...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypto.so.4 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x08051e2d in auth_client_request_continue () (gdb) bt #0 0x08051e2d in auth_client_request_continue () #1 0x0804b8db in client_auth_input () #2 0x080562ec in io_loop_handler_run () #3 0x0805599d in io_loop_run () #4 0x0804d963 in main () (gdb) or what anything info should i provide?

3 13

Re: [Dovecot] Variable krb5 cache location
by Timo Sirainen 04 Dec '07

04 Dec '07

On Sun, 2007-11-25 at 20:24 +0100, Roland Hopferwieser wrote: > First I want to thank you for this great software. > Second, I think it could be a good idea to give the user the chance to > change the location for the kerberos V cache, so there is no potentially > conflict with other processes. With external refreshment of the krb5 > cache, I use the cache also for authentication against the LDAP server > with the gssapi. > Please find attached a little diff file to dovecot 1.0.7. Looks ok, but I hate to add new settings to dovecot.conf nowadays since there are already too many. Also then there would be two Kerberos-related settings that simply set environment variables. I'm beginning to think that maybe something more generic is needed, such as: auth default { .. environment { krb5_ktname = .. krb5ccname = ... who_knows_what_else_in_future = .. } } I think LDAP library also can accept settings from environment. Anyone on mailing list have better ideas?

2 1

[Dovecot] LIST problem with Dovecot1.1beta10
by Dean Brooks 04 Dec '07

04 Dec '07

Hi, Certain IMAP folder problems started popping up with Squirrelmail with the 1.1beta10 version of Dovecot (and possibly earlier versions). Looked into it and I am seeing odd behavior with the LIST command. Here is how to reproduce problem, unless I'm missing something obvious: a001 login dean <password> a001 OK Logged in. a002 list "" * * LIST (\NoInferiors \UnMarked) "/" "Sent" * LIST (\NoInferiors \UnMarked) "/" "Trash" * LIST (\NoInferiors \UnMarked) "/" "Drafts" * LIST (\NoInferiors \UnMarked) "/" "INBOX" a002 OK List completed. a003 list "" Trash* * LIST (\NoInferiors \UnMarked) "/" "Trash" a003 OK List completed. a004 list "" Trash a004 OK List completed. <*** What? Shouldn't it have listed Trash instead of returning nothing?> a005 list "" Trash" <*** Unbalanced quote causes total lockup! It appears that Squirrelmail, and possibly others, are using the LIST command to test for the presence of a mailbox, but it seems impossible to get an exact match. Am I missing something here? -- Dean Brooks dean(a)iglou.com

2 5

Re: [Dovecot] imap process consuming 100% CPU (Dovecot 1.0.3)
by Jackie Hunt 04 Dec '07

04 Dec '07

> > Is this behavior cured, or do you continue to see it? > No, the behavior isn't cured. We still continue to see it with various clients. I have posted a couple of truss outputs, but so far no resolution. Sorry for the slow response. I've been "fighting other fires". Jackie > Jackie Hunt wrote: > >> On Mon, 2007-09-03 at 12:37 +0200, Robert Tomanek wrote: > >> > >>> Hi, > >>> > >>> Sunday, September 2, 2007, 22:42:37, you wrote: > >>> > >>>>> 0x0806049d in imap_sync_more (ctx=3D0x80d9770) at imap-sync.c:104 > >>>>> 104 if (ctx->seq =3D=3D 0) { > >>>>> > >>> A short follow-up on this, looks like an infinite loop to me, unless > >>> some threading magic is supposed to happen here: > >>> > >> Fixed: http://hg.dovecot.org/dovecot-1.0/rev/8e86137a04fb > >> > > > > We also are seeing imap processes consuming 100% CPU. We have > > installed Dovecot v1.0.5 and still see the problem. I have a user > > who can pretty reliably reproduce the problem. It occurs each > > morning after the imap connection has been idling all night. The > > user is running Thunderbird 2.0.0.6 (I think) on Solaris. On the > > client side, he see an hourglass that never goes away, has to shut > > down Thunderbird. > > > > Below is a log, which shows how the CPU is being used (the ps commands > > are done one right after another) and a dbx trace of the state of > > the imap process. I'm hoping that's enough info to figure out what's > > happening. > > > > Thanks much, > > Jackie > > --- > > Jackie Hunt > > ACNS Voice: (970) 663-3789 > > Colorado State University FAX: (970) 491-1958 > > Fort Collins, CO 80523 Email: jackie.hunt(a)colostate.edu > > > > > > $ ps -ef | grep -i xxxxxx | grep imap > > xxxxxx 43004 92942 74 07:15:33 - 38:54 imap > > $ ps -ef | grep -i xxxxxx | grep imap > > xxxxxx 43004 92942 81 07:15:33 - 38:58 imap > > $ ps -ef | grep -i xxxxxx | grep imap > > xxxxxx 43004 92942 120 07:15:33 - 39:01 imap > > $ dbx -a 43004 > > Waiting to attach to process 43004 ... > > Successfully attached to imap. > > warning: Directory containing imap could not be determined. > > Apply 'use' command to initialize source path. > > > > Type 'help' for help. > > reading symbolic information ... > > stopped in pread at 0xd01f06cc > > 0xd01f06cc (pread+0x30) 80410014 lwz r2,0x14(r1) > > (dbx) step > > stopped in istream-file._read at line 120 in file "" > > could not read "istream-file.c" > > (dbx) use src lib src/lib src/lib-storage/index/mbox src/lib-mail src/lib-storage/index src/lib-storage src/imap src/imap-login > > (dbx) use > > src lib src/lib src/lib-storage/index/mbox src/lib-mail src/lib-storage/index src/lib-storage src/imap src/imap-login > > (dbx) where > > istream-file._read(stream = 0x200274a8), line 120 in "istream-file.c" > > i_stream_read(stream = 0x200274cc), line 58 in "istream.c" > > istream-raw-mbox._read(stream = 0x20028ab8), line 160 in "istream-raw-mbox.c" > > i_stream_read(stream = 0x20028adc), line 58 in "istream.c" > > istream-limit._read(stream = 0x20031458), line 56 in "istream-limit.c" > > i_stream_read(stream = 0x2003147c), line 58 in "istream.c" > > istream-header-filter._read(stream = 0x2004aa98), line 234 in "istream-header-filter.c" > > i_stream_read(stream = 0x2004aabc), line 58 in "istream.c" > > i_stream_read_data(stream = 0x2004aabc, data = 0x2ff22630, size = 0x2ff22638, threshold = 1), line 250 in "istream.c" > > message_get_body_size(input = 0x2004aabc, body = 0x2002827c, has_nuls = (nil)), line 107 in "message-size.c" > > index_mail_init_stream(_mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 502 in "index-mail.c" > > mbox_mail_get_stream(_mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 206 in "mbox-mail.c" > > mail_get_stream(mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 107 in "mail.c" > > imap-fetch-body.fetch_body(ctx = 0x200205e0, mail = 0x200281a8, context = 0x20020858), line 331 in "imap-fetch-body.c" > > imap_fetch(ctx = 0x200205e0), line 291 in "imap-fetch.c" > > cmd_fetch(cmd = 0x2001e35c), line 163 in "cmd-fetch.c" > > cmd_uid(cmd = 0x2001e35c), line 19 in "cmd-uid.c" > > client_handle_input(cmd = 0x2001e35c), line 344 in "client.c" > > client_handle_input(cmd = 0x2001e35c), line 398 in "client.c" > > _client_input(context = 0x2001e318), line 441 in "client.c" > > io_loop_handler_run(ioloop = 0x2001d0e8), line 199 in "ioloop-poll.c" > > io_loop_run(ioloop = 0x2001d0e8), line 329 in "ioloop.c" > > main(argc = 1, argv = 0x2ff22b50, envp = 0x2ff22b58), line 290 in "main.c" > > (dbx) list > > 58 return _stream->read(_stream); > > 59 } > > 60 > > 61 void i_stream_skip(struct istream *stream, uoff_t count) > > 62 { > > 63 struct _istream *_stream = stream->real_stream; > > 64 size_t data_size; > > 65 > > 66 data_size = _stream->pos - _stream->skip; > > 67 if (count <= data_size) { > > (dbx) dump > > i_stream_read(stream = 0x2003147c), line 58 in "istream.c" > > _stream = 0x20031458 > > (dbx) up > > istream-header-filter._read(stream = 0x2004aa98), line 234 in "istream-header-filter.c" > > (dbx) list > > 234 if ((ret = i_stream_read(mstream->input)) == -2) { > > 235 if (stream->skip == 0) > > 236 return -2; > > 237 } > > 238 stream->istream.stream_errno = mstream->input->stream_errno; > > 239 stream->istream.eof = mstream->input->eof; > > 240 stream->buffer = i_stream_get_data(mstream->input, &pos); > > 241 } else { > > 242 ret = 0; > > 243 } > > (dbx) dump > > istream-header-filter._read(stream = 0x2004aa98), line 234 in "istream-header-filter.c" > > mstream = 0x2004aa98 > > ret = 4094 > > pos = 1 > > (dbx) up > > i_stream_read(stream = 0x2004aabc), line 58 in "istream.c" > > (dbx) list > > 58 return _stream->read(_stream); > > 59 } > > 60 > > 61 void i_stream_skip(struct istream *stream, uoff_t count) > > 62 { > > 63 struct _istream *_stream = stream->real_stream; > > 64 size_t data_size; > > 65 > > 66 data_size = _stream->pos - _stream->skip; > > 67 if (count <= data_size) { > > (dbx) dump > > i_stream_read(stream = 0x2004aabc), line 58 in "istream.c" > > _stream = 0x2004aa98 > > (dbx) up > > i_stream_read_data(stream = 0x2004aabc, data = 0x2ff22630, size = 0x2ff22638, threshold = 1), line 250 in "istream.c" > > (dbx) list > > 250 ret = i_stream_read(stream); > > 251 if (ret > 0) > > 252 read_more = TRUE; > > 253 } while (ret > 0); > > 254 > > 255 *data = i_stream_get_data(stream, size); > > 256 return ret == -2 ? -2 : > > 257 (read_more || ret == 0 ? 0 : -1); > > 258 } > > 259 > > (dbx) dump > > i_stream_read_data(stream = 0x2004aabc, data = 0x2ff22630, size = 0x2ff22638, threshold = 1), line 250 in "istream.c" > > read_more = 0 > > ret = 0 > > (dbx) up > > message_get_body_size(input = 0x2004aabc, body = 0x2002827c, has_nuls = (nil)), line 107 in "message-size.c" > > (dbx) list > > 107 } while (i_stream_read_data(input, &msg, &size, 1) > 0); > > 108 > > 109 i_stream_skip(input, 1); > > 110 body->physical_size++; > > 111 > > 112 body->virtual_size = body->physical_size + missing_cr_count; > > 113 i_assert(body->virtual_size >= body->physical_size); > > 114 } > > 115 > > 116 void message_size_add(struct message_size *dest, > > (dbx) dump > > message_get_body_size(input = 0x2004aabc, body = 0x2002827c, has_nuls = (nil)), line 107 in "message-size.c" > > last_cr = 0 > > missing_cr_count = 83389 > > msg = "dd" > > i = 4095 > > size = 1 > > (dbx) up > > index_mail_init_stream(_mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 502 in "index-mail.c" > > (dbx) list > > 502 message_get_body_size(data->stream, > > 503 &data->body_size, NULL); > > 504 data->body_size_set = TRUE; > > 505 } > > 506 } > > 507 > > 508 *body_size = data->body_size; > > 509 } > > 510 > > 511 if (data->hdr_size_set && data->body_size_set) { > > (dbx) dump > > index_mail_init_stream(_mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 502 in "index-mail.c" > > data = 0x2002820c > > mail = 0x200281a8 > > (dbx) up > > mbox_mail_get_stream(_mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 206 in "mbox-mail.c" > > (dbx) list > > 206 return index_mail_init_stream(mail, hdr_size, body_size); > > 207 } > > 208 > > 209 struct mail_vfuncs mbox_mail_vfuncs = { > > 210 index_mail_free, > > 211 index_mail_set_seq, > > 212 > > 213 index_mail_get_flags, > > 214 index_mail_get_keywords, > > 215 index_mail_get_parts, > > (dbx) dump > > mbox_mail_get_stream(_mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 206 in "mbox-mail.c" > > raw_stream = 0x2ff22780 > > mbox = 0x20024a98 > > data = 0x2002820c > > mail = 0x200281a8 > > offset = 804398928 > > (dbx) up > > mail_get_stream(mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 107 in "mail.c" > > (dbx) list > > 107 return p->v.get_stream(mail, hdr_size, body_size); > > 108 } > > 109 > > 110 const char *mail_get_special(struct mail *mail, enum mail_fetch_field field) > > 111 { > > 112 struct mail_private *p = (struct mail_private *)mail; > > 113 > > 114 return p->v.get_special(mail, field); > > 115 } > > 116 > > (dbx) dump > > mail_get_stream(mail = 0x200281a8, hdr_size = 0x2ff227a8, body_size = 0x2ff227b8), line 107 in "mail.c" > > p = 0x200281a8 > > (dbx) up > > imap-fetch-body.fetch_body(ctx = 0x200205e0, mail = 0x200281a8, context = 0x20020858), line 331 in "imap-fetch-body.c" > > (dbx) list > > 331 body->section[0] == 'H' ? NULL : &body_size); > > 332 if (ctx->cur_input == NULL) > > 333 return -1; > > 334 > > 335 i_stream_ref(ctx->cur_input); > > 336 ctx->update_partial = TRUE; > > 337 > > 338 switch (body->section[0]) { > > 339 case '\0': > > 340 /* BODY[] - fetch everything */ > > (dbx) dump > > imap-fetch-body.fetch_body(ctx = 0x200205e0, mail = 0x200281a8, context = 0x20020858), line 331 in "imap-fetch-body.c" > > fetch_size = 0x20011dd8 > > hdr_size = (physical_size = 843, virtual_size = 843, lines = 15) > > body = 0x20020858 > > body_size = (physical_size = 804398512, virtual_size = 12, lines = 0) > > (dbx) up > > imap_fetch(ctx = 0x200205e0), line 291 in "imap-fetch.c" > > (dbx) list > > 291 ret = handlers[ctx->cur_handler]. > > 292 handler(ctx, ctx->cur_mail, > > 293 handlers[ctx->cur_handler].context); > > 294 t_pop(); > > 295 > > 296 if (ret == 0) > > 297 return 0; > > 298 > > 299 if (ret < 0) { > > 300 if (ctx->cur_mail->expunged) { > > (dbx) dump > > imap_fetch(ctx = 0x200205e0), line 291 in "imap-fetch.c" > > handlers = 0x200206f8 > > count = 3 > > ret = 1 > > (dbx) up > > cmd_fetch(cmd = 0x2001e35c), line 163 in "cmd-fetch.c" > > (dbx) list > > 163 if ((ret = imap_fetch(ctx)) == 0) { > > 164 /* unfinished */ > > 165 client->command_pending = TRUE; > > 166 cmd->func = cmd_fetch_continue; > > 167 cmd->context = ctx; > > 168 return FALSE; > > 169 } > > 170 if (ret < 0) > > 171 ctx->failed = TRUE; > > 172 > > (dbx) dump > > cmd_fetch(cmd = 0x2001e35c), line 163 in "cmd-fetch.c" > > messageset = "11988" > > search_arg = 0x200205a8 > > ctx = 0x200205e0 > > args = 0x2001e590 > > ret = 536954712 > > client = 0x2001e318 > > (dbx) up > > cmd_uid(cmd = 0x2001e35c), line 19 in "cmd-uid.c" > > (dbx) list > > 19 return cmd->func(cmd); > > 20 } else { > > 21 client_send_tagline(cmd, t_strconcat( > > 22 "BAD Unknown UID command ", cmd_name, NULL)); > > 23 return TRUE; > > 24 } > > 25 } > > (dbx) dump > > cmd_uid(cmd = 0x2001e35c), line 19 in "cmd-uid.c" > > cmd_name = "fetch" > > (dbx) up > > client_handle_input(cmd = 0x2001e35c), line 344 in "client.c" > > (dbx) list > > 344 if (cmd->func(cmd) || cmd->param_error) { > > 345 /* command execution was finished */ > > 346 if (!cmd->param_error) > > 347 client->bad_counter = 0; > > 348 _client_reset_command(client); > > 349 return TRUE; > > 350 } > > 351 > > 352 /* unfinished */ > > 353 if (client->command_pending) > > (dbx) dump > > client_handle_input(cmd = 0x2001e35c), line 344 in "client.c" > > client = 0x2001e318 > > (dbx) up > > client_handle_input(cmd = 0x2001e35c), line 398 in "client.c" > > (dbx) list > > 398 client_handle_input(cmd); > > 399 } > > 400 > > 401 return TRUE; > > 402 } > > 403 > > 404 void _client_input(void *context) > > 405 { > > 406 struct client *client = context; > > 407 struct client_command_context *cmd = &client->cmd; > > (dbx) dump > > client_handle_input(cmd = 0x2001e35c), line 398 in "client.c" > > client = 0x2001e318 > > (dbx) up > > _client_input(context = 0x2001e318), line 441 in "client.c" > > (dbx) list > > 441 ret = client_handle_input(cmd); > > 442 t_pop(); > > 443 } while (ret && !client->disconnected); > > 444 o_stream_uncork(client->output); > > 445 client->handling_input = FALSE; > > 446 > > 447 if (client->command_pending) > > 448 client->input_pending = TRUE; > > 449 > > 450 if (client->output->closed) > > (dbx) dump > > _client_input(context = 0x2001e318), line 441 in "client.c" > > ret = 2 > > cmd = 0x2001e35c > > client = 0x2001e318 > > (dbx) up > > io_loop_handler_run(ioloop = 0x2001d0e8), line 199 in "ioloop-poll.c" > > (dbx) list > > 199 io->callback(io->context); > > 200 if (t_pop() != t_id) { > > 201 i_panic("Leaked a t_pop() call in " > > 202 "I/O handler %p", > > 203 (void *)io->callback); > > 204 } > > 205 } > > 206 } > > 207 } > > 208 } > > (dbx) dump > > io_loop_handler_run(ioloop = 0x2001d0e8), line 199 in "ioloop-poll.c" > > t_id = 2 > > tv = (tv_sec = 1, tv_usec = 54083) > > msecs = 1055 > > pollfd = 0x2001d148 > > ctx = 0x2001d118 > > ret = 0 > > io = 0x2001e4e8 > > call = 1 > > (dbx) up > > io_loop_run(ioloop = 0x2001d0e8), line 329 in "ioloop.c" > > (dbx) list > > 329 io_loop_handler_run(ioloop); > > 330 } > > 331 > > 332 void io_loop_stop(struct ioloop *ioloop) > > 333 { > > 334 ioloop->running = FALSE; > > 335 } > > 336 > > 337 void io_loop_set_running(struct ioloop *ioloop) > > 338 { > > (dbx) dump > > io_loop_run(ioloop = 0x2001d0e8), line 329 in "ioloop.c" > > (dbx) up > > main(argc = 1, argv = 0x2ff22b50, envp = 0x2ff22b58), line 290 in "main.c" > > (dbx) list > > 290 io_loop_run(ioloop); > > 291 main_deinit(); > > 292 > > 293 io_loop_destroy(&ioloop); > > 294 lib_deinit(); > > 295 > > 296 return 0; > > 297 } > > (dbx) dump > > main(argc = 1, argv = 0x2ff22b50, envp = 0x2ff22b58), line 290 in "main.c" > > (dbx) up > > __start() at 0x100001d8 > > (dbx) list > > no source file > > (dbx) dump > > __start() at 0x100001d8 > > (dbx) cont > > > > execution completed > > (dbx) quit > > $ > > > > -- > ==== > Stewart Dean, Unix System Admin, Henderson Computer Resources > Center of Bard College, Annandale-on-Hudson, New York 12504 > sdean(a)bard.edu voice: 845-758-7475, fax: 845-758-7035 > Jackie Hunt ACNS Voice: (970) 663-3789 Colorado State University FAX: (970) 491-1958 Fort Collins, CO 80523 Email: jackie.hunt(a)colostate.edu

4 9

[Dovecot] 1.1beta 10 hang
by Joe Wong 04 Dec '07

04 Dec '07

Hi Timo, I have installed 1.1 beta 10 today with squat enabled. When I do uid search body "something", the imap will sometimes segmentation fault, or sometime hang. Here is what i found in the log when imap did not hang.. dovecot: Dec 04 17:11:07 Panic: IMAP(joewong99:joew.outblaze.com): file squat-uidlist.c: line 178 (uidlist_write_array): assertion failed: ((uid & ~UID_LIST_MASK_RANGE) >= prev) dovecot: Dec 04 17:11:07 Error: IMAP(joewong99:joew.outblaze.com): Raw backtrace: imap [0x80ca1f0] -> imap [0x80ca0fc] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so [0x11b9a9] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so [0x11ba7a] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so [0x11bb6b] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so(squat_uidlist_build_add_uid+0x2e2) [0x11c0d2] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so [0x11a9cb] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so [0x11acf0] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so(squat_trie_build_more+0x16a) [0x11ae6a] -> /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so [0x117b54] -> /usr/local/lib/dovecot/imap/lib20_fts_plugin.so(fts_backend_build_more+0x2e) [0x1125de] -> /usr/local/lib/dovecot/imap/lib20_fts_plugin.so [0x113f3d] -> imap [0x805d196] -> imap [0x805d518] -> imap(io_loop_handle_timeouts+0x10b) [0x80d0ceb] -> imap(io_loop_handler_run+0x69) [0x80d1509] -> imap(io_loop_run+0x28) [0x80d0bc8] -> imap(main+0x4cc) [0x8066c4c] -> dovecot: Dec 04 17:11:07 Error: IMAP(joewong99:joew.outblaze.com): /lib/libc.so.6(__libc_start_main+0xe0) [0x77df70] -> imap [0x8059571] dovecot: Dec 04 17:11:07 Error: child 32571 (imap) killed with signal 6 --

2 1