From ben at morrow.me.uk Fri Feb 1 00:35:42 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 31 Jan 2013 22:35:42 +0000 Subject: [Dovecot] Userdb passwd and 'nologin' users Message-ID: <20130131223541.GC25874@anubis.morrow.me.uk> I am running Dovecot with system users (userdb passwd), but some of those users don't have shell accounts on the IMAP server so their shell on that machine is set to /usr/sbin/nologin. Currently I am using maildirs and this is not a problem, but I am in the process of switching to dbox which means I will need a cronjob running 'doveadm purge -A'. During testing I found that those users with a 'nologin' shell are not included in the list returned by the userdb iterator, and that the iterator doesn't honour the first/last_valid_uid settings. This inconsistency seems undesirable, so the attached patch - makes lookup perform the same checks as iteration, - makes the 'nologin' check configurable, - adds a new optional check that the user owns their home directory. The last check was the one performed by qmail, and seems to me to be a more reliable 'is this a real user' check than a nologin shell. If this patch is applied, the release notes for the next release should probably mention that system users with a 'nologin' shell will no longer be allowed to log in to IMAP until the 'auth_check_nologin' setting is changed from true to false. Also, there seem to be two first/last_valid_uid settings: first_valid_uid itself, which is honoured by the storage subsystem, and auth_first_valid_uid, which is honoured by the 'passwd' userdb. Is this intentional? Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: userdb-passwd-nologin.patch Type: text/x-diff Size: 4203 bytes Desc: not available URL: From ben at indietorrent.org Fri Feb 1 00:49:14 2013 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 31 Jan 2013 17:49:14 -0500 Subject: [Dovecot] Reviewing end-user ham/spam submissions before feeding them to sa-learn via Dovecot Antispam plug-in In-Reply-To: References: <50F6D7A6.50307@indietorrent.org> Message-ID: <510AF4EA.5050401@indietorrent.org> On 1/17/2013 4:31 AM, Steffen Kaiser wrote: > On Wed, 16 Jan 2013, Ben Johnson wrote: > >> Currently, I'm using the Dovecot Antispam plug-in with great >> success. Everything works as expected. > >> However, I would like to change the plug-in's behavior such that >> it simply sends a copy of a message that is moved from Inbox -> >> Junk (or Junk -> Inbox) to an administrator, instead of calling >> sa-learn (I'm using SpamAssassin) automatically. > >> Basically, I would like to be able to review messages that >> end-users submit for training before they are actually fed to >> sa-learn. > > Hmm, if you use Maildir: > > plugin { antispam_backend = spool2dir antispam_allow_append_to_spam > = true [snip] antispam_spool2dir_spam = > /path/to/admin/Maildir/.TrainingReview.spam/new/%%020lu-%%05lu > antispam_spool2dir_notspam = > /path/to/admin/Maildir/.TrainingReview.not_spam/new/%%020lu-%%05lu > } > > any of your mail users need write permission those directories, the > admin needs read permissions for the spooled files, you need some > method to pass the reviewed messages to sa-learn. > > IMHO, for 3rd step: I would either flag messages to be learned or > move them into another folder. And a cron job feeds them to > sa-learn, so they do not get into the message queue again and are > probably re-filterred or modified before sa-learn picks them up. > > If you do not have Maildir to drop the files to or get problems, > because the messages appear in "new" before they are spooled on > disk and hence they may get indexed wrongly, because their content > changes (which is forbidden in IMAP), you could spool them into > > antispam_spool2dir_spam = /tmp/spamspool/spam/%%020lu-%%05lu-%u > antispam_spool2dir_notspam = /tmp/spamspool/ham/%%020lu-%%05lu-%u > -or- antispam_spool2dir_spam = /tmp/spamspool/%%020lu-%%05lu-%u-S > antispam_spool2dir_notspam = /tmp/spamspool/%%020lu-%%05lu-%u-H > > and have another cron job or inotify-wrapper move the messages to > the reviewer's mailbox. In this case, you can make use of the "%u" > component, which is expanded by the username, who spooled the > message. So you could ignore messages of some users and/or ... . > > Kind regards, > > -- Steffen Kaiser > Steffen, thank you very much for the thorough reply. I apologize for taking so long to digest it and respond. I am using the Maildir format indeed. >> any of your mail users need write permission those directories, >> the admin needs read permissions for the spooled files, By "mail users", do you mean, e.g., the "vmail" user account (I'm on Debian/Ubuntu)? My understanding is that the "vmail" user account handles all IMAP transactions; if this is true, then are you saying that the only requisite to your suggestions is that the "vmail" user has read/write access to the following two directories? /path/to/admin/Maildir/.TrainingReview.spam/new/ and /path/to/admin/Maildir/.TrainingReview.not_spam/new/ >> you need some method to pass the reviewed messages to sa-learn. In the past, I have simply sorted the messages into "Ham" and "Spam" sub-folders of the admin's training Inbox, and called sa-learn, with the appropriate --ham/--spam switch on each, using a cron job. It sounds as though this is what you are suggesting, and I can continue with this approach. I went ahead and tried to reconfigure Dovecot's Antispam plug-in to use the spool2dir backend, but I'm receiving a less-than-helpful message from the plug-in when I try to move a message from Inbox to Junk or vice versa: "CANNOT: antispam plugin not configured". Initially, I was using paths to the admin's mailbox, as demonstrated in your initial response, but I simplified the spool location in order to eliminate permission problems. Please note that I am using Dovecot 1.2.9 in Ubuntu 10.04 LTS. By extension, I am using the Antispam plug-in for Dovecot 1 (not 2), the manpage for which is at http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html . So, the configuration option names and expected values differ slightly from those in your example. These are the only configuration directives that I am using: ------------------------------------- plugin { # Unrelated plugin options here [...] antispam_spam_pattern_ignorecase = SPAM;JUNK # Is this next directive actually required for this approach? antispam_allow_append_to_spam = yes antispam_spool2dir_spam = /tmp/spamspool/%%020lu-%u-%%05lus antispam_spool2dir_notspam = /tmp/spamspool/%%020lu-%u-%%05luh } ------------------------------------- The spool directory that is specified exists and, for testing purposes, has 0777 permissions: # ls -lah /tmp | grep "spamspool" drwxrwxrwx 2 root root 4.0K Jan 31 14:22 spamspool Based on the above-cited manpage, these are be the only options that are required (perhaps with the exception of antispam_allow_append_to_spam). What have I overlooked here? Thanks again for all your help! -Ben From tss at iki.fi Fri Feb 1 01:03:38 2013 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Feb 2013 01:03:38 +0200 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: <20130131223541.GC25874@anubis.morrow.me.uk> References: <20130131223541.GC25874@anubis.morrow.me.uk> Message-ID: On 1.2.2013, at 0.35, Ben Morrow wrote: > I am running Dovecot with system users (userdb passwd), but some of > those users don't have shell accounts on the IMAP server so their shell > on that machine is set to /usr/sbin/nologin. Currently I am using > maildirs and this is not a problem, but I am in the process of switching > to dbox which means I will need a cronjob running 'doveadm purge -A'. > > During testing I found that those users with a 'nologin' shell are not > included in the list returned by the userdb iterator, and that the > iterator doesn't honour the first/last_valid_uid settings. This > inconsistency seems undesirable, so the attached patch > > - makes lookup perform the same checks as iteration, Hmmh. You could also just have them aliased to other users, so this wouldn't be necessary.. > - makes the 'nologin' check configurable, > - adds a new optional check that the user owns their home directory. These settings are passwd-specific, so they would have to something like: userdb { driver = passwd args = check-nologin=n check-home=y } > The last check was the one performed by qmail, and seems to me to be a > more reliable 'is this a real user' check than a nologin shell. It also performs disk I/O, slowing down the lookup. > If this patch is applied, the release notes for the next release should > probably mention that system users with a 'nologin' shell will no longer > be allowed to log in to IMAP until the 'auth_check_nologin' setting is > changed from true to false. The default will in any case be the same as it is now. > Also, there seem to be two first/last_valid_uid settings: > first_valid_uid itself, which is honoured by the storage subsystem, and > auth_first_valid_uid, which is honoured by the 'passwd' userdb. Is this > intentional? Nope, that's a bug. Fixed that in v2.2: http://hg.dovecot.org/dovecot-2.2/rev/18661d1d6ed0 From daniel.parthey at informatik.tu-chemnitz.de Fri Feb 1 05:17:42 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 1 Feb 2013 04:17:42 +0100 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: <20130131223541.GC25874@anubis.morrow.me.uk> References: <20130131223541.GC25874@anubis.morrow.me.uk> Message-ID: <20130201031742.GA18862@daniel.localdomain> Hi Ben, Ben Morrow wrote: > + if (set->check_nologin) { > + /* skip entries that don't have a valid shell. > + they're again probably not real users. */ > + if (strcmp(pw->pw_shell, "/bin/false") == 0 || > + strcmp(pw->pw_shell, "/sbin/nologin") == 0 || > + strcmp(pw->pw_shell, "/usr/sbin/nologin") == 0) > + return FALSE; > + } Valid shells are defined in /etc/shells and "locked" users, I would strongly discourage from hardcoding a list of no-login shells here. Users locked with "passwd -l" can also be detected by a ! at the beginning of the password hash. Regards Daniel -- https://plus.google.com/103021802792276734820 From dovecot at lists.wgwh.ch Fri Feb 1 14:28:25 2013 From: dovecot at lists.wgwh.ch (Oli Schacher) Date: Fri, 1 Feb 2013 13:28:25 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <1359663448.3230.0.camel@hurina> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> Message-ID: <20130201132825.5363b1f4@boscos> On Thu, 31 Jan 2013 22:17:28 +0200 Timo Sirainen wrote: > On Thu, 2013-01-31 at 21:51 +0200, Timo Sirainen wrote: > > On 31.1.2013, at 19.41, Oli Schacher wrote: > > > > >>> Jan 31 17:13:11 doco1 dovecot: dsync-local(user1): Error: > > >>> Mailbox INBOX: Remote didn't send mail > > >>> GUID=33dabe0f11980a51200c0000960042f4 (UID=104) > > > > I guess there's some bug that causes this to happen in some > > situations.. But the reason for mail duplication should be fixed > > by: http://hg.dovecot.org/dovecot-2.2/rev/138f1c76c0ec > > > > Except that shouldn't have been necessary. doveadm-server returns > > success before it has finished running dsync. Not sure why, need to > > debug it further. > > Fixed with a bit of a kludge: > http://hg.dovecot.org/dovecot-2.2/rev/e9e6a95cea21 > > I can confirm that it has become significantly harder to produce errors with the latest patches. There still seems to be a problem when changes to both mailboxes at the same time are involved, however, today I didn't have time to test "scientifically", i just updated to latest hg and clicked around, so this report probably won't be of much use to you,sorry. I'll try to make reproducible tests again next week. I'll post the errors from my clicking session anyway, maybe it helps you figuring out what went wrong even without knowing how to reproduce. At least the "Operation not permitted" error below when killing the dsync process sounds unintended? Logoutput is from changeset 78bdcb6642c7 running on both servers. Server 1: Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=211) Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=205) Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=208) Feb 1 07:12:54 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=205: 7a30ff22af5b0b510f0c0000960042f4 != 8230ff22af5b0b510f0c0000960042f4 Feb 1 07:12:54 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=228) [...] Feb 1 07:12:55 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Importing mailbox INBOX failed Feb 1 07:12:56 doco1 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.62) failed: EOF Feb 1 07:12:56 doco1 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.62) failed: Broken pipe Feb 1 07:12:56 doco1 dovecot: dsync-local(user1): Error: Remote command returned error 75 [...] Feb 1 07:12:57 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=291: 7b30ff22af5b0b510f0c0000960042f4 != 8d30ff22af5b0b510f0c0000960042f4 Feb 1 07:12:57 doco1 dovecot: doveadm: Error: dsync-remote(user1): Panic: file dsync-mailbox-import.c: line 1112 (dsync_mailbox_import_change): assertion failed: (change->type == DSYNC_MAIL_CHANGE_TYPE_SAVE) Feb 1 07:12:57 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x5d4ea) [0x7f19cf5954ea] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f19cf5955d2] -> /usr/lib64/dovecot/libdovecot.so.0(+0x1f6ca) [0x7f19cf5576ca] -> /usr/bin/doveadm(dsync_mailbox_import_change+0x501) [0x42c881] -> /usr/bin/doveadm(dsync_brain_sync_mails+0x3a2) [0x4290c2] -> /usr/bin/doveadm(dsync_brain_run+0x169) [0x425e29] -> /usr/bin/doveadm() [0x426380] -> /usr/bin/doveadm() [0x434aa0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f19cf5a4076] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f19cf5a5107] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f19cf5a4018] -> /usr/bin/doveadm() [0x424134] -> /usr/bin/doveadm() [0x40fe4f] -> /usr/bin/doveadm() [0x41067d] -> /usr/bin/doveadm(doveadm_mail_try_run+0x141) [0x410ba1] -> /usr/bin/doveadm(main+0x3f1) [0x417bc1] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f19cf1c3cdd] -> /usr/bin/doveadm() [0x40f839] Feb 1 07:12:57 doco1 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.62) failed: EOF Server 2: Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=205: 7a30ff22af5b0b510f0c0000960042f4 != 8230ff22af5b0b510f0c0000960042f4 Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=228) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=234) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=238) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7c30ff22af5b0b510f0c0000960042f4 (UID=256) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7c30ff22af5b0b510f0c0000960042f4 (UID=235) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7c30ff22af5b0b510f0c0000960042f4 (UID=239) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=255) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=226) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=237) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Importing mailbox INBOX failed Feb 1 07:13:24 doco2 dovecot: dsync-local(user1): Error: Remote command process isn't dying, killing it Feb 1 07:13:24 doco2 dovecot: dsync-local(user1): Error: kill() failed: Operation not permitted -- message transmitted on 100% recycled electrons From skdovecot at smail.inf.fh-brs.de Fri Feb 1 15:45:50 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 1 Feb 2013 14:45:50 +0100 (CET) Subject: [Dovecot] Reviewing end-user ham/spam submissions before feeding them to sa-learn via Dovecot Antispam plug-in In-Reply-To: <510AF4EA.5050401@indietorrent.org> References: <50F6D7A6.50307@indietorrent.org> <510AF4EA.5050401@indietorrent.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 31 Jan 2013, Ben Johnson wrote: > On 1/17/2013 4:31 AM, Steffen Kaiser wrote: >> On Wed, 16 Jan 2013, Ben Johnson wrote: >> > I am using the Maildir format indeed. > >>> any of your mail users need write permission those directories, >>> the admin needs read permissions for the spooled files, > > By "mail users", do you mean, e.g., the "vmail" user account (I'm on > Debian/Ubuntu)? My understanding is that the "vmail" user account Yes, I mean that Unix account, Dovecot accesses the fils with. In a setup with virtual users "vmail" makes sense. > handles all IMAP transactions; if this is true, then are you saying > that the only requisite to your suggestions is that the "vmail" user > has read/write access to the following two directories? > > /path/to/admin/Maildir/.TrainingReview.spam/new/ > > and > > /path/to/admin/Maildir/.TrainingReview.not_spam/new/ Yes. >>> you need some method to pass the reviewed messages to sa-learn. > > In the past, I have simply sorted the messages into "Ham" and "Spam" > sub-folders of the admin's training Inbox, and called sa-learn, with > the appropriate --ham/--spam switch on each, using a cron job. It > sounds as though this is what you are suggesting, and I can continue That's what I mean. > with this approach. > > I went ahead and tried to reconfigure Dovecot's Antispam plug-in to > use the spool2dir backend, but I'm receiving a less-than-helpful > message from the plug-in when I try to move a message from Inbox to > Junk or vice versa: "CANNOT: antispam plugin not configured". > > Please note that I am using Dovecot 1.2.9 in Ubuntu 10.04 LTS. By Oh, I have no experience with Dovecot v1.2; in v1.0 you have to compile one particular backend into antispam-plugin. Maybe, distributors have another, non-Dovecot way to select between the backends. > extension, I am using the Antispam plug-in for Dovecot 1 (not 2), the > manpage for which is at > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > . So, the configuration option names and expected values differ > slightly from those in your example. The man-page contains: "INSTALLATION First copy the ?defconfig? file to ?.config? and edit it as necessary. You need to have the dovecot headers installed and possibly other things depending on the backend you choose. Then, assuming you have configured the INSTALLDIR correctly, simply run ?make install?. " - -and- "BACKENDS The plugin supports multiple backends, there are currently two working backends included in the distribution: " This suggests that my guess is true - although I don't understand the "there are currently two backends included" part. ====== So you could re-compile another antispam-plugin with spool2dir backend, or - - you are using these settings, right? #===================== # mail sending plugin # # Because of the way this plugin works, you can also use it # to train via an arbitrary program that receives the message # on standard input, in that case you can use the config # options antispam_mail_spam and antispam_mail_notspam for # the argument that distinguishes between ham and spam. # For example: # antispam_mail_sendmail = /path/to/mailtrain # antispam_mail_sendmail_args = --for;%u # antispam_mail_spam = --spam # antispam_mail_notspam = --ham change antispam_mail_sendmail into a script, that drops the message into the correct mail folder, e.g.: #!/bin/bash mode= for opt; do if test "x$opt" == x--ham; then mode=HAM break elif test "x$opt" == x--ham; then mode=SPAM break fi done if test -n "$mode"; then # options from http://wiki1.dovecot.org/LDA /path/to/dovecot-deliver -d spamadmin -m Training.$mode fi This sends the message bypassing a MTA to the spamadmin user. Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUQvHDl3r2wJMiz2NAQJEeAf/XmxSzh+cPqviAax/ucThVaYfygrARz6G qXRLbea/8fnhlRfO2seL75tElDmRsirVXGPu5awpf0WUEzFD96HWrmcrKMMRPfyE uFylqzVB2dnmc+KOLolGb08hKRooMOaTPQt1Y9eVDGAplQM8PoNu+K3QE+rqkCbf OJiL2pxJrEbiTOxzVhFSgUY/VdJVYLUBY4BpC5iZp7nNvXvub4scNlcd7OX0T1Kj nlPnjpw2eNWX+UBCmjbfuVQKVFLBIFQFL9gnxZMphCMzjjYYgPaGHpSBlO00C+aM ddiR46SrcjJIP4pXZsJyf5xw5aOCIUk2PXGr4aQFj409rcVJaK3CsQ== =NgZB -----END PGP SIGNATURE----- From micah at riseup.net Fri Feb 1 18:09:28 2013 From: micah at riseup.net (micah anderson) Date: Fri, 01 Feb 2013 11:09:28 -0500 Subject: [Dovecot] dsync timeout? In-Reply-To: <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> References: <87libacola.fsf@minnow.riseup.net> <3B764B67-F536-4BB3-BF2A-4777F4B6B3F0@iki.fi> <87halyck05.fsf@minnow.riseup.net> <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> Message-ID: <87sj5gaudj.fsf@minnow.riseup.net> Sean Kamath writes: > On Jan 30, 2013, at 3:46 PM, micah anderson wrote: >> Seems that only the above process was still around and no other dsync >> processes. I have three machines that all have this happening it seems. >> >> I wonder if there is a ssh configuration option I could set to make >> these die off. > > If the ssh process isn't sending anything, and just waiting for read()s, and keepalives are turned off, the SSH session might never know the remote side is long gone. . . > > If any data were transmitted, it would discover the remote side is turned off. > > See man ssh_config and the option TCPKeepAlive. > > BTW: Since it's not on the command line, it's likely in /etc/ssh_config or /etc/ssh/ssh_config. Or ~/.ssh/config. In /etc/ssh/sshd_config on the server I'm sending to, TCPKeepAlive yes is set. The default on this system, according to the man page, seems to be to have TCPKeepAlive set. Perhaps I should set ServerAliveInterval? micah From ben at indietorrent.org Fri Feb 1 18:18:12 2013 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 01 Feb 2013 11:18:12 -0500 Subject: [Dovecot] Reviewing end-user ham/spam submissions before feeding them to sa-learn via Dovecot Antispam plug-in In-Reply-To: References: <50F6D7A6.50307@indietorrent.org> <510AF4EA.5050401@indietorrent.org> Message-ID: <510BEAC4.8080207@indietorrent.org> On 2/1/2013 8:45 AM, Steffen Kaiser wrote: > On Thu, 31 Jan 2013, Ben Johnson wrote: > >> On 1/17/2013 4:31 AM, Steffen Kaiser wrote: >>> On Wed, 16 Jan 2013, Ben Johnson wrote: >>> >> I am using the Maildir format indeed. > >>>> any of your mail users need write permission those >>>> directories, the admin needs read permissions for the spooled >>>> files, > >> By "mail users", do you mean, e.g., the "vmail" user account (I'm >> on Debian/Ubuntu)? My understanding is that the "vmail" user >> account > > Yes, I mean that Unix account, Dovecot accesses the fils with. In > a setup with virtual users "vmail" makes sense. > >> handles all IMAP transactions; if this is true, then are you >> saying that the only requisite to your suggestions is that the >> "vmail" user has read/write access to the following two >> directories? > >> /path/to/admin/Maildir/.TrainingReview.spam/new/ > >> and > >> /path/to/admin/Maildir/.TrainingReview.not_spam/new/ > > Yes. > >>>> you need some method to pass the reviewed messages to >>>> sa-learn. > >> In the past, I have simply sorted the messages into "Ham" and >> "Spam" sub-folders of the admin's training Inbox, and called >> sa-learn, with the appropriate --ham/--spam switch on each, using >> a cron job. It sounds as though this is what you are suggesting, >> and I can continue > > That's what I mean. > >> with this approach. > >> I went ahead and tried to reconfigure Dovecot's Antispam plug-in >> to use the spool2dir backend, but I'm receiving a >> less-than-helpful message from the plug-in when I try to move a >> message from Inbox to Junk or vice versa: "CANNOT: antispam >> plugin not configured". > >> Please note that I am using Dovecot 1.2.9 in Ubuntu 10.04 LTS. >> By > > Oh, I have no experience with Dovecot v1.2; in v1.0 you have to > compile one particular backend into antispam-plugin. Maybe, > distributors have another, non-Dovecot way to select between the > backends. > >> extension, I am using the Antispam plug-in for Dovecot 1 (not 2), >> the manpage for which is at >> http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html >> >> . So, the configuration option names and expected values differ >> slightly from those in your example. > > The man-page contains: > > "INSTALLATION > > First copy the ?defconfig? file to ?.config? and edit it as > necessary. You need to have the dovecot headers installed and > possibly other things depending on the backend you choose. Then, > assuming you have configured the INSTALLDIR correctly, simply run > ?make install?. " > > -and- > > "BACKENDS > > The plugin supports multiple backends, there are currently two > working backends included in the distribution: " > > This suggests that my guess is true - although I don't understand > the "there are currently two backends included" part. > > ====== > > So you could re-compile another antispam-plugin with spool2dir > backend, or - - you are using these settings, right? > > #===================== # mail sending plugin # # Because of the way > this plugin works, you can also use it # to train via an arbitrary > program that receives the message # on standard input, in that case > you can use the config # options antispam_mail_spam and > antispam_mail_notspam for # the argument that distinguishes between > ham and spam. # For example: # antispam_mail_sendmail = > /path/to/mailtrain # antispam_mail_sendmail_args = --for;%u # > antispam_mail_spam = --spam # antispam_mail_notspam = --ham > > change antispam_mail_sendmail into a script, that drops the > message into the correct mail folder, e.g.: > > #!/bin/bash > > mode= for opt; do if test "x$opt" == x--ham; then mode=HAM break > elif test "x$opt" == x--ham; then mode=SPAM break fi done > > if test -n "$mode"; then # options from > http://wiki1.dovecot.org/LDA /path/to/dovecot-deliver -d spamadmin > -m Training.$mode fi > > This sends the message bypassing a MTA to the spamadmin user. > > Kind regards, > > -- Steffen Kaiser Steffen, It seems you're correct in that the spool2dir back-end isn't included with version 1 of the plug-in. I like the idea of using a pipe script better than changing the back-end. Thank you for providing a solid example; it works beautifully with a couple of small changes. Here's the final script: --------------------------------------- #!/bin/bash mode= for opt; do if test "x$*" == "x--ham"; then mode=HAM break elif test "x$*" == "x--spam"; then mode=SPAM break fi done if test -n "$mode"; then # options from http://wiki1.dovecot.org/LDA /usr/lib/dovecot/deliver -d user at example.com -m Training.$mode fi exit 0 --------------------------------------- For anyone who is curious, here are the Antispam plug-in options: --------------------------------------- # For Dovecot < 2.0. antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_mail_tmpdir = /tmp antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham --------------------------------------- Looks like we're in business! Thanks again for sharing your valuable insights; your help is greatly appreciated. -Ben From frank at mclien.de Fri Feb 1 18:22:52 2013 From: frank at mclien.de (Frank Lienhard) Date: Fri, 01 Feb 2013 17:22:52 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? Message-ID: <510BEBDC.6020802@mclien.de> I'm about to replace my old server. Both servers are on my internal network. The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. I set up the users on both systems identically (same gid uid). My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server. I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. So which could be the most painless way to solve that? From h.reindl at thelounge.net Fri Feb 1 18:25:35 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 01 Feb 2013 17:25:35 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BEBDC.6020802@mclien.de> References: <510BEBDC.6020802@mclien.de> Message-ID: <510BEC7F.20409@thelounge.net> Am 01.02.2013 17:22, schrieb Frank Lienhard: > I'm about to replace my old server. Both servers are on my internal network. > The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. > I set up the users on both systems identically (same gid uid). > My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the > maildir format, but this results in ~30% doublicated mails on the new server. > I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. > So which could be the most painless way to solve that? imapsync is version agnostic and if you have a database with the user-logins it should be easy to geenrate a shellscript i did migration of around 200000 messages and some hundret users from Apple OSX EIMS to dbmail/dovecot in 2009 this way -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From janfrode at tanso.net Fri Feb 1 19:00:07 2013 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 1 Feb 2013 18:00:07 +0100 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? Message-ID: We upgraded our two dovecot directors from v2.0.14 to dovecot-ee 2.1.10.3 this week, and after that mail seems to be flowing a lot slower than before. The backend mailstores are untouched, on v2.0.14 still. After the upgrade we've been hitting process_limit for lmtp a lot, and we're struggeling with large queues in the incoming mailservers that are using LMTP virtual transport towards our two directors. I seem to remember 2.1 should have a new lmtp-proxying code. Is there anything in this that maybe needs to be tuned that's different from v2.0 ? I'm a bit scheptical to just increasing the process_limit for LMTP proxying, as I doubt running many hundreds of simultaneous deliveries should work that much better against the backend storage.. ###### doveconf -n ########## # 2.1.10.3: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.32.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) default_client_limit = 4000 director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 disable_plaintext_auth = no listen = * lmtp_proxy = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = proxy=y nopassword=y driver = static } protocols = imap pop3 lmtp sieve service anvil { client_limit = 6247 } service auth { client_limit = 8292 unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 4096 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { address = * port = 24 } process_limit = 100 } service managesieve-login { executable = managesieve-login director inet_listener sieve { address = * port = 4190 } process_limit = 50 } service pop3-login { executable = pop3-login director process_limit = 2048 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } ssl_cert = References: <510BEBDC.6020802@mclien.de> <510BEC7F.20409@thelounge.net> Message-ID: <510BF90D.4000709@mclien.de> Reindl Harald wrote: > Am 01.02.2013 17:22, schrieb Frank Lienhard: > >> I'm about to replace my old server. Both servers are on my internal network. >> The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. >> I set up the users on both systems identically (same gid uid). >> My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the >> maildir format, but this results in ~30% doublicated mails on the new server. >> I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. >> So which could be the most painless way to solve that? >> > > imapsync is version agnostic and if you have a database > with the user-logins it should be easy to geenrate a shellscript > > i did migration of around 200000 messages and some hundret users > from Apple OSX EIMS to dbmail/dovecot in 2009 this way > > sadly, this is not availble from the debian repos. I searched a bit around: syncmaildir, imapsync: only available in in Lenny OR Wheezy, but needed on both servers isync, mailsync: mentioned to hold two maildirs in sync with a more or less difficult to configure setup which leaves me one option (from programms I found); offlineimap: mentioned to backup mails from a remote/ISP mailserver in a local directory of a user. Wonder if that works , when the local dir is one of an imap server....... From h.reindl at thelounge.net Fri Feb 1 19:22:12 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 01 Feb 2013 18:22:12 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BF90D.4000709@mclien.de> References: <510BEBDC.6020802@mclien.de> <510BEC7F.20409@thelounge.net> <510BF90D.4000709@mclien.de> Message-ID: <510BF9C4.4070907@thelounge.net> Am 01.02.2013 18:19, schrieb Frank Lienhard: > Reindl Harald wrote: >> Am 01.02.2013 17:22, schrieb Frank Lienhard: >> >>> I'm about to replace my old server. Both servers are on my internal network. >>> The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. >>> I set up the users on both systems identically (same gid uid). >>> My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the >>> maildir format, but this results in ~30% doublicated mails on the new server. >>> I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't >>> work. >>> So which could be the most painless way to solve that? >>> >> >> imapsync is version agnostic and if you have a database >> with the user-logins it should be easy to geenrate a shellscript >> >> i did migration of around 200000 messages and some hundret users >> from Apple OSX EIMS to dbmail/dovecot in 2009 this way >> >> > sadly, this is not availble from the debian repos. > I searched a bit around: > syncmaildir, imapsync: only available in in Lenny OR Wheezy, but needed on both servers needed on both servers is simply not true i have even a template in my admin-backend to generate imapsync scripts out of a db imapsync --host1 [source] --user1 [user1] --password1 [pwd1] --authmech1 LOGIN --host2 [target] --user2 [user2] --password2 [pwd2] --authmech2 LOGIN --skipsize -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From ben at morrow.me.uk Fri Feb 1 20:31:27 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 1 Feb 2013 18:31:27 +0000 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: References: <20130131223541.GC25874@anubis.morrow.me.uk> Message-ID: <20130201183126.GD25874@anubis.morrow.me.uk> At 1AM +0200 on 1/02/13 you (Timo Sirainen) wrote: > On 1.2.2013, at 0.35, Ben Morrow wrote: > > > I am running Dovecot with system users (userdb passwd), but some of > > those users don't have shell accounts on the IMAP server so their shell > > on that machine is set to /usr/sbin/nologin. Currently I am using > > maildirs and this is not a problem, but I am in the process of switching > > to dbox which means I will need a cronjob running 'doveadm purge -A'. > > > > During testing I found that those users with a 'nologin' shell are not > > included in the list returned by the userdb iterator, and that the > > iterator doesn't honour the first/last_valid_uid settings. This > > inconsistency seems undesirable, so the attached patch > > > > - makes lookup perform the same checks as iteration, > > Hmmh. You could also just have them aliased to other users, so this > wouldn't be necessary.. I don't understand what you mean. Alias them where? > > - makes the 'nologin' check configurable, > > - adds a new optional check that the user owns their home directory. > > These settings are passwd-specific, so they would have to something like: > > userdb { > driver = passwd > args = check-nologin=n check-home=y > } OK. New patch attached. > > The last check was the one performed by qmail, and seems to me to be a > > more reliable 'is this a real user' check than a nologin shell. > > It also performs disk I/O, slowing down the lookup. Hmm. OK, I've left that part out: my real users are segregated by UID anyway, so all I really care about is getting rid of the nologin check. (I would be perfectly happy if the check were just removed altogether.) > > If this patch is applied, the release notes for the next release should > > probably mention that system users with a 'nologin' shell will no longer > > be allowed to log in to IMAP until the 'auth_check_nologin' setting is > > changed from true to false. > > The default will in any case be the same as it is now. Well, yes; but authentication will now check for a nologin shell by default, which it didn't before, so the visible behaviour will have changed. > > Also, there seem to be two first/last_valid_uid settings: > > first_valid_uid itself, which is honoured by the storage subsystem, and > > auth_first_valid_uid, which is honoured by the 'passwd' userdb. Is this > > intentional? > > Nope, that's a bug. Fixed that in v2.2: > http://hg.dovecot.org/dovecot-2.2/rev/18661d1d6ed0 Cool. Will that be backported to 2.1 at some point? Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: userdb-passwd-nologin2.patch Type: text/x-diff Size: 4142 bytes Desc: not available URL: From ben at morrow.me.uk Fri Feb 1 20:37:46 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 1 Feb 2013 18:37:46 +0000 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: <20130201031742.GA18862@daniel.localdomain> References: <20130131223541.GC25874@anubis.morrow.me.uk> <20130201031742.GA18862@daniel.localdomain> Message-ID: <20130201183746.GE25874@anubis.morrow.me.uk> At 4AM +0100 on 1/02/13 you (Daniel Parthey) wrote: > Hi Ben, > > Ben Morrow wrote: > > + if (set->check_nologin) { > > + /* skip entries that don't have a valid shell. > > + they're again probably not real users. */ > > + if (strcmp(pw->pw_shell, "/bin/false") == 0 || > > + strcmp(pw->pw_shell, "/sbin/nologin") == 0 || > > + strcmp(pw->pw_shell, "/usr/sbin/nologin") == 0) > > + return FALSE; > > + } > > Valid shells are defined in /etc/shells and "locked" users, I would > strongly discourage from hardcoding a list of no-login shells here. That list isn't mine, my patch just moves that code from one part of the file to another and makes it conditional. Personally I don't think checking the shell is sensible at all, which is why I'm trying to make it optional. > Users locked with "passwd -l" can also be detected by a ! at > the beginning of the password hash. That is system-specific, and in any case you have to be root (and on non-BSD systems you have to make a shadow password call) to see the password field. The userdb shouldn't be doing that: locked users will already be prevented from logging in with a password because the password won't match. (Of course, this doesn't cover e.g. Kerberos logins, though I would usually lock a Kerberos user by telling the KDC not to issue tickets rather than by locking the passwd account.) Ben From trashcan at odo.in-berlin.de Fri Feb 1 22:39:09 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 1 Feb 2013 20:39:09 +0000 (UTC) Subject: [Dovecot] cmsg cancel Message-ID: ignore Article cancelled by slrn 1.0.1 From trashcan at odo.in-berlin.de Fri Feb 1 22:53:55 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 1 Feb 2013 21:53:55 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <20130201132825.5363b1f4@boscos> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> Message-ID: [Sorry Oli for my previous mail to your address, only. Resent here] Oli Schacher wrote: > There still seems to be a problem when changes to both mailboxes at > the same time are involved I can confirm your observation, although triggered by a different test scenario, similar to the one I did use with 2.1 replicator before (http://www.dovecot.org/list/dovecot/2012-March/064354.html). This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test" at both servers "mx1" and "mx2", and replicator uses ssh for remote access. Both servers run a recent postfix, use lmtp for local delivery, and "test" is a virtual user. Test script to produce local testmails of equal size at mx1: | #!/bin/csh | set INDEX = 101 | set endINDEX = 200 | while ( $INDEX <= $endINDEX ) | echo $INDEX | echo "test" | mail -s $INDEX test at mx1 | if ( $INDEX % 1000 == 0 ) then | sleep 1 | endif | @ INDEX = $INDEX + 1 |end |exit 0 Test script to produce testmails of equal size at mx2: | #!/bin/csh | set INDEX = 1101 | set endINDEX = 1200 | while ( $INDEX <= $endINDEX ) | echo $INDEX | echo "test" | mail -s $INDEX test at mx2 | if ( $INDEX % 1000 == 0 ) then | sleep 1 | endif | @ INDEX = $INDEX + 1 |end |exit 0 All tests are run with vanilla mailboxes, after restarting dovecot, and without imap connections by MUA: 1) Simultaneous mailbomb approach: run both scripts simultaneously, and you'll end up with numerous duplicates in mailboxes "test". Very often you'll find multiples. 2) Mailbomb approach: run one script at one server only, and all mails will become perfectly well synchronised. 3) Mofify both scripts to "( $INDEX % 1 == 0 )" to add a second waiting between every mail injection, and run them simultaneously at both servers, and you'll end up with significantly less duplicates and no more multiples. > Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=211) > Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Importing mailbox INBOX failed > Feb 1 07:13:24 doco2 dovecot: dsync-local(user1): Error: Remote command process isn't dying, killing it I do see those error messages as well, and in addition numerous of those: | dovecot: dsync-local(test): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=7153: 82c5df0a4ffa0b5141e300006a0d5a02 != 29cc9f284ffa0b5141c2000036abecbd | doveadm: Error: dsync-remote(test): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=7153: 82c5df0a4ffa0b5141e300006a0d5a02 != 29cc9f284ffa0b5141c2000036abecbd | dovecot: lmtp(49752, test): Error: Corrupted index cache file /.../test/mailboxes/INBOX/dbox-Mails/dovecot.index.cache: File too small | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: mdbox /.../test/mailboxes/INBOX/dbox-Mails: Storage keeps breaking | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Disconnected: Logged out in=425 out=1100 JFYI, and regards, Michael From tss at iki.fi Sat Feb 2 00:00:56 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 2 Feb 2013 00:00:56 +0200 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: Message-ID: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> On 1.2.2013, at 19.00, Jan-Frode Myklebust wrote: > We upgraded our two dovecot directors from v2.0.14 to dovecot-ee > 2.1.10.3 this week, and after that mail seems to be flowing a lot > slower than before. The backend mailstores are untouched, on v2.0.14 > still. After the upgrade we've been hitting process_limit for lmtp a > lot, and we're struggeling with large queues in the incoming > mailservers that are using LMTP virtual transport towards our two > directors. > > I seem to remember 2.1 should have a new lmtp-proxying code. Is there > anything in this that maybe needs to be tuned that's different from > v2.0 ? I'm a bit scheptical to just increasing the process_limit for > LMTP proxying, as I doubt running many hundreds of simultaneous > deliveries should work that much better against the backend storage.. Hmm. The main difference is that v2.1 writes temporary files to mail_temp_dir. If that's in tmpfs (and probably even if it isn't), it should still be pretty fast.. Have you checked if there's an increase in disk I/O usage, or system cpu usage? Or actually .. It could simply be that in v2.0.15 service lmtp { client_limit } default was changed to 1 (from default_client_limit=1000). This is important with the backend, because writing to message store can be slow, but proxying should be able to handle more than 1 client per process, even with the new temporary file writing. So you could see if it helps to set lmtp { client_limit = 100 } or something. From daniel.parthey at informatik.tu-chemnitz.de Sat Feb 2 04:32:20 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 2 Feb 2013 03:32:20 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BF90D.4000709@mclien.de> References: <510BEBDC.6020802@mclien.de> <510BEC7F.20409@thelounge.net> <510BF90D.4000709@mclien.de> Message-ID: <20130202023220.GA11927@daniel.localdomain> Hi Frank, Frank Lienhard wrote: > Reindl Harald wrote: > >Am 01.02.2013 17:22, schrieb Frank Lienhard: > >>I'm about to replace my old server. Both servers are on my internal network. > >>The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. > > > > imapsync is version agnostic and if you have a database > > with the user-logins it should be easy to generate a shellscript > > > sadly, this is not availble from the debian repos. imapsync is written in perl and available on github: https://github.com/imapsync/imapsync This is IMHO the best tool where dsync cannot be applied (as in your case). The only drawback of imapsync is that it cannot preserve UIDs since it uploads the messages via IMAP and the server will assign new message UIDs. Regards Daniel -- https://plus.google.com/103021802792276734820 From kamath at moltingpenguin.com Sat Feb 2 10:07:31 2013 From: kamath at moltingpenguin.com (Sean Kamath) Date: Sat, 2 Feb 2013 00:07:31 -0800 Subject: [Dovecot] dsync timeout? In-Reply-To: <87sj5gaudj.fsf@minnow.riseup.net> References: <87libacola.fsf@minnow.riseup.net> <3B764B67-F536-4BB3-BF2A-4777F4B6B3F0@iki.fi> <87halyck05.fsf@minnow.riseup.net> <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> <87sj5gaudj.fsf@minnow.riseup.net> Message-ID: <762DC7CC-1CF5-404E-AF36-41BA08B7C74B@moltingpenguin.com> On Feb 1, 2013, at 8:09 AM, micah anderson wrote: > Sean Kamath writes: > >> On Jan 30, 2013, at 3:46 PM, micah anderson wrote: >>> Seems that only the above process was still around and no other dsync >>> processes. I have three machines that all have this happening it seems. >>> >>> I wonder if there is a ssh configuration option I could set to make >>> these die off. >> >> If the ssh process isn't sending anything, and just waiting for read()s, and keepalives are turned off, the SSH session might never know the remote side is long gone. . . >> >> If any data were transmitted, it would discover the remote side is turned off. >> >> See man ssh_config and the option TCPKeepAlive. >> >> BTW: Since it's not on the command line, it's likely in /etc/ssh_config or /etc/ssh/ssh_config. Or ~/.ssh/config. > > In /etc/ssh/sshd_config on the server I'm sending to, TCPKeepAlive yes > is set. Did you check ~/.ssh/config for the user running the dsync? > The default on this system, according to the man page, seems to be to > have TCPKeepAlive set. > > Perhaps I should set ServerAliveInterval? Perhaps. That states how long to send the KeepAlive packet. There are many settings that can affect this, including ServerAliveCountMax ServerAliveInterval TCPKeepAlive There is also the sshd_config settings ClientAliveCountMax ClientAliveInterval TCPKeepAlive At this point, I think you need to see what's happening on both sides of the SSH connection. I don't recall what system you're on, but for linux you can use netstat -anp (as root) to find out what process is connected to which port, and on linux and other systems you can use lsof to find out what is connected to ports. Maybe the TCP port is open and valid and there's no data coming through? This can happen if, for example, you have any port forwarding or X session forwarding through SSH (i.e., if ssh -X is the default) and something accidentally is holding that port open (this can happen in your regular shell if, for example, you have something open an X application and you forget (because you backgrounded it) -- you're logout of the server will hang until the X applications are closed. Note that it isn't always a visible client that will do this. :-(). Sean From janfrode at tanso.net Sat Feb 2 12:59:04 2013 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 2 Feb 2013 11:59:04 +0100 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: On Fri, Feb 1, 2013 at 11:00 PM, Timo Sirainen wrote: > On 1.2.2013, at 19.00, Jan-Frode Myklebust wrote: > > Have you checked if there's an increase in disk I/O usage, or system cpu usage? > On the directors, cpu usage, and load averages seems to have gone down by about 50% since the upgrade. On the backend mailstores running 2.0.14 I see no effect (but these are quite busy, so less LMTP might just have lead to better response on other services). > Or actually .. It could simply be that in v2.0.15 service lmtp { client_limit } default was changed to 1 (from default_client_limit=1000). This is important with the backend, because writing to message store can be slow, but proxying should be able to handle more than 1 client per process, even with the new temporary file writing. So you could see if it helps to set lmtp { client_limit = 100 } or something. > My backend lmtp services are configured with client_limit = 1, process_limit = 25, and there are 6 backends I.e. max 150 backend LMTP processes if all lmtp is spread evenly between the backends, which it woun't be since backends are weighted differently (2x 50, 2x75 and 2x100). I assume each director will max proxy process_limit*client_limit to my backends. Will it be OK to have a much higher process_limit*client_limit on the directors than on the backends? It will not be a problem if directors are configured to seemingly handle a lot more simultaneous connections than the backends? -jf From tss at iki.fi Sat Feb 2 18:12:07 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 2 Feb 2013 18:12:07 +0200 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: On 2.2.2013, at 12.59, Jan-Frode Myklebust wrote: >> Or actually .. It could simply be that in v2.0.15 service lmtp { client_limit } default was changed to 1 (from default_client_limit=1000). This is important with the backend, because writing to message store can be slow, but proxying should be able to handle more than 1 client per process, even with the new temporary file writing. So you could see if it helps to set lmtp { client_limit = 100 } or something. >> > > My backend lmtp services are configured with client_limit = 1, > process_limit = 25, and there are 6 backends I.e. max 150 backend LMTP > processes if all lmtp is spread evenly between the backends, which it > woun't be since backends are weighted differently (2x 50, 2x75 and > 2x100). > > I assume each director will max proxy process_limit*client_limit to my > backends. Will it be OK to have a much higher > process_limit*client_limit on the directors than on the backends? It > will not be a problem if directors are configured to seemingly handle > a lot more simultaneous connections than the backends? Best to keep the bottleneck closest to MTA. If director can handle more connections than backend, then MTA is uselessly waiting on the extra LMTP connections to timeout. So I'd keep the director's process_limit*client_limit somewhat close to what backends can handle (somewhat more is probably ok too). Anyway, if backend reaches the limit it logs a warning about it and then just doesn't accept the connections until one of the existing ones finish. From joe.beaubien at gmail.com Mon Feb 4 01:07:59 2013 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Sun, 3 Feb 2013 18:07:59 -0500 Subject: [Dovecot] dsync backup questions Message-ID: Hi, I'm currently trying to setup remote backups of my emails but i'm running into issues (mdbox format, indexes and storage in the same folder hierarchy). Local backup command: dsync -u "my_user" backup /backups/my_user (1) Recently, I noticed that the local backup takes up twice the size as the original mail location (8gb vs. 4gb). I purged alot of emails from the original location, so the size shrunk, but the local backup just keeps on getting bigger. I couldn't find any dsync option that would delete extra emails. - Question: Why isn't the local backup synced properly and remove the extra emails? (2) What is the best why to copy this local backup to a remote location that does NOT have the possibility to run dsync. - Question 1: is rsync safe to use and will this data work for restore? - Question 2: Would it be safe to simply rsync the original mail_location to the remote server? I searched the net, but it seems that mdbox is tricky to backup. Thanks in advance, -Joe From ben at morrow.me.uk Mon Feb 4 02:57:29 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 4 Feb 2013 00:57:29 +0000 Subject: [Dovecot] dsync backup questions In-Reply-To: References: Message-ID: <20130204005729.GA68109@anubis.morrow.me.uk> I can't give authoratitive answers to either of these, but... At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: > > I'm currently trying to setup remote backups of my emails but i'm running > into issues (mdbox format, indexes and storage in the same folder > hierarchy). > > Local backup command: dsync -u "my_user" backup /backups/my_user > > (1) Recently, I noticed that the local backup takes up twice the size as > the original mail location (8gb vs. 4gb). I purged alot of emails from the > original location, so the size shrunk, but the local backup just keeps on > getting bigger. I couldn't find any dsync option that would delete extra > emails. > > - Question: Why isn't the local backup synced properly and remove the extra > emails? Are you running 'doveadm purge' on the backed-up dbox? It looks to me as though dsync doesn't do that. I don't know if there's any (simple) way to do that without a running Dovecot instance attached the dbox directory: it's not entirely clear to me whether doveadm will run locally without contacting a doveadm-server instance running under Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. It might be easier to dsync to a Maildir instead. This should preserve all the Dovecot-visible metadata, and dsyncing back to the dbox for restore should put it all back. > (2) What is the best why to copy this local backup to a remote location > that does NOT have the possibility to run dsync. > > - Question 1: is rsync safe to use and will this data work for restore? > > - Question 2: Would it be safe to simply rsync the original mail_location > to the remote server? AFAICT, if Dovecot is stopped on both sides of the transfer it should be safe. If either side has a currently running Dovecot instance (or any other Dovecot tools, like a concurrent dsync) using that dbox, it's likely rsync will copy an inconsistent snapshot of the data, and the result will be corrupted. If you have truly atomic filesystem snapshots, rsyncing out of a snapshot also *ought* to be safe, since the situation is the same as if Dovecot crashed at the moment the snapshot was taken, though it might be a good idea to run doveadm force-resync to make sure the result is consistent from Dovecot's point of view. Ben From dave at dawoodfall.net Mon Feb 4 04:01:05 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 4 Feb 2013 02:01:05 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc Message-ID: <20130204020105.GA15960@Blackswan> Version 2.1.10 For some reason Mutt doesn't list any info about IMAP folders and seems to ignore folder_format. My questions is does anyone have any workarounds for this? From dave at dawoodfall.net Mon Feb 4 04:01:05 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 4 Feb 2013 02:01:05 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc Message-ID: <20130204020105.GA15960@Blackswan> Version 2.1.10 For some reason Mutt doesn't list any info about IMAP folders and seems to ignore folder_format. My questions is does anyone have any workarounds for this? From dave at dawoodfall.net Mon Feb 4 04:01:05 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 4 Feb 2013 02:01:05 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc Message-ID: <20130204020105.GA15960@Blackswan> Version 2.1.10 For some reason Mutt doesn't list any info about IMAP folders and seems to ignore folder_format. My questions is does anyone have any workarounds for this? From daniel.parthey at informatik.tu-chemnitz.de Mon Feb 4 04:08:24 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 4 Feb 2013 03:08:24 +0100 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <20130204020105.GA15960@Blackswan> References: <20130204020105.GA15960@Blackswan> Message-ID: <20130204020824.GA14425@daniel.localdomain> David Woodfall wrote: > Version 2.1.10 > > For some reason Mutt doesn't list any info about IMAP folders and > seems to ignore folder_format. Doesn't have anything to with dovecot, does it? It looks as if folder_format option of mutt as only for local folders, and will not work for IMAP. Regards Daniel -- https://plus.google.com/103021802792276734820 From dave at dawoodfall.net Mon Feb 4 04:19:33 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 04 Feb 2013 02:19:33 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <20130204020824.GA14425@daniel.localdomain> References: <20130204020105.GA15960@Blackswan> <20130204020824.GA14425@daniel.localdomain> Message-ID: <510F1AB5.6080503@dawoodfall.net> On 04/02/13 02:08, Daniel Parthey wrote: > David Woodfall wrote: >> Version 2.1.10 >> >> For some reason Mutt doesn't list any info about IMAP folders and >> seems to ignore folder_format. > > Doesn't have anything to with dovecot, does it? > > It looks as if folder_format option of mutt > as only for local folders, and will not work for IMAP. I was grasping at straws and hoping someone here may know. From skdovecot at smail.inf.fh-brs.de Mon Feb 4 10:37:59 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 4 Feb 2013 09:37:59 +0100 (CET) Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BEBDC.6020802@mclien.de> References: <510BEBDC.6020802@mclien.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 1 Feb 2013, Frank Lienhard wrote: > My first attempt was tho simply rsync the Maildirs along with the homes. Both > deovecot versions are set up with the maildir format, but this results in > ~30% doublicated mails on the new server. Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)?? b) Or do you see them, when you login via IMAP to the the new server? === If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve hardlinks and/or "--delete" if you run rsync multiple times? If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh and empty local cache / profile / ... . Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUQ9zZ13r2wJMiz2NAQI67wf/X6XVPzXOygUZWX8s6Wthf8UyHs+Pew9v QKKuwOvm4lb7mF9vzVYMtTFwSHyYUCvSgB+oSPmRpX/bGReSq1+PwpIYQOpKvU+b EpaWTyBMXDyzToPuwnd/VZgnzzVuBa5IVbvg6neLIi7yrco+VjbwQJ1B/VPLHjRO YVusoMn0ddJWNE01ix2w54kBc9eSS54hcIuzkac3fyp2vTLl3LYgBJ0lRUp2ztkX C4sn1ejOqXkfBLc20jL52RW+FW0pH0P7HTCCz+XWkJFmiMXcDEoymBL62fI1MuAh UMNPvUIVcZcT/Q5FA8QGVRnRyKH5S4snRPkHEqSP2A39tDhbM/882A== =XIiO -----END PGP SIGNATURE----- From adam at xbsd.pl Mon Feb 4 10:51:36 2013 From: adam at xbsd.pl (Adam Maciejewski) Date: Mon, 4 Feb 2013 09:51:36 +0100 Subject: [Dovecot] Subfolders problem Message-ID: I have moved from dovecot 1.x to 2.x and I have big problem with subfolders. When I'm moving subfolder with other subfolders is moving only main subfolder, without subfolders, example : mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep -e "janusz|jarek" ./jarek ./jarek/dbox-Mails ./jarek/dbox-Mails/dovecot.index.log ./jarek/jarek2 ./jarek/jarek2/dbox-Mails ./jarek/jarek2/dbox-Mails/dovecot.index.log ./jarek/jarek2/jarek3 ./jarek/jarek2/jarek3/dbox-Mails ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log ./janusz ./janusz/dbox-Mails ./janusz/dbox-Mails/dovecot.index.log ./janusz/janusz2 ./janusz/janusz2/dbox-Mails ./janusz/janusz2/dbox-Mails/dovecot.index.log ./janusz/janusz2/janusz3 ./janusz/janusz2/janusz3/dbox-Mails ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log And now I'm moving jarek2 with jarek3 as subfolder to janusz2 subfolder, result : mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep -e "janusz|jarek" ./jarek ./jarek/dbox-Mails ./jarek/dbox-Mails/dovecot.index.log ./jarek/jarek2 ./jarek/jarek2/jarek3 ./jarek/jarek2/jarek3/dbox-Mails ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log ./janusz ./janusz/dbox-Mails ./janusz/dbox-Mails/dovecot.index.log ./janusz/janusz2 ./janusz/janusz2/dbox-Mails ./janusz/janusz2/dbox-Mails/dovecot.index.log ./janusz/janusz2/dbox-Mails/dovecot.index.cache ./janusz/janusz2/janusz3 ./janusz/janusz2/janusz3/dbox-Mails ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log ./janusz/janusz2/jarek2 ./janusz/janusz2/jarek2/dbox-Mails ./janusz/janusz2/jarek2/dbox-Mails/dovecot.index.log As You can see there is jarek2 subfolder but jarek3 havent been moved. I can see this folder in Thunderbird, after clicking error message pops out saying folder dosent exist. -- Adam Maciejewski From alessio at skye.it Mon Feb 4 11:15:56 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 10:15:56 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled Message-ID: <510F7C4C.7020303@skye.it> Hi, during the weekend I upgrade my mail servers from dovecot 2.1.13 to dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). After upgrade all works fine, but today the load avarage un MXs is doubled. Mailboxes are in Maildir/ on NFS and I'm using dovecot as LDA and dict quota only for quota usage. May be a problem with the new version? This is my configuration: $ ./configure --prefix=/usr/local/dovecot-2.1 --with-vpopmail --with-mysql --with-bzlib --with-zlib # cat /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve require ["vnd.dovecot.duplicate", "fileinto", "mailbox"]; if header :contains "X-Spam-Status" "Yes" { fileinto "Spam"; } if header :contains "Subject" "TEST 123" { fileinto "Spam"; } if duplicate { fileinto "Trash"; } ##################### # 2.1.14: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf expire = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl zlib expire maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.duplicate mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=192.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl expire = Trash expire2 = Spam expire_dict = proxy::expire quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.duplicate } protocols = imap pop3 sieve service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } service quota-warning { executable = script /usr/local/dovecot-2.1/etc/dovecot/scripts/quota-warning.sh unix_listener quota-warning { user = vpopmail } user = vpopmail } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From skdovecot at smail.inf.fh-brs.de Mon Feb 4 13:08:58 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 4 Feb 2013 12:08:58 +0100 (CET) Subject: [Dovecot] SOLVED for me: how to setup different quota for multiple namespaces In-Reply-To: <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 22 Jan 2013, Timo Sirainen wrote: Hey, it works (for me) now: namespace { type = private separator = . prefix = archive. location = mdbox:/home/%u/archive #subscriptions = no #list = children } plugin { quota = dict:User quota::ns=:proxy::quota quota2 = dict:Archive quota:%u.archive:ns=archive.:proxy::quota quota2_rule = *:storage=1048576 } Note the %u.archive _and_ ns=archive. ! The "%u.archive" is used as "username" in the SQL table and you require another name (primary key) there for both namespaces, see below. Seems to work for any type of namespace. @Timo: I will going to document this setup in the Wiki unless you say the syntax is wrong and/or works because of a bug only. :-) ================================== Old response with my debugging/findings for information only. ==== all the name spaces use one entry in the quota dict table, hence, it does not work. > On 17.1.2013, at 16.58, Andreas Oster wrote: > >>> I just saw on important difference in the doc and this configuration: >>> >>> see >>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>> >>> the ns=name syntax is for _public_ namespaces only. I just tested it >>> with this setup and every message is counted for both namespaces, if >>> delivered into INBOX or a mailbox of the Archive namespace. > > It should work for all namespaces. > >>>> plugin { >>>> quota = dict:User quota::proxy::quota >>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=Archive.:proxy::quota > > That should work? Worked at least in latest v2.1 hg. I use these settings now: # 2.1.14 (ea7e45c1da72+): /usr/local/dovecot-2.1.14/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 [snip] dict { quota = pgsql:/usr/local/dovecot-2.1.14/etc/dovecot/dovecot-dict-sql.conf.ext } namespace { location = mdbox:/home/%u/archive prefix = archive. separator = . type = private } plugin { quota = dict:User quota::ns=:proxy::quota quota2 = dict:Archive quota::ns=archive.:proxy::quota quota2_rule = *:storage=1048576 quota_rule = *:storage=300MB quota_rule2 = Trash:storage=+30M } The problem is now, that there is one row in the quota dict SQL table, that holds the quota of an user, e.g. doveadm quota recalc executes these SQL statements: 2013-02-04 11:52:07 CET LOG: statement: BEGIN 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (bytes,username) VALUES ('2173894','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (messages,username) VALUES ('89','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: COMMIT 2013-02-04 11:52:07 CET LOG: statement: BEGIN 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (bytes,username) VALUES ('2582','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (messages,username) VALUES ('2','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: COMMIT Deliveries to INBOX or a folder of "archive." yields: UPDATE quota SET bytes=bytes+1220,messages=messages+1 WHERE username = 'dvtest1' This is true if I change the type of name spaces "archive" from "private" into "shared" or "public". - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUQ+Wyl3r2wJMiz2NAQL2fwf/SV2SAl+lvzwN9TEEZfwtsHwTb9D7W1QH wfoMEJLWBFp8/i+gpjhbGNWbi9P5ogqljuJvALYVIXFNfoGllJBlAEStxBhpv5Z/ xQCY5lB9nCbnGa0g2MA0DMLF+dCmQrUdP0kl7R6moXaa4tn/C40pmkxvGSJjQsFk qMESy+cuREBSJq3SRP5g8tnlBC14dgoVfg+cWPVzOd/iQatIVQGghC5PqtLj36g3 rYJNvMZ8inGSf9pui21F9rHaN1RLeeACkAEF256Pt1lHBWKaecAhlvK+e3Ps1vru mssHT+z50T7jee8pwjMK6NMW4IbDgeU8wv9yaf4/c5LmfVPHP6E2cA== =D38F -----END PGP SIGNATURE----- From aoster at novanetwork.de Mon Feb 4 13:20:53 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 12:20:53 +0100 Subject: [Dovecot] SOLVED for me: how to setup different quota for multiple namespaces In-Reply-To: References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F9995.8060404@novanetwork.de> Am 04.02.2013 12:08, schrieb Steffen Kaiser: > On Tue, 22 Jan 2013, Timo Sirainen wrote: > > Hey, it works (for me) now: > > namespace { > type = private > separator = . > prefix = archive. > location = mdbox:/home/%u/archive > #subscriptions = no > #list = children > } > > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota:%u.archive:ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > } > > Note the %u.archive _and_ ns=archive. ! The "%u.archive" is used as > "username" in the SQL table and you require another name (primary key) > there for both namespaces, see below. Seems to work for any type of > namespace. > > @Timo: I will going to document this setup in the Wiki unless you say > the syntax is wrong and/or works because of a bug only. > > :-) > > ================================== > > Old response with my debugging/findings for information only. > > ==== > > all the name spaces use one entry in the quota dict table, hence, > it does not work. > >> On 17.1.2013, at 16.58, Andreas Oster wrote: > >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. > >> It should work for all namespaces. > >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota > >> That should work? Worked at least in latest v2.1 hg. > > I use these settings now: > > # 2.1.14 (ea7e45c1da72+): > /usr/local/dovecot-2.1.14/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 > > [snip] > > dict { > quota = > pgsql:/usr/local/dovecot-2.1.14/etc/dovecot/dovecot-dict-sql.conf.ext > } > namespace { > location = mdbox:/home/%u/archive > prefix = archive. > separator = . > type = private > } > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > quota_rule = *:storage=300MB > quota_rule2 = Trash:storage=+30M > } > > The problem is now, that there is one row in the quota dict SQL table, > that holds the quota of an user, e.g. doveadm quota recalc executes > these SQL statements: > > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2173894','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('89','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2582','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('2','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > > Deliveries to INBOX or a folder of "archive." yields: > > UPDATE quota SET bytes=bytes+1220,messages=messages+1 WHERE username = > 'dvtest1' > > This is true if I change the type of name spaces "archive" from "private" > into "shared" or "public". > > -- Steffen Kaiser > Hello Steffen, I do not use a SQL backend to store the quota information, so I guess the %u parameter is of little use with my setup. I have also tried to configure quota with dict and flat file, but this also did seem not work. Thanks best regards Andreas From aoster at novanetwork.de Mon Feb 4 13:20:53 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 12:20:53 +0100 Subject: [Dovecot] SOLVED for me: how to setup different quota for multiple namespaces In-Reply-To: References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F9995.8060404@novanetwork.de> Am 04.02.2013 12:08, schrieb Steffen Kaiser: > On Tue, 22 Jan 2013, Timo Sirainen wrote: > > Hey, it works (for me) now: > > namespace { > type = private > separator = . > prefix = archive. > location = mdbox:/home/%u/archive > #subscriptions = no > #list = children > } > > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota:%u.archive:ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > } > > Note the %u.archive _and_ ns=archive. ! The "%u.archive" is used as > "username" in the SQL table and you require another name (primary key) > there for both namespaces, see below. Seems to work for any type of > namespace. > > @Timo: I will going to document this setup in the Wiki unless you say > the syntax is wrong and/or works because of a bug only. > > :-) > > ================================== > > Old response with my debugging/findings for information only. > > ==== > > all the name spaces use one entry in the quota dict table, hence, > it does not work. > >> On 17.1.2013, at 16.58, Andreas Oster wrote: > >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. > >> It should work for all namespaces. > >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota > >> That should work? Worked at least in latest v2.1 hg. > > I use these settings now: > > # 2.1.14 (ea7e45c1da72+): > /usr/local/dovecot-2.1.14/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 > > [snip] > > dict { > quota = > pgsql:/usr/local/dovecot-2.1.14/etc/dovecot/dovecot-dict-sql.conf.ext > } > namespace { > location = mdbox:/home/%u/archive > prefix = archive. > separator = . > type = private > } > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > quota_rule = *:storage=300MB > quota_rule2 = Trash:storage=+30M > } > > The problem is now, that there is one row in the quota dict SQL table, > that holds the quota of an user, e.g. doveadm quota recalc executes > these SQL statements: > > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2173894','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('89','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2582','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('2','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > > Deliveries to INBOX or a folder of "archive." yields: > > UPDATE quota SET bytes=bytes+1220,messages=messages+1 WHERE username = > 'dvtest1' > > This is true if I change the type of name spaces "archive" from "private" > into "shared" or "public". > > -- Steffen Kaiser > Hello Steffen, I do not use a SQL backend to store the quota information, so I guess the %u parameter is of little use with my setup. I have also tried to configure quota with dict and flat file, but this also did seem not work. Thanks best regards Andreas From alessio at skye.it Mon Feb 4 15:28:01 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 14:28:01 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <510F7C4C.7020303@skye.it> References: <510F7C4C.7020303@skye.it> Message-ID: <510FB761.3040908@skye.it> Il 04/02/2013 10:15, Alessio Cecchi ha scritto: > Hi, > > during the weekend I upgrade my mail servers from dovecot 2.1.13 to > dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). > Moreover, in the log file I found these errors never seen before: Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory What do they mean? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From tss at iki.fi Mon Feb 4 15:58:16 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 15:58:16 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <510FB761.3040908@skye.it> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> Message-ID: <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> On 4.2.2013, at 15.28, Alessio Cecchi wrote: > Il 04/02/2013 10:15, Alessio Cecchi ha scritto: >> Hi, >> >> during the weekend I upgrade my mail servers from dovecot 2.1.13 to >> dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). >> > > Moreover, in the log file I found these errors never seen before: > > Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory Well .. Those errors should go away if you revert this patch: http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a But does it fix the performance? .. From tss at iki.fi Mon Feb 4 17:07:19 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 04 Feb 2013 17:07:19 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> Message-ID: <1359990439.3230.4.camel@hurina> On Mon, 2013-02-04 at 15:58 +0200, Timo Sirainen wrote: > On 4.2.2013, at 15.28, Alessio Cecchi wrote: > > > Il 04/02/2013 10:15, Alessio Cecchi ha scritto: > >> Hi, > >> > >> during the weekend I upgrade my mail servers from dovecot 2.1.13 to > >> dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). > >> > > > > Moreover, in the log file I found these errors never seen before: > > > > Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > > Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > > Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory > > Well .. Those errors should go away if you revert this patch: > > http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a > > But does it fix the performance? .. No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f From alessio at skye.it Mon Feb 4 17:38:30 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 16:38:30 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <1359990439.3230.4.camel@hurina> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> Message-ID: <510FD5F6.60808@skye.it> Il 04/02/2013 16:07, Timo Sirainen ha scritto: > On Mon, 2013-02-04 at 15:58 +0200, Timo Sirainen wrote: >> On 4.2.2013, at 15.28, Alessio Cecchi wrote: >> >>> Il 04/02/2013 10:15, Alessio Cecchi ha scritto: >>>> Hi, >>>> >>>> during the weekend I upgrade my mail servers from dovecot 2.1.13 to >>>> dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). >>>> >>> >>> Moreover, in the log file I found these errors never seen before: >>> >>> Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>> Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>> Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory >> >> Well .. Those errors should go away if you revert this patch: >> >> http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a >> >> But does it fix the performance? .. > > No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f > > Thanks Timo, tonight I will apply the second patch to 2.1.14 for the performance problem. Should I also remove the path "2b76d357a56a" or is only optional/for test? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From tss at iki.fi Mon Feb 4 17:44:07 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 17:44:07 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <510FD5F6.60808@skye.it> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: On 4.2.2013, at 17.38, Alessio Cecchi wrote: >>>> Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>> Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>> Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory >>> >>> Well .. Those errors should go away if you revert this patch: >>> >>> http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a >>> >>> But does it fix the performance? .. >> >> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f >> >> > > Thanks Timo, > > tonight I will apply the second patch to 2.1.14 for the performance problem. > > Should I also remove the path "2b76d357a56a" or is only optional/for test? Removing it removes those errors from logs, but other than that it shouldn't make any difference. Instead of removing you could also apply these patches that should remove them more correctly: http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 From kharford at constantcontact.com Mon Feb 4 17:45:41 2013 From: kharford at constantcontact.com (Harford, Ken) Date: Mon, 4 Feb 2013 15:45:41 +0000 Subject: [Dovecot] Overloading IMAP Server In-Reply-To: Message-ID: Okay, I set a wait period of 30 seconds and I am still having issues. I am sending about 50 messages from 6 servers (300 messages) to the same IMAP mailbox at the same time and dovecot doesn't seem to be able handle that kind of message count. I am using dovecot delivery instead of postfix delivery Any other ideas? Thanks Ken On 1/22/13 4:04 AM, "Steffen Kaiser" wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Mon, 21 Jan 2013, Harford, Ken wrote: > >> I am having an issue with dovecot IMAP reading and writing >>to >> one specific mailbox. I have a check email delivery script that sends > >What's the purpose of the verification: > >+ The prompt delivery? > >+ The delivery at all? > >+ The correct delivery? > >> out a email and then logs into dovecot to check if the mailbox that the >> message has been delivered to. However, the servers that send these >> messages send them in bunches depending on how many postfix spools it >>is >> checking. So I could get 60 messages all being sent to 1 mailbox and >> then having to read that same mailbox to check if the message actually >> got delivered. >> >> I am getting a lot of errors such as the following: >> >> Could not connect to IMAP server >> imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX > >Is this an error logged by Dovecot? > >> Any ideas as to what I can tweak in dovecot to allow these messages to >>be verified. > >a) >Wait a bit before logging into the IMAP server and check the messages >sequentionally, not in parallel. E.g. your script could synchronize >itself >with other parallel instances by a shared lock. > >b) >Use mail_log and a script on the mail server monitoring Dovecot logs and >register delivered message ids and/or recipient. Then figure out a good >way to probe, if one pariticular message has been seen. >- -or- >make note of message id/recipient on the sender side, somehow push (ssh >or >something) that information on regular basis to the mail server, where a >script verifies that data. > >c) To verify the correct delivery: use some sort of checksumming. > >Kind regards, > >- -- >Steffen Kaiser >-----BEGIN PGP SIGNATURE----- >Version: GnuPG v1.4.11 (GNU/Linux) > >iQEVAwUBUP5WEF3r2wJMiz2NAQIatAgAjwZR1VGnyURuncLRNeeY0dBaqWzAUcNK >g4YoiXB6NLxP6Xr/QO6Pw+6Q3sECJ0CJdwImOz56FVFNuiOygUMQ1dCdZ+Tpy9Wu >xeAdobrkOo2uzgRnUh7+WLeZ8k32gZZztCAQA90uTvB3MBox+7J/P8/mApp+8/tr >JKyhUy8RJIYXH+yrw/ZkbjmOSEEkf/wTPnBQcMKw5/4odha4lu+yEKx766mm0YCe >3USL7eVqN0Q9FwUH+iCjnRJAdKRETP4CnfQpjVp7eiJX7HD0iNwhucDwf037Ev2N >PG173M40pMllRyesjiV8gwOu8mAHu6V3QUiFJkYAKFOotUVO6xMrNQ== >=SeaP >-----END PGP SIGNATURE----- From jerry at seibercom.net Mon Feb 4 18:01:37 2013 From: jerry at seibercom.net (Jerry) Date: Mon, 4 Feb 2013 11:01:37 -0500 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: <20130204110137.637541f3@scorpio> On Mon, 4 Feb 2013 17:44:07 +0200 Timo Sirainen articulated: [snip > Removing it removes those errors from logs, but other than that it > shouldn't make any difference. Instead of removing you could also > apply these patches that should remove them more correctly: > > http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 Looks like you are going to have to release version 2.1.15 in the very near future. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Mon Feb 4 18:05:01 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 18:05:01 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <20130204110137.637541f3@scorpio> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <20130204110137.637541f3@scorpio> Message-ID: On 4.2.2013, at 18.01, Jerry wrote: > On Mon, 4 Feb 2013 17:44:07 +0200 > Timo Sirainen articulated: > > [snip > >> Removing it removes those errors from logs, but other than that it >> shouldn't make any difference. Instead of removing you could also >> apply these patches that should remove them more correctly: >> >> http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d >> http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > > Looks like you are going to have to release version 2.1.15 in the very > near future. Yeah. :( Would be nice if people tested those new patches, so I don't have to put out 2.1.16 soon also. From alessio at skye.it Mon Feb 4 11:15:56 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 10:15:56 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled Message-ID: <510F7C4C.7020303@skye.it> Hi, during the weekend I upgrade my mail servers from dovecot 2.1.13 to dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). After upgrade all works fine, but today the load avarage un MXs is doubled. Mailboxes are in Maildir/ on NFS and I'm using dovecot as LDA and dict quota only for quota usage. May be a problem with the new version? This is my configuration: $ ./configure --prefix=/usr/local/dovecot-2.1 --with-vpopmail --with-mysql --with-bzlib --with-zlib # cat /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve require ["vnd.dovecot.duplicate", "fileinto", "mailbox"]; if header :contains "X-Spam-Status" "Yes" { fileinto "Spam"; } if header :contains "Subject" "TEST 123" { fileinto "Spam"; } if duplicate { fileinto "Trash"; } ##################### # 2.1.14: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf expire = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl zlib expire maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.duplicate mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=192.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl expire = Trash expire2 = Spam expire_dict = proxy::expire quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.duplicate } protocols = imap pop3 sieve service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } service quota-warning { executable = script /usr/local/dovecot-2.1/etc/dovecot/scripts/quota-warning.sh unix_listener quota-warning { user = vpopmail } user = vpopmail } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From aoster at novanetwork.de Mon Feb 4 11:40:02 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 10:40:02 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F81F2.1020809@novanetwork.de> Am 22.01.2013 12:54, schrieb Timo Sirainen: > On 17.1.2013, at 16.58, Andreas Oster wrote: > >>> I just saw on important difference in the doc and this configuration: >>> >>> see >>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>> >>> the ns=name syntax is for _public_ namespaces only. I just tested it >>> with this setup and every message is counted for both namespaces, if >>> delivered into INBOX or a mailbox of the Archive namespace. > > It should work for all namespaces. > >>>> plugin { >>>> quota = dict:User quota::proxy::quota >>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=Archive.:proxy::quota > > That should work? Worked at least in latest v2.1 hg. > > Hello Timo, can you please confirm, that it is actually possible to have multiple quota roots with separate/different quota values. Currently I have this config: namespace { disabled = no hidden = no ignore_on_failure = no inbox = no list = children location = maildir:/var/vmail/archives/%n/Maildir prefix = Archives/ separator = / subscriptions = yes type = private } namespace inbox { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota_rule3 = ns=Archives/:storage=5G } With this setup I get the following error: Debug: Quota root: name=User quota backend=maildir args=ns= Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete configuration for rule 'storage=5G' should be changed to 'storage=+5G' Debug: Quota rule: root=User quota mailbox=ns=Archives/ bytes=+5368709120 messages=0 Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ Thank you for your kind help. best regards Andreas From aoster at novanetwork.de Mon Feb 4 11:40:02 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 10:40:02 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F81F2.1020809@novanetwork.de> Am 22.01.2013 12:54, schrieb Timo Sirainen: > On 17.1.2013, at 16.58, Andreas Oster wrote: > >>> I just saw on important difference in the doc and this configuration: >>> >>> see >>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>> >>> the ns=name syntax is for _public_ namespaces only. I just tested it >>> with this setup and every message is counted for both namespaces, if >>> delivered into INBOX or a mailbox of the Archive namespace. > > It should work for all namespaces. > >>>> plugin { >>>> quota = dict:User quota::proxy::quota >>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=Archive.:proxy::quota > > That should work? Worked at least in latest v2.1 hg. > > Hello Timo, can you please confirm, that it is actually possible to have multiple quota roots with separate/different quota values. Currently I have this config: namespace { disabled = no hidden = no ignore_on_failure = no inbox = no list = children location = maildir:/var/vmail/archives/%n/Maildir prefix = Archives/ separator = / subscriptions = yes type = private } namespace inbox { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota_rule3 = ns=Archives/:storage=5G } With this setup I get the following error: Debug: Quota root: name=User quota backend=maildir args=ns= Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete configuration for rule 'storage=5G' should be changed to 'storage=+5G' Debug: Quota rule: root=User quota mailbox=ns=Archives/ bytes=+5368709120 messages=0 Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ Thank you for your kind help. best regards Andreas From tlx at leuxner.net Mon Feb 4 12:56:53 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 4 Feb 2013 11:56:53 +0100 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <510F1AB5.6080503@dawoodfall.net> References: <20130204020105.GA15960@Blackswan> <20130204020824.GA14425@daniel.localdomain> <510F1AB5.6080503@dawoodfall.net> Message-ID: <20130204105653.GA21474@nihlus.leuxner.net> * David Woodfall 2013.02.04 03:19: > >It looks as if folder_format option of mutt > >as only for local folders, and will not work for IMAP. > > I was grasping at straws and hoping someone here may know. As mentioned this may be one for the mutt list. But it should work with IMAP: set imap_user="user at domain.tld" set folder="imap://host.domain.tld/" set spoolfile="imap://host.domain.tld/INBOX" set index_format="%4C %Z %2M %[!%Y.%m.%d %H:%M] %-30.30F (%5c) %s" Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From jerry at seibercom.net Mon Feb 4 18:01:37 2013 From: jerry at seibercom.net (Jerry) Date: Mon, 4 Feb 2013 11:01:37 -0500 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: <20130204110137.637541f3@scorpio> On Mon, 4 Feb 2013 17:44:07 +0200 Timo Sirainen articulated: [snip > Removing it removes those errors from logs, but other than that it > shouldn't make any difference. Instead of removing you could also > apply these patches that should remove them more correctly: > > http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 Looks like you are going to have to release version 2.1.15 in the very near future. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Mon Feb 4 18:05:01 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 18:05:01 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <20130204110137.637541f3@scorpio> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <20130204110137.637541f3@scorpio> Message-ID: On 4.2.2013, at 18.01, Jerry wrote: > On Mon, 4 Feb 2013 17:44:07 +0200 > Timo Sirainen articulated: > > [snip > >> Removing it removes those errors from logs, but other than that it >> shouldn't make any difference. Instead of removing you could also >> apply these patches that should remove them more correctly: >> >> http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d >> http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > > Looks like you are going to have to release version 2.1.15 in the very > near future. Yeah. :( Would be nice if people tested those new patches, so I don't have to put out 2.1.16 soon also. From jerry at seibercom.net Mon Feb 4 18:24:26 2013 From: jerry at seibercom.net (Jerry) Date: Mon, 4 Feb 2013 11:24:26 -0500 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <20130204110137.637541f3@scorpio> Message-ID: <20130204112426.2d483478@scorpio> On Mon, 4 Feb 2013 18:05:01 +0200 Timo Sirainen articulated: > On 4.2.2013, at 18.01, Jerry wrote: > > > On Mon, 4 Feb 2013 17:44:07 +0200 > > Timo Sirainen articulated: > > > > [snip > > > >> Removing it removes those errors from logs, but other than that it > >> shouldn't make any difference. Instead of removing you could also > >> apply these patches that should remove them more correctly: > >> > >> http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > >> http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > > > > Looks like you are going to have to release version 2.1.15 in the > > very near future. > > Yeah. :( > > Would be nice if people tested those new patches, so I don't have to > put out 2.1.16 soon also. I think that most people, myself included, are hesitant about applying patches to systems that are all ready up and running satisfactory. As you will no doubt concur, there is nothing worse that updating or applying a patch to a critical application and then discovering it is now broken. Perhaps, and this is just my 2? on the matter, you could form a small group of BETA testers. I have no idea if that would work out for you or not. In any event, Dovecot rarely has a serious BUG problem. Just those pesky "nits & gnats" that have to be tended to occasionally. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From andy at xecu.net Mon Feb 4 19:08:33 2013 From: andy at xecu.net (Andy Dills) Date: Mon, 04 Feb 2013 12:08:33 -0500 Subject: [Dovecot] Errors with doveadm when using checkpassword Message-ID: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> We have a checkpassword authentication with mysql pre-fetch for the userdb lookups. When trying to do: doveadm search -u andytest at xecu.net mailbox Trash DELETED I get: doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS lookup failed doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb lookup failed (to see if user is proxied, because doveadm_proxy_port is set) And I see this in the logs: Feb 4 12:02:04 mail-out01 dovecot: auth: Error: userdb-checkpassword(andytest at xecu.net): Child 12591 exited with status 1 I'm not surprised the checkpassword lookup is failing; the password isn't being supplied. Shouldn't it just be doing the prefetch lookup instead, like the lmtp service? Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From frank at mclien.de Mon Feb 4 19:59:30 2013 From: frank at mclien.de (Frank) Date: Mon, 04 Feb 2013 18:59:30 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: References: <510BEBDC.6020802@mclien.de> Message-ID: <510FF702.4030208@mclien.de> Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 1 Feb 2013, Frank Lienhard wrote: > >> My first attempt was tho simply rsync the Maildirs along with the >> homes. Both deovecot versions are set up with the maildir format, but >> this results in ~30% doublicated mails on the new server. > > Er, a) you got 30% more files on the new server (aka target), than you > have on the old server (aka source)?? > > b) Or do you see them, when you login via IMAP to the the new server? > > === > > If a), did you ran rsync, while the source host is in production > still? Do you use "H" option to preserve hardlinks and/or "--delete" > if you run rsync multiple times? > > If b), did you verified the duplication with a IMAP client, that does > _no_ caching on its own or with a fresh and empty local cache / > profile / ... . > I used -avu, where: -a equals: -rlptgoD (no -H,-A,-X) -u skip file that are newer on the receiver I stopped the getmail cronjob to ensure that no new messages get fetched while sync. After that I created a new account in Icedove and at first all went well, but after a while I got duoplicates of abou 900 old messages. I admit I only looked that up from Icedove and not in the dir from cli. From metro_domain_admin at fastmail.fm Mon Feb 4 22:34:50 2013 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Mon, 04 Feb 2013 15:34:50 -0500 Subject: [Dovecot] dsync-local(user): Error: proxy client timed out Message-ID: <1360010090.18396.140661186767549.3BEC6B22@webmail.messagingengine.com> Hello, I have an intermittent but frequent problem using dsync backup. The command outputs: dsync-local(user): Error: proxy client timed out Then dies with exit code 75. Any subsequent attempts to re-run the backup invariably fail like this: dsync-local(user): Warning: Destination mailbox INBOX has been modified, need to recreate it before we can continue syncing dsync-local(user): Error: read() from worker server failed: EOF dsync-local(user): Error: msg-copy returned invalid input: dsync-local(user): Error: remote: dsync-remote(user at domain.com): Error: Can't delete mailbox INBOX: INBOX can't be deleted. dsync-remote(l_user at domain.com): Error: command MSG-SAVE failed dsync-local(user): Warning: Mailbox changes caused a desync. You may want to run dsync again. The syntax I am using is: dsync -u $user backup \ ssh -i \ dsync -u l_$user We have this running in a script that loops through all users. The error happens mostly on the larger mailboxes but I've seen it succeed on the largest ones then fail on smaller. One mid-sized account now fails consistently. The backup is running from a Mac OS PPC server running 2.1.12 The target is a Linux Intel box running 2.1.13, but I had the same problem sending to a Mac OS Intel on 2.1.12. Mdbox in all cases. Thanks, Tony From list at airstreamcomm.net Mon Feb 4 23:24:14 2013 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 04 Feb 2013 15:24:14 -0600 Subject: [Dovecot] METADATA / ANNOTATE extensions Message-ID: <511026FE.3090901@airstreamcomm.net> According to the roadmap http://wiki2.dovecot.org/Roadmap implementation of the METADATA / ANNOTATE extensions is fairly high on the list. I was just curious if there is a target release in mind for this functionality? From tss at iki.fi Tue Feb 5 03:53:47 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 03:53:47 +0200 Subject: [Dovecot] METADATA / ANNOTATE extensions In-Reply-To: <511026FE.3090901@airstreamcomm.net> References: <511026FE.3090901@airstreamcomm.net> Message-ID: <1360029227.10326.42.camel@innu> On Mon, 2013-02-04 at 15:24 -0600, list at airstreamcomm.net wrote: > According to the roadmap http://wiki2.dovecot.org/Roadmap implementation > of the METADATA / ANNOTATE extensions is fairly high on the list. I was > just curious if there is a target release in mind for this functionality? v2.2 has the underlying functionality to set key=value to mailboxes, so it could still be added to v2.2. It needs: - dsync needs to be able to sync them - ACL plugin needs to restrict the set/get - IMAP commands to access them The dsync change is the main reason I haven't yet added it. From tss at iki.fi Tue Feb 5 03:55:01 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 03:55:01 +0200 Subject: [Dovecot] dsync-local(user): Error: proxy client timed out In-Reply-To: <1360010090.18396.140661186767549.3BEC6B22@webmail.messagingengine.com> References: <1360010090.18396.140661186767549.3BEC6B22@webmail.messagingengine.com> Message-ID: <1360029301.10326.43.camel@innu> On Mon, 2013-02-04 at 15:34 -0500, Metro Domain Admin wrote: > I have an intermittent but frequent problem using dsync backup. The > command outputs: > > dsync-local(user): Error: proxy client timed out > > Then dies with exit code 75. Any subsequent attempts to re-run the > backup invariably fail like this: > > dsync-local(user): Warning: Destination mailbox INBOX has been modified, > need to recreate it before we can continue syncing Looks like there are two bugs. v2.2 will have a redesigned dsync that should fix these problems. Too much trouble to try to fix it for v2.1. From tss at iki.fi Tue Feb 5 03:56:47 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 03:56:47 +0200 Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> Message-ID: <1360029407.10326.45.camel@innu> On Mon, 2013-02-04 at 12:08 -0500, Andy Dills wrote: > We have a checkpassword authentication with mysql pre-fetch for the > userdb lookups. > > When trying to do: > > doveadm search -u andytest at xecu.net mailbox Trash DELETED > > I get: > > doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS > lookup failed > doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb > lookup failed (to see if user is proxied, because doveadm_proxy_port is > set) Do you have proxying enabled? If not, remove doveadm_proxy_port and this solves itself. > I'm not surprised the checkpassword lookup is failing; the password > isn't being supplied. Shouldn't it just be doing the prefetch lookup > instead, like the lmtp service? doveadm doesn't care about the password. It cares about the proxy destination host so that it can automatically connect to the correct host to run the command. From tss at iki.fi Tue Feb 5 04:01:20 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:01:20 +0200 Subject: [Dovecot] dsync backup questions In-Reply-To: <20130204005729.GA68109@anubis.morrow.me.uk> References: <20130204005729.GA68109@anubis.morrow.me.uk> Message-ID: <1360029680.10326.48.camel@innu> On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote: > I can't give authoratitive answers to either of these, but... > > At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: > > > > I'm currently trying to setup remote backups of my emails but i'm running > > into issues (mdbox format, indexes and storage in the same folder > > hierarchy). > > > > Local backup command: dsync -u "my_user" backup /backups/my_user > > > > (1) Recently, I noticed that the local backup takes up twice the size as > > the original mail location (8gb vs. 4gb). I purged alot of emails from the > > original location, so the size shrunk, but the local backup just keeps on > > getting bigger. I couldn't find any dsync option that would delete extra > > emails. > > > > - Question: Why isn't the local backup synced properly and remove the extra > > emails? > > Are you running 'doveadm purge' on the backed-up dbox? It looks to me as > though dsync doesn't do that. I don't know if there's any (simple) way > to do that without a running Dovecot instance attached the dbox > directory: it's not entirely clear to me whether doveadm will run > locally without contacting a doveadm-server instance running under > Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. Right. doveadm -o mail=mdbox:/backups/my_user purge > It might be easier to dsync to a Maildir instead. This should preserve > all the Dovecot-visible metadata, and dsyncing back to the dbox for > restore should put it all back. Better sdbox than maildir. > > (2) What is the best why to copy this local backup to a remote location > > that does NOT have the possibility to run dsync. > > > > - Question 1: is rsync safe to use and will this data work for restore? > > > > - Question 2: Would it be safe to simply rsync the original mail_location > > to the remote server? > > AFAICT, if Dovecot is stopped on both sides of the transfer it should be > safe. If either side has a currently running Dovecot instance (or any > other Dovecot tools, like a concurrent dsync) using that dbox, it's > likely rsync will copy an inconsistent snapshot of the data, and the > result will be corrupted. It won't be badly corrupted though. At worst Dovecot will rebuild the indexes, which takes a while. And most users probably won't get any corruption at all. From tss at iki.fi Tue Feb 5 04:03:16 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:03:16 +0200 Subject: [Dovecot] Subfolders problem In-Reply-To: References: Message-ID: <1360029796.10326.49.camel@innu> On Mon, 2013-02-04 at 09:51 +0100, Adam Maciejewski wrote: > I have moved from dovecot 1.x to 2.x and I have big problem with > subfolders. > When I'm moving subfolder with other subfolders is moving only main > subfolder, without subfolders, example : What v2.x exactly? Looks like a bug. Probably fixed in a newer version. If you already have a recent v2.1, show your doveconf -n. > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > -e "janusz|jarek" > ./jarek > ./jarek/dbox-Mails > ./jarek/dbox-Mails/dovecot.index.log > ./jarek/jarek2 > ./jarek/jarek2/dbox-Mails > ./jarek/jarek2/dbox-Mails/dovecot.index.log > ./jarek/jarek2/jarek3 > ./jarek/jarek2/jarek3/dbox-Mails > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > ./janusz > ./janusz/dbox-Mails > ./janusz/dbox-Mails/dovecot.index.log > ./janusz/janusz2 > ./janusz/janusz2/dbox-Mails > ./janusz/janusz2/dbox-Mails/dovecot.index.log > ./janusz/janusz2/janusz3 > ./janusz/janusz2/janusz3/dbox-Mails > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > > And now I'm moving jarek2 with jarek3 as subfolder to janusz2 subfolder, > result : > > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > -e "janusz|jarek" > ./jarek > ./jarek/dbox-Mails > ./jarek/dbox-Mails/dovecot.index.log > ./jarek/jarek2 > ./jarek/jarek2/jarek3 > ./jarek/jarek2/jarek3/dbox-Mails > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > ./janusz > ./janusz/dbox-Mails > ./janusz/dbox-Mails/dovecot.index.log > ./janusz/janusz2 > ./janusz/janusz2/dbox-Mails > ./janusz/janusz2/dbox-Mails/dovecot.index.log > ./janusz/janusz2/dbox-Mails/dovecot.index.cache > ./janusz/janusz2/janusz3 > ./janusz/janusz2/janusz3/dbox-Mails > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > ./janusz/janusz2/jarek2 > ./janusz/janusz2/jarek2/dbox-Mails > ./janusz/janusz2/jarek2/dbox-Mails/dovecot.index.log > > As You can see there is jarek2 subfolder but jarek3 havent been moved. I > can see this folder in Thunderbird, after clicking error message pops out > saying folder dosent exist. > > From tss at iki.fi Tue Feb 5 04:06:42 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:06:42 +0200 Subject: [Dovecot] Broken physical size caching in Dovecot 2.1.10 In-Reply-To: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> References: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> Message-ID: <1360030002.10326.52.camel@innu> On Tue, 2013-01-29 at 19:58 +0000, Rob Redpath wrote: > I'm encountering two, probably related, errors in my Dovecot 2.1.10 > install. The issue relates to compressed email stored in Maildir format > directories. > > In some cases, a mailbox will become inaccessible, and the following > will be logged from a doveadm fetch:- > > doveadm(info at example.com): Error: Corrupted index cache file > /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken > physical size for mail UID 2777 > doveadm(info at example.com): Error: Cached message size smaller than > expected (18996 < 64624) > doveadm(info at example.com): Error: Maildir filename has wrong S value, > renamed the file from > /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z > to > /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z > > Interestingly, the file mentioned in the error ( > /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't > exist. > > The uncompressed size of the file is 64624 bytes, the compressed size is > 18996. Dovecot's zlib plugin requires that the maildir filenames have ,S=uncompressed size. Otherwise you get those errors. http://dovecot.org/tools/maildir-size-check.sh can be used to fix the filenames. From tss at iki.fi Tue Feb 5 04:09:00 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:09:00 +0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> Message-ID: <1360030140.10326.54.camel@innu> On Tue, 2013-01-29 at 16:29 -0200, Gustavo Sansone wrote: > hello everyone, i'm new to this list > im running dovecot as imap+pop (2.1.12) and roundcube (0.8.4) as > client. > > having problems on saving to imap folders, with errors on some accounts > > my folder structure is as follow: > > /home/user/.maildir > /home/user/.maildir/.Drafts > /home/user/.maildir/.Junk > /home/user/.maildir/.Sent > /home/user/.maildir/.Trash > /home/user/.maildir/.Archives > /home/user/.maildir/.Archives.2010 > ... > > i'm very confused with the namespace settings. i don't understand what > is the correct way to configure it. > don't need public or shared folders. Easiest to use the default namespace settings. There you wouldn't have any namespace prefix. > on dovecot.log i can see this error with my account (and all the > others), but it seems to work ok using roundcube: > imap(gsansone): Error: autocreate: No namespace found for mailbox: Junk In your current setup you have namespace prefix=INBOX. so the mailbox name would be INBOX.Junk. > with a few accounts, i have this on roundcube log: > IMAP Error: Could not save message in Drafts in > /var/www/localhost/htdocs/mail/program/steps/mail/sendmail.inc > with error on saving to imap folder in roundcube UI Roundcube also isn't configured to have INBOX. prefix. Sounds like you should just remove it. From andy at xecu.net Tue Feb 5 04:16:41 2013 From: andy at xecu.net (Andy Dills) Date: Mon, 4 Feb 2013 21:16:41 -0500 Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <1360029407.10326.45.camel@innu> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> Message-ID: On Feb 4, 2013, at 8:56 PM, Timo Sirainen wrote: > On Mon, 2013-02-04 at 12:08 -0500, Andy Dills wrote: >> We have a checkpassword authentication with mysql pre-fetch for the >> userdb lookups. >> >> When trying to do: >> >> doveadm search -u andytest at xecu.net mailbox Trash DELETED >> >> I get: >> >> doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS >> lookup failed >> doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb >> lookup failed (to see if user is proxied, because doveadm_proxy_port is >> set) > > Do you have proxying enabled? If not, remove doveadm_proxy_port and this > solves itself Yes, I apologize, it appears I edited out the paragraph where I described our setup. Cluster of dovecot dual instance servers, one proxy/director, one backend. The backend config lives in dovecot.conf, the proxy config in proxy.conf (let me know if the specific config would help). Version 2.1.12 (current version in FreeBSD ports). I should mention, I have to specify the director-admin socket manually when using doveadm director, it's as if doveadm isn't quite aware of the config in the proxy instance. However, the authentication happens in the primary "dovecot.conf" instance...that's where checkpassword gets called, and where the prefetch is configured. The proxy is "dumb". >> I'm not surprised the checkpassword lookup is failing; the password >> isn't being supplied. Shouldn't it just be doing the prefetch lookup >> instead, like the lmtp service? > > doveadm doesn't care about the password. It cares about the proxy > destination host so that it can automatically connect to the correct > host to run the command. I understand doveadm doesn't care about the password; i was confused why checkpassword was being called when the password is unknown rather than using the mysql prefetch. So, I guess what you're saying is doveadm should be talking to the proxy instance rather than the backend instance? I assume I need to move the backend to a differently named config and make my proxy instances live in dovecot.conf, so that doveadm sees the "right" config? Thanks, Andy From tss at iki.fi Tue Feb 5 04:39:07 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:39:07 +0200 Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> Message-ID: <1360031947.10326.67.camel@innu> On Mon, 2013-02-04 at 21:16 -0500, Andy Dills wrote: > On Feb 4, 2013, at 8:56 PM, Timo Sirainen wrote: > > > On Mon, 2013-02-04 at 12:08 -0500, Andy Dills wrote: > >> We have a checkpassword authentication with mysql pre-fetch for the > >> userdb lookups. > >> > >> When trying to do: > >> > >> doveadm search -u andytest at xecu.net mailbox Trash DELETED > >> > >> I get: > >> > >> doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS > >> lookup failed > >> doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb > >> lookup failed (to see if user is proxied, because doveadm_proxy_port is > >> set) > > > > Do you have proxying enabled? If not, remove doveadm_proxy_port and this > > solves itself > > Yes, I apologize, it appears I edited out the paragraph where I described our setup. > > Cluster of dovecot dual instance servers, one proxy/director, one backend. The backend config lives in dovecot.conf, the proxy config in proxy.conf (let me know if the specific config would help). Version 2.1.12 (current version in FreeBSD ports). > > I should mention, I have to specify the director-admin socket manually when using doveadm director, it's as if doveadm isn't quite aware of the config in the proxy instance. > > However, the authentication happens in the primary "dovecot.conf" instance...that's where checkpassword gets called, and where the prefetch is configured. The proxy is "dumb". OK, so you are using director, so it is good that doveadm is doing a proxy lookup, otherwise it might run it on the wrong server and cause troubles with NFS caching (which is exactly what director is supposed to prevent). > >> I'm not surprised the checkpassword lookup is failing; the password > >> isn't being supplied. Shouldn't it just be doing the prefetch lookup > >> instead, like the lmtp service? > > > > doveadm doesn't care about the password. It cares about the proxy > > destination host so that it can automatically connect to the correct > > host to run the command. > > > I understand doveadm doesn't care about the password; i was confused why checkpassword was being called when the password is unknown rather than using the mysql prefetch. > > So, I guess what you're saying is doveadm should be talking to the proxy instance rather than the backend instance? I assume I need to move the backend to a differently named config and make my proxy instances live in dovecot.conf, so that doveadm sees the "right" config? I think you need to remove doveadm_proxy_port from the backend dovecot.conf. Then it doesn't perform the PASS lookup. But you also should run doveadm via the proxy instance so that it gets run in the correct server (doveadm -c /etc/dovecot/proxy.conf or doveadm -i proxy if you've given it a name). From andy at xecu.net Tue Feb 5 05:15:19 2013 From: andy at xecu.net (Andy Dills) Date: Mon, 4 Feb 2013 22:15:19 -0500 (EST) Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <1360031947.10326.67.camel@innu> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> <1360031947.10326.67.camel@innu> Message-ID: <20130204214631.Y69315@shell.xecu.net> On Tue, 5 Feb 2013, Timo Sirainen wrote: > > So, I guess what you're saying is doveadm should be talking to the > > proxy instance rather than the backend instance? I assume I need to > > move the backend to a differently named config and make my proxy > > instances live in dovecot.conf, so that doveadm sees the "right" > > config? > > I think you need to remove doveadm_proxy_port from the backend > dovecot.conf. Thanks Timo. That gave me: doveadm -c /usr/local/etc/dovecot/proxy.conf search -u andynew at xecu.net mailbox Trash DELETED doveadm(andytest at xecu.net): Error: doveadm_password not set, can't authenticate to remote server doveadm(andytest at xecu.net): Error: 10.0.0.47:30003: Internal failure for andytest at xecu.net So, on a whim, I moved the doveadm_password out from the local {} sections into the main level, and then things started to work as expected. Is that how it should be setup now? Thanks again for help getting me straightened out! This all started because I want to start purging the trash with doveadm instead of 'find' with -delete. Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From adam at xbsd.pl Tue Feb 5 10:00:52 2013 From: adam at xbsd.pl (Adam Maciejewski) Date: Tue, 5 Feb 2013 09:00:52 +0100 Subject: [Dovecot] Subfolders problem In-Reply-To: <1360029796.10326.49.camel@innu> References: <1360029796.10326.49.camel@innu> Message-ID: # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.1 auth_debug = yes auth_master_user_separator = * auth_username_format = %Ln default_vsz_limit = 2 G dict { sieve = mysql:/etc/dovecot/dict-sql-sieve.conf sieve_dir = mysql:/etc/dovecot/dict-sql-sieve_dir.conf } disable_plaintext_auth = no dotlock_use_excl = no dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l5 log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = xbsd it solutions mail_access_groups = mail mail_debug = yes mail_location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n mail_plugins = acl namespace { hidden = no list = children location = maildir:/var/vmail/public/xbsd/Maildir:INDEX=~/mdbox/public/xbsd:CONTROL=~/mdbox/public/xbsd prefix = xbsd/ separator = / subscriptions = no type = public } namespace deleted { hidden = yes list = no location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n:MAILBOXDIR=deleted:SUBSCRIPTIONS=subscriptions-deleted prefix = .DELETED/ separator = / type = private } namespace deleted_expunged { hidden = yes list = no location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n:MAILBOXDIR=deleted_expunged:SUBSCRIPTIONS=subscriptions-deleted_expunged prefix = .DELETED/.EXPUNGED/ separator = / type = private } namespace expunged { hidden = yes list = no location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n:MAILBOXDIR=expunged:SUBSCRIPTIONS=subscriptions-expunged prefix = .EXPUNGED/ separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/master.users.%s driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile lazy_expunge = .EXPUNGED/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append save mailbox_create mail_log_fields = uid box msgid size vsize flags from subject mail_replica = remote:mail-storage-2.atm sieve = dict:proxy::sieve;name=active;bindir=~/.sieve-bin sieve_dir = dict:proxy::sieve_dir;bindir=~/.sieve-bin } protocols = imap lmtp pop3 service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service auth { executable = /usr/lib/dovecot/auth unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } unix_listener auth-userdb { mode = 0666 } user = root } service dict { unix_listener dict { mode = 0666 } } service doveadm { process_min_avail = 1 service_count = 1024 } service imap-login { chroot = login executable = /usr/lib/dovecot/imap-login inet_listener imap { address = * port = 143 } service_count = 0 user = dovecot } service imap { executable = /usr/lib/dovecot/imap } service lmtp { inet_listener lmtp { port = 24 } process_limit = 48 } service pop3-login { chroot = login executable = /usr/lib/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } service_count = 0 user = dovecot } service pop3 { executable = /usr/lib/dovecot/pop3 } service replicator { process_min_avail = 1 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol doveadm { namespace deleted { list = yes location = prefix = } namespace deleted_expunged { list = yes location = prefix = } namespace expunged { list = yes location = prefix = } } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = acl lazy_expunge notify replication } protocol lmtp { mail_plugins = acl notify replication sieve } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_plugins = notify mail_log sieve acl } 2013/2/5 Timo Sirainen > On Mon, 2013-02-04 at 09:51 +0100, Adam Maciejewski wrote: > > I have moved from dovecot 1.x to 2.x and I have big problem with > > subfolders. > > When I'm moving subfolder with other subfolders is moving only main > > subfolder, without subfolders, example : > > What v2.x exactly? Looks like a bug. Probably fixed in a newer version. > If you already have a recent v2.1, show your doveconf -n. > > > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > > -e "janusz|jarek" > > ./jarek > > ./jarek/dbox-Mails > > ./jarek/dbox-Mails/dovecot.index.log > > ./jarek/jarek2 > > ./jarek/jarek2/dbox-Mails > > ./jarek/jarek2/dbox-Mails/dovecot.index.log > > ./jarek/jarek2/jarek3 > > ./jarek/jarek2/jarek3/dbox-Mails > > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > > ./janusz > > ./janusz/dbox-Mails > > ./janusz/dbox-Mails/dovecot.index.log > > ./janusz/janusz2 > > ./janusz/janusz2/dbox-Mails > > ./janusz/janusz2/dbox-Mails/dovecot.index.log > > ./janusz/janusz2/janusz3 > > ./janusz/janusz2/janusz3/dbox-Mails > > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > > > > And now I'm moving jarek2 with jarek3 as subfolder to janusz2 subfolder, > > result : > > > > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > > -e "janusz|jarek" > > ./jarek > > ./jarek/dbox-Mails > > ./jarek/dbox-Mails/dovecot.index.log > > ./jarek/jarek2 > > ./jarek/jarek2/jarek3 > > ./jarek/jarek2/jarek3/dbox-Mails > > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > > ./janusz > > ./janusz/dbox-Mails > > ./janusz/dbox-Mails/dovecot.index.log > > ./janusz/janusz2 > > ./janusz/janusz2/dbox-Mails > > ./janusz/janusz2/dbox-Mails/dovecot.index.log > > ./janusz/janusz2/dbox-Mails/dovecot.index.cache > > ./janusz/janusz2/janusz3 > > ./janusz/janusz2/janusz3/dbox-Mails > > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > > ./janusz/janusz2/jarek2 > > ./janusz/janusz2/jarek2/dbox-Mails > > ./janusz/janusz2/jarek2/dbox-Mails/dovecot.index.log > > > > As You can see there is jarek2 subfolder but jarek3 havent been moved. I > > can see this folder in Thunderbird, after clicking error message pops out > > saying folder dosent exist. > > > > > > > -- Adam Maciejewski xBSD IT Solutions ul. R??yckiego 8 - 78-100 Ko?obrzeg tel. kom. +48 665 138 884 GG: 13821 Regon: 320797083 NIP: 671-171-49-61 From alessio at skye.it Tue Feb 5 11:07:43 2013 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 05 Feb 2013 10:07:43 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: <5110CBDF.4030302@skye.it> Il 04/02/2013 16:44, Timo Sirainen ha scritto: > On 4.2.2013, at 17.38, Alessio Cecchi wrote: > >>>>> Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>>> Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>>> Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory >>>> >>>> Well .. Those errors should go away if you revert this patch: >>>> >>>> http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a >>>> >>>> But does it fix the performance? .. >>> >>> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f >>> >>> >> >> Thanks Timo, >> >> tonight I will apply the second patch to 2.1.14 for the performance problem. >> >> Should I also remove the path "2b76d357a56a" or is only optional/for test? > > Removing it removes those errors from logs, but other than that it shouldn't make any difference. Instead of removing you could also apply these patches that should remove them more correctly: > > http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > Also after the application of these patches the situation of the load is the same. If there are no other solutions I will return to 2.1.13 ASAP. Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From janfrode at tanso.net Tue Feb 5 11:57:02 2013 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 5 Feb 2013 10:57:02 +0100 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: I think there must be some bug I'm hitting here. One of my directors is still running with "client_limit = 1, process_limit = 100" for the lmtp service, and now it's logging: master: Warning: service(lmtp): process_limit (100) reached, client connections are being dropped Checking "sudo netstat -anp|grep ":24 " I see 287 ports in TIME_WAIT, one in CLOSE_WAIT and the listening "0.0.0.0:24". No active connections. There are 100 lmtp-processes running. When trying to connect to the lmtp-port I immediately get dropped: $ telnet localhost 24 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. Connection closed by foreign host. Is there maybe some counter that's getting out of sync, or some back off penalty algorithm that kicks in when it first hit the process limit ? -jf From andy at xecu.net Tue Feb 5 14:53:50 2013 From: andy at xecu.net (Andy Dills) Date: Tue, 5 Feb 2013 07:53:50 -0500 (EST) Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <1360031947.10326.67.camel@innu> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> <1360031947.10326.67.camel@innu> Message-ID: <20130205072304.C88642@shell.xecu.net> On Tue, 5 Feb 2013, Timo Sirainen wrote: > I think you need to remove doveadm_proxy_port from the backend > dovecot.conf. Then it doesn't perform the PASS lookup. But you also > should run doveadm via the proxy instance so that it gets run in the > correct server (doveadm -c /etc/dovecot/proxy.conf or doveadm -i proxy > if you've given it a name). On a seperate note I'm sure a lot of people would benefit from -c/-i being mentioned on http://wiki2.dovecot.org/Tools/Doveadm. You are one man with only so much time so I tried registering on the wiki to propose an edit for you, but I'm not allowed. I think all we need is to know that -c and -i exist, and a note about how people in proxy/director configurations need to make sure to tell doveadm to communicate with the instance that is running director. For some reason, my intuition would be that since doveadm is aware of both instances, that it should be aware of which one's config to use for connecting to director for proxy information. Thanks, Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From valerius at afterlogic.com Tue Feb 5 15:58:14 2013 From: valerius at afterlogic.com (Valery V. Sedletski) Date: Tue, 5 Feb 2013 13:58:14 +0000 Subject: [Dovecot] "Header is huge" in fts-solr Message-ID: Hi, Timo and all! I am trying to index mail in a test mailbox using fts_solr plugin for full-text search. On most mailboxes, it works fine, but on some big messages I get warnings like the following, and then I get an Out of memory error from Solr, then the indexer-worker process (or doveadm) crashes with "assertion failed" error and the backtrace: ========================================================== doveadm(valerius at test.afterlogic.com): Warning: fts-solr(valerius at test.afterlogic.com): Mailbox gmail.com UID=48 header size is huge doveadm(valerius at test.afterlogic.com): Warning: fts-solr(valerius at test.afterlogic.com): Mailbox gmail.com UID=49 header size is huge doveadm(valerius at test.afterlogic.com): Panic: file ../../../../src/plugins/fts-solr/solr-connection.c: line 548 (solr_connection_post_more): assertion failed: (maxfd >= 0) doveadm(valerius at test.afterlogic.com): Error: Raw backtrace: /usr/mailsuite/lib/dovecot/libdovecot.so.0(+0x58f04) [0x7fe8a908af04] -> /usr/mailsuite/lib/dovecot/libdovecot.so.0(default_error_handler+0) [0x7fe8a908af93] -> /usr/mailsuite/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fe8a908b274] -> /usr/mailsuite/lib/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x2d2) [0x7fe8a75fe973] -> /usr/mailsuite/lib/dovecot/lib21_fts_solr_plugin.so(+0x4d03) [0x7fe8a75f9d03] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_build_more+0x77) [0x7fe8a7c1d401] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0x7fe2) [0x7fe8a7c1dfe2] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0x80d5) [0x7fe8a7c1e0d5] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0x89e4) [0x7fe8a7c1e9e4] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2b) [0x7fe8a7c1ebf5] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0xe7cf) [0x7fe8a7c247cf] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0xe8ba) [0x7fe8a7c248ba] -> /usr/mailsuite/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x25) [0x7fe8a9379bc9] -> /usr/mailsuite/bin/doveadm() [0x4139de] -> /usr/mailsuite/bin/doveadm() [0x413c17] -> /usr/mailsuite/bin/doveadm() [0x413f18] -> /usr/mailsuite/bin/doveadm() [0x40fea6] -> /usr/mailsuite/bin/doveadm(doveadm_mail_single_user+0x154) [0x410069] -> /usr/mailsuite/bin/doveadm() [0x41090a] -> /usr/mailsuite/bin/doveadm(doveadm_mail_try_run+0xac) [0x410b81] -> /usr/mailsuite/bin/doveadm(main+0x28d) [0x41a92c] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fe8a8cc9ead] -> /usr/mailsuite/bin/doveadm() [0x40f499] ========================================================== And Solr log, at the same time: ========================================================== 2013-02-01 18:03:53.342:INFO::Logging to STDERR via org.mortbay.log.StdErrLog 2013-02-01 18:03:53.425:INFO::jetty-6.1-SNAPSHOT 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader INFO: new SolrResourceLoader for deduced Solr Home: 'solr/' 01.02.2013 18:03:53 org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:53 org.apache.solr.core.CoreContainerSInitializer initialize INFO: looking for solr.xml: /home/valerius/apache-solr-3.6.2/example/solr/solr.xml 01.02.2013 18:03:53 org.apache.solr.core.CoreContainer load INFO: Loading CoreContainer using Solr Home: 'solr/' 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader INFO: new SolrResourceLoader for directory: 'solr/' 01.02.2013 18:03:53 org.apache.solr.core.CoreContainer create INFO: Creating SolrCore '' using instanceDir: solr/. 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader INFO: new SolrResourceLoader for directory: 'solr/./' 01.02.2013 18:03:53 org.apache.solr.core.SolrConfig initLibs INFO: Adding specified lib dirs to ClassLoader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-cell-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-ooxml-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/jdom-1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/bcprov-jdk15-1.45.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/xmlbeans-2.3.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/boilerpipe-1.1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/xercesImpl-2.8.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/tika-core-1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/metadata-extractor-2.4.0-beta-1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/apache-mime4j-core-0.7.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/fontbox-1.6.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/pdfbox-1.6.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/dom4j-1.6.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/asm-3.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/jempbox-1.6.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-ooxml-schemas-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/rome-0.9.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/bcmail-jdk15-1.45.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/tagsoup-1.2.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/icu4j-4.8.1.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/tika-parsers-1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-scratchpad-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/commons-compress-1.3.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/xml-apis-1.0.b2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/apache-mime4j-dom-0.7.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-clustering-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/jackson-mapper-asl-1.7.4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/mahout-collections-0.3.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/mahout-math-0.3.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/hppc-0.3.4-jdk15.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/carrot2-core-3.5.0.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/jackson-core-asl-1.7.4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/simple-xml-2.4.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-dataimporthandler-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/dataimporthandler/lib/mail-1.4.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/dataimporthandler/lib/activation-1.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-langid-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/langid/lib/langdetect-1.1-20120112.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/langid/lib/jsonic-1.2.7.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-velocity-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/velocity-1.6.4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/commons-collections-3.2.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/commons-beanutils-1.7.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/velocity-tools-2.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrConfig INFO: Using Lucene MatchVersion: LUCENE_36 01.02.2013 18:03:54 org.apache.solr.core.SolrConfig INFO: Loaded SolrConfig: solrconfig.xml 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: Reading Solr Schema 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: Schema name=dovecot 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created string: org.apache.solr.schema.StrField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created long: org.apache.solr.schema.LongField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created slong: org.apache.solr.schema.SortableLongField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created float: org.apache.solr.schema.FloatField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created boolean: org.apache.solr.schema.BoolField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WhitespaceTokenizerFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.SynonymFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.StopFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WordDelimiterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.LowerCaseFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.SnowballPorterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.RemoveDuplicatesTokenFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WhitespaceTokenizerFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.StopFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WordDelimiterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.LowerCaseFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.SnowballPorterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.NGramFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.RemoveDuplicatesTokenFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created text: org.apache.solr.schema.TextField 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: default search field in schema is body 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: query parser default operator is AND 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: unique key field: id 01.02.2013 18:03:54 org.apache.solr.core.SolrCore INFO: [] Opening new SolrCore at solr/./, dataDir=solr/./data/ 01.02.2013 18:03:54 org.apache.solr.core.JmxMonitoredMap INFO: JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer at 27b4c1d7 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initListeners INFO: [] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]} 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initListeners INFO: [] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]} 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: created json: solr.JSONResponseWriter 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: adding lazy queryResponseWriter: solr.VelocityResponseWriter 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: created velocity: solr.VelocityResponseWriter 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: created xslt: solr.XSLTResponseWriter 01.02.2013 18:03:54 org.apache.solr.response.XSLTResponseWriter init INFO: xsltCacheLifetimeSeconds=5 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /select: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /browse: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update: solr.XmlUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/javabin: solr.BinaryUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.CSVRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/csv: solr.CSVRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.JsonUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/json: solr.JsonUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.extraction.ExtractingRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/extract: solr.extraction.ExtractingRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.XsltUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/xslt: solr.XsltUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.FieldAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /analysis/field: solr.FieldAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.DocumentAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /analysis/document: solr.DocumentAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /admin/: solr.admin.AdminHandlers 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /admin/ping: solr.PingRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /debug/dump: solr.DumpRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /spell: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /tvrh: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /terms: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /elevate: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.search.SolrIndexSearcher INFO: Opening Searcher at 5c29ea31 main 01.02.2013 18:03:54 org.apache.solr.update.CommitTracker INFO: commitTracker AutoCommit: disabled 01.02.2013 18:03:54 org.apache.solr.handler.component.SpellCheckComponent inform INFO: Initializing spell checkers 01.02.2013 18:03:54 org.apache.solr.handler.component.SpellCheckComponent inform INFO: No queryConverter defined, using default converter 01.02.2013 18:03:54 org.apache.solr.handler.component.QueryElevationComponent inform INFO: Loading QueryElevation from: /home/valerius/apache-solr-3.6.2/example/solr/./conf/elevate.xml 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.QueryComponent at 4d16318b 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.FacetComponent at 6c0ec436 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.MoreLikeThisComponent at 3d73eca6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.HighlightComponent at 5b5a5cf 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.StatsComponent at 5898bbf6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding debug component:org.apache.solr.handler.component.DebugComponent at 135c40eb 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting socketTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting urlScheme to: http:// 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting connTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxConnectionsPerHost to: 20 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting corePoolSize to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maximumPoolSize to: 2147483647 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxThreadIdleTime to: 5 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting sizeOfQueue to: -1 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting fairnessPolicy to: false 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.QueryComponent at 4d16318b 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.FacetComponent at 6c0ec436 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.MoreLikeThisComponent at 3d73eca6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.HighlightComponent at 5b5a5cf 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.StatsComponent at 5898bbf6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.SpellCheckComponent at 18f2225f 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding debug component:org.apache.solr.handler.component.DebugComponent at 135c40eb 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting socketTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting urlScheme to: http:// 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting connTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxConnectionsPerHost to: 20 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting corePoolSize to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maximumPoolSize to: 2147483647 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxThreadIdleTime to: 5 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting sizeOfQueue to: -1 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting fairnessPolicy to: false 01.02.2013 18:03:54 org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener sending requests to Searcher at 5c29ea31 main 01.02.2013 18:03:54 org.apache.solr.core.CoreContainer register INFO: registering core: 01.02.2013 18:03:54 org.apache.solr.servlet.SolrDispatchFilter init INFO: user.dir=/home/valerius/apache-solr-3.6.2/example 01.02.2013 18:03:54 org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() done 01.02.2013 18:03:54 org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1330) at org.apache.solr.schema.IndexSchemaSSolrQueryAnalyzer.getAnalyzer(IndexSchema.java:408) at org.apache.solr.schema.IndexSchemaSSolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:383) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:574) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:206) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1436) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1319) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1234) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79) at org.apache.solr.search.QParser.getQuery(QParser.java:143) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:105) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:59) at org.apache.solr.core.SolrCoreS3.call(SolrCore.java:1182) at java.util.concurrent.FutureTaskSSync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutorSWorker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 01.02.2013 18:03:54 org.apache.solr.core.SolrCore execute INFO: [] webapp=null path=null params={event=firstSearcher&q=static+firstSearcher+warming+in+solrconfig.xml} status=400 QTime=22 01.02.2013 18:03:54 org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener done. 01.02.2013 18:03:54 org.apache.solr.handler.component.SpellCheckComponentSSpellCheckerListener newSearcher INFO: Loading spell index for spellchecker: default 01.02.2013 18:03:54 org.apache.solr.core.SolrCore registerSearcher INFO: [] Registered new searcher Searcher at 5c29ea31 main 01.02.2013 18:03:54 org.apache.solr.servlet.SolrServlet init INFO: SolrServlet.init() 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:54 org.apache.solr.servlet.SolrServlet init INFO: SolrServlet.init() done 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:54 org.apache.solr.servlet.SolrUpdateServlet init INFO: SolrUpdateServlet.init() done 2013-02-01 18:03:54.449:INFO::Started SocketConnector at 0.0.0.0:8983 01.02.2013 18:04:40 org.apache.solr.core.SolrDeletionPolicy onInit INFO: SolrDeletionPolicy.onInit: commits:num=1 commit{dir=/home/valerius/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1,version=1359741540635,generation=1,filenames=[segments_1] 01.02.2013 18:04:40 org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: newest commit = 1359741540635 01.02.2013 18:05:05 org.apache.solr.update.processor.LogUpdateProcessor finish INFO: {add=[1/5cc6513b4dc6de5093660000b304ded5/valerius at test.afterlogic.com, 48/5cc6513b4dc6de5093660000b304ded5/valerius at test.afterlogic.com]} 0 24629 01.02.2013 18:05:05 org.apache.solr.common.SolrException log SEVERE: java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.index.FreqProxTermsWriterPerFieldSFreqProxPostingsArray.(FreqProxTermsWriterPerField.java:193) at org.apache.lucene.index.FreqProxTermsWriterPerFieldSFreqProxPostingsArray.newInstance(FreqProxTermsWriterPerField.java:204) at org.apache.lucene.index.ParallelPostingsArray.grow(ParallelPostingsArray.java:48) at org.apache.lucene.index.TermsHashPerField.growParallelPostingsArray(TermsHashPerField.java:157) at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:460) at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:189) at org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocFieldProcessorPerThread.java:278) at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:766) at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:2328) at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:2300) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:240) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:157) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.mortbay.jetty.servlet.ServletHandlerSCachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnectionSRequestHandler.content(HttpConnection.java:945) 2013-02-01 18:05:42.595:INFO::Shutdown hook executing 2013-02-01 18:05:42.596:INFO::Graceful shutdown SocketConnector at 0.0.0.0:8983 2013-02-01 18:05:42.596:INFO::Graceful shutdown org.mortbay.jetty.webapp.WebAppContext at 5484ff20{/solr,file:/home/valerius/apache-solr-3.6.2/example/webapps/solr.war} 2013-02-01 18:05:43.596:INFO::Stopped SocketConnector at 0.0.0.0:8983 2013-02-01 18:05:43.784:INFO::Shutdown hook complete ========================================================== So, it seems that Dovecot tries to parse messages in the mailbox, and can't correctly determine where the message header ends. So, it thinks that the message header is big, and passes very big data to Solr. When trying to index it, Solr exhausts the available memory (though, I have 8 Gb of RAM on my machine, and java eats more than 2 Gb when indexing). Then connections to Solr get closed, and maxfd is invalid, hence the assertion is failed. Note also the following error ========================================================== SEVERE: org.apache.solr.common.SolrException: undefined field text ========================================================== before an out of memory error. I also tried to tweak the decode2text.sh script to ignore all attachments bigger than 1 Mb (just test if the file is bigger than 1 Mb, and if so, return "1"). This won't help. As I understood, this is because of big header, so attachments doesn't matter. I separated the set of messages which cause this error (by their UID's). So, I can give them as a testcase, the size of them all in archive is about 40 Mb. The error can be reproduced if put all these messages into an empty mailbox, and do reindexing, via IMAP search, or via "doveadm index -u ". I use Dovecot version 2.1.13. My doveconf -n : ========================================================== # 2.1.13: /usr/mailsuite/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_debug_passwords = yes auth_default_realm = test.afterlogic.com auth_mechanisms = plain login auth_socket_path = /usr/mailsuite/var/run/dovecot/auth-master base_dir = /usr/mailsuite/var/run/dovecot/ debug_log_path = /usr/mailsuite/var/log/dovecot/debug.log default_internal_user = exim dict { domainquotadict = mysql:/usr/mailsuite/etc/dovecot/dovecot-domain-quota-dict.conf realmquotadict = mysql:/usr/mailsuite/etc/dovecot/dovecot-realm-quota-dict.conf userquotadict = mysql:/usr/mailsuite/etc/dovecot/dovecot-user-quota-dict.conf } disable_plaintext_auth = no info_log_path = /usr/mailsuite/var/log/dovecot/info.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /usr/mailsuite/var/log/dovecot/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_gid = exim mail_home = /usr/mailsuite/data/settings/%d/%n mail_location = maildir:/usr/mailsuite/data/%d/%n mail_plugins = listescape fts fts_solr virtual mail_uid = exim managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave spamtest spamtestplus mbox_write_locks = fcntl namespace { inbox = yes location = maildir:/usr/mailsuite/data/%d/%n prefix = separator = / type = private } namespace { hidden = yes inbox = no list = no location = virtual:/usr/mailsuite/data/virtual/:LAYOUT=maildir++:INDEX=/usr/mailsuite/data/indexes/ prefix = All/ separator = / type = private } passdb { args = /usr/mailsuite/etc/dovecot/dovecot-sql.conf driver = sql } passdb { args = /usr/mailsuite/etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_allow_append_to_spam = no antispam_backend = mailtrain antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/mailsuite/scripts/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = error antispam_spam = SPAM antispam_spam_pattern = SPAM antispam_spam_pattern_ignorecase = SPAM antispam_trash = trash;Trash;Deleted Items; Deleted Messages antispam_trash_pattern = trash;Trash;Deleted * antispam_trash_pattern_ignorecase = trash;Deleted * antispam_unsure = antispam_unsure_pattern = antispam_unsure_pattern_ignorecase = fts = solr fts_decoder = decode2text fts_solr = break-imap-search url=http://localhost:8983/solr/ quota = dict:User quota::proxy::userquotadict quota2 = dict::%d:proxy::domainquotadict quota_rule = *:storage=100G quota_rule2 = Trash:storage=+20%% quota_rule3 = Spam:ignore quota_warning = storage=99%% quota-warning user 99 %d %n quota_warning2 = storage=95%% quota-warning user 95 %d %n quota_warning3 = storage=80%% quota-warning user 80 %d %n sieve = /usr/mailsuite/data/%d/%n/sieve.sieve sieve_before = /usr/mailsuite/etc/sieve/system.sieve sieve_dir = /usr/mailsuite/data/%d/%n/filters/ sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric sieve_global_dir = /usr/mailsuite/etc/sieve/ } protocols = imap pop3 lmtp sieve sendmail_path = /usr/mailsuite/bin/sendmail service auth { unix_listener auth-client { group = exim mode = 0666 user = exim } unix_listener auth-master { group = exim mode = 0666 user = exim } user = Sdefault_internal_user } service decode2text { executable = script /usr/mailsuite/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 } user = exim } service dict { unix_listener dict { group = exim mode = 0600 user = exim } } service imap-login { inet_listener imap { port = 143 } } service imap-postlogin { executable = script-login /usr/mailsuite/scripts/dovecot-postlogin.sh user = root } service imap { executable = imap imap-postlogin } service indexer-worker { user = root vsz_limit = 768 M } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } service_count = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/mailsuite/scripts/send_mail_warnquota.sh unix_listener quota-warning { user = exim } user = exim } ssl_cert = From gsansone at eumus.edu.uy Tue Feb 5 16:10:44 2013 From: gsansone at eumus.edu.uy (Gustavo Sansone) Date: Tue, 05 Feb 2013 12:10:44 -0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <1360030140.10326.54.camel@innu> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> <1360030140.10326.54.camel@innu> Message-ID: <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> El 2013-02-05 0:09, Timo Sirainen escribi?: > On Tue, 2013-01-29 at 16:29 -0200, Gustavo Sansone wrote: >> hello everyone, i'm new to this list >> im running dovecot as imap+pop (2.1.12) and roundcube (0.8.4) as >> client. >> >> having problems on saving to imap folders, with errors on some >> accounts >> >> my folder structure is as follow: >> >> /home/user/.maildir >> /home/user/.maildir/.Drafts >> /home/user/.maildir/.Junk >> /home/user/.maildir/.Sent >> /home/user/.maildir/.Trash >> /home/user/.maildir/.Archives >> /home/user/.maildir/.Archives.2010 >> ... >> >> i'm very confused with the namespace settings. i don't understand >> what >> is the correct way to configure it. >> don't need public or shared folders. > > Easiest to use the default namespace settings. There you wouldn't > have > any namespace prefix. if i just remove the namespace 'INBOX.' , on 10-mail.conf then there is something that creates new Send and Draft folders into my maildir folder like this INBOX.Send and INBOX.Drafts. and Roundcube shows me these new folders under INBOX (screenshot attached). can't figure out why it happens. >> on dovecot.log i can see this error with my account (and all the >> others), but it seems to work ok using roundcube: >> imap(gsansone): Error: autocreate: No namespace found for mailbox: >> Junk > > In your current setup you have namespace prefix=INBOX. so the mailbox > name would be INBOX.Junk. > >> with a few accounts, i have this on roundcube log: >> IMAP Error: Could not save message in Drafts in >> /var/www/localhost/htdocs/mail/program/steps/mail/sendmail.inc >> with error on saving to imap folder in roundcube UI > > Roundcube also isn't configured to have INBOX. prefix. Sounds like > you > should just remove it. -------------- next part -------------- A non-text attachment was scrubbed... Name: maildir.png Type: image/png Size: 49918 bytes Desc: not available URL: From tss at iki.fi Tue Feb 5 16:54:21 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 5 Feb 2013 16:54:21 +0200 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: <3E83F6B6-0DDA-4B7B-B796-00E831F39784@iki.fi> On 5.2.2013, at 11.57, Jan-Frode Myklebust wrote: > I think there must be some bug I'm hitting here. One of my directors > is still running with "client_limit = 1, process_limit = 100" for the > lmtp service, and now it's logging: > > master: Warning: service(lmtp): process_limit (100) reached, client > connections are being dropped > > Checking "sudo netstat -anp|grep ":24 " I see 287 ports in TIME_WAIT, > one in CLOSE_WAIT and the listening "0.0.0.0:24". No active > connections. There are 100 lmtp-processes running. Sounds like the LMTP processes are hanging for some reason.. http://hg.dovecot.org/dovecot-2.1/rev/63117ab893dc might show something interesting, although I'm pretty sure it will just say that the processes are hanging in DATA command. Other interesting things to check: gdb -p bt full strace -tt -p (for a few seconds to see if anything is happening) If lmtp proxy is hanging, it should have a timeout (default 30 secs) and it should log about it if it triggers. (Although maybe not to error log.) > When trying to > connect to the lmtp-port I immediately get dropped: > > $ telnet localhost 24 > Trying 127.0.0.1... > Connected to localhost.localdomain (127.0.0.1). > Escape character is '^]'. > Connection closed by foreign host. This happens when the master process notices that all the service processes are full. > Is there maybe some counter that's getting out of sync, or some back > off penalty algorithm that kicks in when it first hit the process > limit ? Shouldn't be, but the proctitle patch should make it clearer. Strange anyway, I haven't heard of anything like this happening before. From tss at iki.fi Tue Feb 5 17:46:43 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 5 Feb 2013 17:46:43 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <5110CBDF.4030302@skye.it> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <5110CBDF.4030302@skye.it> Message-ID: <5C275EA9-B689-4845-AE30-CF6880FFBB8E@iki.fi> On 5.2.2013, at 11.07, Alessio Cecchi wrote: >>>>> But does it fix the performance? .. >>>> >>>> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f > > Also after the application of these patches the situation of the load is the same. > > If there are no other solutions I will return to 2.1.13 ASAP. You definitely included the above patch? That's the important one. And it really should have fixed the performance problems.. From alessio at skye.it Tue Feb 5 18:02:15 2013 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 05 Feb 2013 17:02:15 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <5C275EA9-B689-4845-AE30-CF6880FFBB8E@iki.fi> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <5110CBDF.4030302@skye.it> <5C275EA9-B689-4845-AE30-CF6880FFBB8E@iki.fi> Message-ID: <51112D07.9090506@skye.it> Il 05/02/2013 16:46, Timo Sirainen ha scritto: > On 5.2.2013, at 11.07, Alessio Cecchi wrote: > >>>>>> But does it fix the performance? .. >>>>> >>>>> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f >> >> Also after the application of these patches the situation of the load is the same. >> >> If there are no other solutions I will return to 2.1.13 ASAP. > > You definitely included the above patch? That's the important one. And it really should have fixed the performance problems.. > Sure Timo, during in the afternoon the situations seems better, wait a few days for tests. Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From arnaud.abelard at univ-nantes.fr Tue Feb 5 19:33:16 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 18:33:16 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 Message-ID: <5111425C.4000902@univ-nantes.fr> Hello, I've upgraded our servers and now dovecot seems to be running out of memory when accessing my own mailbox (and only mine, which is in a way pretty fortunate): dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is 72% full of 8GB. any idea of what is happening? Thanks in advance, Arnaud -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From arnaud.abelard at univ-nantes.fr Tue Feb 5 19:38:51 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 18:38:51 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <5111425C.4000902@univ-nantes.fr> References: <5111425C.4000902@univ-nantes.fr> Message-ID: <511143AB.3060501@univ-nantes.fr> Hrm... exactly when I pressed sent, dovecot started giving me more information: master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it)) Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were running...). Arnaud On 02/05/2013 06:33 PM, Arnaud Ab?lard wrote: > Hello, > > I've upgraded our servers and now dovecot seems to be running out of > memory when accessing my own mailbox (and only mine, which is in a way > pretty fortunate): > > dovecot: imap(abelard-a): Error: mmap() failed with file > /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory > > imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory > > /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is > 72% full of 8GB. > > any idea of what is happening? > > Thanks in advance, > > Arnaud > > -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From h.reindl at thelounge.net Tue Feb 5 19:44:25 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 05 Feb 2013 18:44:25 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <511143AB.3060501@univ-nantes.fr> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> Message-ID: <511144F9.30408@thelounge.net> hard to say without config-details shared prcoess or one per connection? in case of shared 256 MB is really really low Am 05.02.2013 18:38, schrieb Arnaud Ab?lard: > Hrm... exactly when I pressed sent, dovecot started giving me more information: > > master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you > may need to increase it)) > > Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How > come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were > running...). > > Arnaud > > On 02/05/2013 06:33 PM, Arnaud Ab?lard wrote: >> Hello, >> >> I've upgraded our servers and now dovecot seems to be running out of >> memory when accessing my own mailbox (and only mine, which is in a way >> pretty fortunate): >> >> dovecot: imap(abelard-a): Error: mmap() failed with file >> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >> >> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory >> >> /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is >> 72% full of 8GB. >> >> any idea of what is happening? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From arnaud.abelard at univ-nantes.fr Tue Feb 5 20:15:28 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 19:15:28 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <511144F9.30408@thelounge.net> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> <511144F9.30408@thelounge.net> Message-ID: <51114C40.3050801@univ-nantes.fr> On 02/05/2013 06:44 PM, Reindl Harald wrote: > hard to say without config-details > > shared prcoess or one per connection? One per connection, around 1000 connections right now. > in case of shared 256 MB is really really low I changed the vsz_limit to 512MB and it seems better, but I'm still surprised my mailbox actually hit the memory limit since I doubt it's most used one. Arnaud > > Am 05.02.2013 18:38, schrieb Arnaud Ab?lard: >> Hrm... exactly when I pressed sent, dovecot started giving me more information: >> >> master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you >> may need to increase it)) >> >> Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How >> come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were >> running...). >> >> Arnaud >> >> On 02/05/2013 06:33 PM, Arnaud Ab?lard wrote: >>> Hello, >>> >>> I've upgraded our servers and now dovecot seems to be running out of >>> memory when accessing my own mailbox (and only mine, which is in a way >>> pretty fortunate): >>> >>> dovecot: imap(abelard-a): Error: mmap() failed with file >>> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >>> >>> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory >>> >>> /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is >>> 72% full of 8GB. >>> >>> any idea of what is happening? > -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From dz at ct.heise.de Tue Feb 5 20:36:29 2013 From: dz at ct.heise.de (Dusan Zivadinovic) Date: Tue, 5 Feb 2013 19:36:29 +0100 Subject: [Dovecot] dsync: Invalid server handshake Message-ID: <36C41C6B-F529-4787-A12E-C55975B44629@ct.heise.de> Hi list, I recently tried to backup mailboxes from an older server machine to a new one in order to move the service to the new machine. Both machines are in the same LAN, I used this command: dsync -R -u username backup ssh -i .ssh/id_rsa username at 192.168.1.11 /opt/local/bin/dsync and I get this error: dsync-local(dz): Error: Invalid server handshake: dsync-server 2 dsync-remote(dz): Error: Invalid client handshake: dsync-client 1 # lokal machine: dovecot 2.0.19apple1 running on Mac OS X 10.8.2 Server using standard Apple-Dovecot-Configuration # remote machine: 2.1.12 running on Mac OS X 10.4 using PAM authentication I cant seem to find any documentation to this error, nor do I find any dsync entries in the logs of both machines. Does anyone have a hint? Thank you, Dusan From radikn at seznam.cz Tue Feb 5 21:52:15 2013 From: radikn at seznam.cz (=?UTF-8?Q?Radek_Novotn=C3=BD?=) Date: Tue, 05 Feb 2013 20:52:15 +0100 Subject: [Dovecot] Per user special-use folder names Message-ID: Hi all, let me ask a question, please. Is it possible in dovecot to set up per user special-use folder names? Imagine situation with two users where first prefere another language that second. mailbox Sent { special_use = \Sent } for english speaking users and mailbox "Odeslan? po?ta" { special_use = \Sent } for czech speaking users. Thanks for your answers. Radek From dave at dawoodfall.net Tue Feb 5 22:14:51 2013 From: dave at dawoodfall.net (David Woodfall) Date: Tue, 5 Feb 2013 20:14:51 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <20130204105653.GA21474@nihlus.leuxner.net> References: <20130204020105.GA15960@Blackswan> <20130204020824.GA14425@daniel.localdomain> <510F1AB5.6080503@dawoodfall.net> <20130204105653.GA21474@nihlus.leuxner.net> Message-ID: <20130205201451.GA26648@Blackswan> On Mon, Feb 04, 2013 at 11:56:53AM +0100, Thomas Leuxner put forth the proposition: >* David Woodfall 2013.02.04 03:19: > >> >It looks as if folder_format option of mutt >> >as only for local folders, and will not work for IMAP. >> >> I was grasping at straws and hoping someone here may know. > >As mentioned this may be one for the mutt list. But it should work with IMAP: > >set imap_user="user at domain.tld" >set folder="imap://host.domain.tld/" >set spoolfile="imap://host.domain.tld/INBOX" >set index_format="%4C %Z %2M %[!%Y.%m.%d %H:%M] %-30.30F (%5c) %s" After much messing about with mutt and suggestions on the list, it seems that when mutt displays the imap mailbox list it works, but the normal folder browser doesn't work with %N. Thanks for you help From stan at hardwarefreak.com Tue Feb 5 22:25:35 2013 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 05 Feb 2013 14:25:35 -0600 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <51114C40.3050801@univ-nantes.fr> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> <511144F9.30408@thelounge.net> <51114C40.3050801@univ-nantes.fr> Message-ID: <51116ABF.2080204@hardwarefreak.com> On 2/5/2013 12:15 PM, Arnaud Ab?lard wrote: > I changed the vsz_limit to 512MB and it seems better, but I'm still > surprised my mailbox actually hit the memory limit since I doubt it's > most used one. According to the wiki, vsz_limit only affects login processes, not IMAP processes, which is odd given the second error you posted, which seems to indicate a relationship between vsz_limit and imap. >>>> dovecot: imap(abelard-a): Error: mmap() failed with file >>>> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >>>> >>>> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory This error suggests your mail_process_size variable is set too low. If it is currently set to 128MB, increase it to 512MB. The bulk of this memory is used for mmap()ing files, and is virtual, not physical. Thus you don't actually need 51GB of RAM to support 100 users. Also, a dovecot.index.cache file of size 224MB seems rather large. My largest list mail folder is 150MB, contains 17,647 msgs, and has a dovecot.index.cache file of only 16MB. I use mbox. This would seem to suggest you may have a single folder with tens of thousands of msgs in it. Or maybe indexes are handled differently for your mailbox format--I've not used any others. If the former, I suggest you cull your large folders to get them down to manageable size. -- Stan From p at sys4.de Tue Feb 5 22:28:24 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Tue, 5 Feb 2013 21:28:24 +0100 Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: <20130205202824.GA23270@sys4.de> * Radek Novotn? : > Hi all, > > let me ask a question, please. Is it possible in dovecot to set up > per user special-use folder names? > > Imagine situation with two users where first prefere another > language that second. You don't need per-user folder SPECIAL-USE names, because the client must take care of the correct mapping. If the client runs in an German environment it might mount the "special_use = \Sent" mailbox as "Gesendete Objekte" and if it is Czechian it might call it "Odeslan? po?ta". That's part of what makes SPECIAL-USE so sexy. It is language independent. All it does is say "This mailbox is reserved for that particular usage." How you call it, is up to you (client). p at rick > > mailbox Sent { > special_use = \Sent > } > > for english speaking users and > > mailbox "Odeslan? po?ta" { > special_use = \Sent > } > > for czech speaking users. > > > > Thanks for your answers. Radek -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From slusarz at curecanti.org Tue Feb 5 22:50:12 2013 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 05 Feb 2013 13:50:12 -0700 Subject: [Dovecot] Per user special-use folder names In-Reply-To: <20130205202824.GA23270@sys4.de> References: <20130205202824.GA23270@sys4.de> Message-ID: <20130205135012.Horde.mB4kE_oo2esQ-uGfpIN3rg2@bigworm.curecanti.org> Quoting Patrick Ben Koetter

: > That's part of what makes SPECIAL-USE so sexy. It is language independent. > All it does is say "This mailbox is reserved for that particular usage." How > you call it, is up to you (client). Well.... not quite. The problem comes when you have *multiple* sent mailboxes on your server, which is perfectly acceptable and quite useful (e.g. an MUA allows multiple identities, and each identity uses a separate sent-mail mailbox). You can't just blindly show the local translation for "Sent" for all of the mailboxes, or else you've now eliminated the user's ability to differentiate between them. In practical use, SPECIAL-USE isn't tremendously helpful for "auto-configuration" of an MUA because of these kind of vagaries. michael From arnaud.abelard at univ-nantes.fr Tue Feb 5 23:16:53 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 22:16:53 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <51116ABF.2080204@hardwarefreak.com> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> <511144F9.30408@thelounge.net> <51114C40.3050801@univ-nantes.fr> <51116ABF.2080204@hardwarefreak.com> Message-ID: <511176C5.8010501@univ-nantes.fr> On 02/05/2013 09:25 PM, Stan Hoeppner wrote: > On 2/5/2013 12:15 PM, Arnaud Ab?lard wrote: > >> I changed the vsz_limit to 512MB and it seems better, but I'm still >> surprised my mailbox actually hit the memory limit since I doubt it's >> most used one. > > According to the wiki, vsz_limit only affects login processes, not IMAP > processes, which is odd given the second error you posted, which seems > to indicate a relationship between vsz_limit and imap. Actually, service { vsz_limit } replaced mail_process_size (at least according to what dovecot said upon restart...) so service imap { vsz_limit=512MB } which i added earlier actually does what you suggested. > >>>>> dovecot: imap(abelard-a): Error: mmap() failed with file >>>>> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >>>>> >>>>> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory > > This error suggests your mail_process_size variable is set too low. If > it is currently set to 128MB, increase it to 512MB. The bulk of this > memory is used for mmap()ing files, and is virtual, not physical. Thus > you don't actually need 51GB of RAM to support 100 users. > > Also, a dovecot.index.cache file of size 224MB seems rather large. My > largest list mail folder is 150MB, contains 17,647 msgs, and has a > dovecot.index.cache file of only 16MB. I use mbox. This would seem to > suggest you may have a single folder with tens of thousands of msgs in > it. Or maybe indexes are handled differently for your mailbox > format--I've not used any others. If the former, I suggest you cull > your large folders to get them down to manageable size. Yep I have a huge INBOX folder right now, I haven't archived my 2011 mail yet (I keep the previous year in my INBOX). Thanks, Arnaud > -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From forall at stalowka.info Tue Feb 5 23:49:18 2013 From: forall at stalowka.info (For@ll) Date: Tue, 05 Feb 2013 22:49:18 +0100 Subject: [Dovecot] Dovecot main process killed by KILL signal Message-ID: Hi, I have a dovecot 2.0.7 installed on ubuntu 12.10 and often in dmesg I see "init: dovecot main process (7104) killed by KILL signal", I must restarted dovecot, because I can't access to mailbox. Server hardware: 2x1TB disks 4GB memory 1xCPU 2,4GHz HARDWARE RAID 1 Mailbox ~ 1k dovecot.conf # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-22-generic x86_64 Ubuntu 12.10 xfs auth_mechanisms = plain login auth_verbose = yes default_client_limit = 1500 info_log_path = /var/log/dovecot.info log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/mail/virtual/%d/%n/ maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } passdb { driver = pam } plugin { sieve_dir = /var/mail/virtual/%d/%n/sieve sieve_global_dir = /var/mail/virtual/sieve } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = root } service imap-login { inet_listener imap { port = 0 } process_limit = 512 } service imap { process_limit = 512 service_count = 0 } service managesieve-login { inet_listener sieve { port = 33919 } } service pop3-login { inet_listener pop3 { port = 0 } } ssl_cert = References: Message-ID: W dniu 2013-02-05 22:49, For at ll pisze: > Hi, > > I have a dovecot 2.0.7 installed on ubuntu 12.10 and often in dmesg I > see "init: dovecot main process (7104) killed by KILL signal", I must > restarted dovecot, because I can't access to mailbox. Sorry, I have dovecot 2.1.7. From slusarz at curecanti.org Wed Feb 6 01:02:38 2013 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 05 Feb 2013 16:02:38 -0700 Subject: [Dovecot] Possible sort optimization (?) Message-ID: <20130205160238.Horde.lxh7MCMj2ohf5zU-z9VJug6@bigworm.curecanti.org> Maybe this is just noise... but I can reproduce this fairly reliably. Mailbox with 21,000+ messages This query: a UID SORT RETURN (ALL COUNT) (DATE) UTF-8 SUBJECT "foo" is always about 10 percent slower than this split query (I've done this 4-5 times, and the numbers are similar): a UID SEARCH RETURN (SAVE) CHARSET UTF-8 SUBJECT "foo" b UID SORT RETURN (ALL COUNT) (DATE) UTF-8 UID $ (The particular query I used matched 5 messages out of the 21,000+) My not-very-scientific benchmarking process: 1.) Stop dovecot process 2.) Delete all dovecot index files for that mailbox 3.) Flush linux paging cache (sync && echo 3 > /proc/sys/vm/drop_caches) 4.) Restart dovecot 5.) Access dovecot via command-line (PREAUTH) 6.) SELECT mailbox 7.) Issue command(s) Could be a potential area for performance improvement or could simply be lazy benchmarking. michael From joe.beaubien at gmail.com Wed Feb 6 05:41:59 2013 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Tue, 5 Feb 2013 22:41:59 -0500 Subject: [Dovecot] dsync backup questions In-Reply-To: <1360029680.10326.48.camel@innu> References: <20130204005729.GA68109@anubis.morrow.me.uk> <1360029680.10326.48.camel@innu> Message-ID: On Mon, Feb 4, 2013 at 9:01 PM, Timo Sirainen wrote: > On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote: > > I can't give authoratitive answers to either of these, but... > > > > At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: > > > > > > I'm currently trying to setup remote backups of my emails but i'm > running > > > into issues (mdbox format, indexes and storage in the same folder > > > hierarchy). > > > > > > Local backup command: dsync -u "my_user" backup /backups/my_user > > > > > > (1) Recently, I noticed that the local backup takes up twice the size > as > > > the original mail location (8gb vs. 4gb). I purged alot of emails from > the > > > original location, so the size shrunk, but the local backup just keeps > on > > > getting bigger. I couldn't find any dsync option that would delete > extra > > > emails. > > > > > > - Question: Why isn't the local backup synced properly and remove the > extra > > > emails? > > > > Are you running 'doveadm purge' on the backed-up dbox? It looks to me as > > though dsync doesn't do that. I don't know if there's any (simple) way > > to do that without a running Dovecot instance attached the dbox > > directory: it's not entirely clear to me whether doveadm will run > > locally without contacting a doveadm-server instance running under > > Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. > > Right. doveadm -o mail=mdbox:/backups/my_user purge > This worked (at least it seems to, the source and destination are roughly the same size).. However, if the the original email location has already been purge, does the backup email location also need to be purged? > > It might be easier to dsync to a Maildir instead. This should preserve > > all the Dovecot-visible metadata, and dsyncing back to the dbox for > > restore should put it all back. > > Better sdbox than maildir. > I'd rather stick to mdbox for my remote backups. I have a single email account with over 1.5 million emails in it. With a 1-file-per-message storage, this would be slow/hell to run. Unless there is a better way. > > > > (2) What is the best why to copy this local backup to a remote location > > > that does NOT have the possibility to run dsync. > > > > > > - Question 1: is rsync safe to use and will this data work for restore? > > > > > > - Question 2: Would it be safe to simply rsync the original > mail_location > > > to the remote server? > > > > AFAICT, if Dovecot is stopped on both sides of the transfer it should be > > safe. If either side has a currently running Dovecot instance (or any > > other Dovecot tools, like a concurrent dsync) using that dbox, it's > > likely rsync will copy an inconsistent snapshot of the data, and the > > result will be corrupted. > > It won't be badly corrupted though. At worst Dovecot will rebuild the > indexes, which takes a while. And most users probably won't get any > corruption at all. > > > I think there was a misunderstanding of the setup. dsync is only running on the local side, the remote side is a "dumb" rsync server that I don't fully control. From joe.beaubien at gmail.com Wed Feb 6 06:00:59 2013 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Tue, 5 Feb 2013 23:00:59 -0500 Subject: [Dovecot] dsync backup questions In-Reply-To: References: <20130204005729.GA68109@anubis.morrow.me.uk> <1360029680.10326.48.camel@innu> Message-ID: On Tue, Feb 5, 2013 at 10:41 PM, Joe Beaubien wrote: > > > > On Mon, Feb 4, 2013 at 9:01 PM, Timo Sirainen wrote: > >> On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote: >> > I can't give authoratitive answers to either of these, but... >> > >> > At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: >> > > >> > > I'm currently trying to setup remote backups of my emails but i'm >> running >> > > into issues (mdbox format, indexes and storage in the same folder >> > > hierarchy). >> > > >> > > Local backup command: dsync -u "my_user" backup /backups/my_user >> > > >> > > (1) Recently, I noticed that the local backup takes up twice the size >> as >> > > the original mail location (8gb vs. 4gb). I purged alot of emails >> from the >> > > original location, so the size shrunk, but the local backup just >> keeps on >> > > getting bigger. I couldn't find any dsync option that would delete >> extra >> > > emails. >> > > >> > > - Question: Why isn't the local backup synced properly and remove the >> extra >> > > emails? >> > >> > Are you running 'doveadm purge' on the backed-up dbox? It looks to me as >> > though dsync doesn't do that. I don't know if there's any (simple) way >> > to do that without a running Dovecot instance attached the dbox >> > directory: it's not entirely clear to me whether doveadm will run >> > locally without contacting a doveadm-server instance running under >> > Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. >> >> Right. doveadm -o mail=mdbox:/backups/my_user purge >> > > This worked (at least it seems to, the source and destination are roughly > the same size).. > > However, if the the original email location has already been purge, does > the backup email location also need to be purged? > > >> > It might be easier to dsync to a Maildir instead. This should preserve >> > all the Dovecot-visible metadata, and dsyncing back to the dbox for >> > restore should put it all back. >> >> Better sdbox than maildir. >> > > I'd rather stick to mdbox for my remote backups. I have a single email > account with over 1.5 million emails in it. With a 1-file-per-message > storage, this would be slow/hell to run. Unless there is a better way. > > >> >> > > (2) What is the best why to copy this local backup to a remote >> location >> > > that does NOT have the possibility to run dsync. >> > > >> > > - Question 1: is rsync safe to use and will this data work for >> restore? >> > > >> > > - Question 2: Would it be safe to simply rsync the original >> mail_location >> > > to the remote server? >> > >> > AFAICT, if Dovecot is stopped on both sides of the transfer it should be >> > safe. If either side has a currently running Dovecot instance (or any >> > other Dovecot tools, like a concurrent dsync) using that dbox, it's >> > likely rsync will copy an inconsistent snapshot of the data, and the >> > result will be corrupted. >> >> It won't be badly corrupted though. At worst Dovecot will rebuild the >> indexes, which takes a while. And most users probably won't get any >> corruption at all. >> >> >> > I think there was a misunderstanding of the setup. dsync is only running > on the local side, the remote side is a "dumb" rsync server that I don't > fully control. > > > what i was trying to ask with my last question is the following: I'm trying to do remote backups of an mdbox setup, and considering that I only have dsync running on the local side (not on the remote side), is it safe to simply do an rsync of the mail_location to the remote server, or should I first do a dsync (make a local duplicate) and then rsync the duplcate out to the remote server? (wish i had a whiteboard right about now). Thanks, -Joe From jtam.home at gmail.com Wed Feb 6 06:51:44 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 5 Feb 2013 20:51:44 -0800 (PST) Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On Wed, 6 Feb 2013, Michael M Slusarz wrote: > Quoting Patrick Ben Koetter

: > >> That's part of what makes SPECIAL-USE so sexy. It is language independent. >> All it does is say "This mailbox is reserved for that particular usage." How >> you call it, is up to you (client). > > Well.... not quite. The problem comes when you have *multiple* sent > mailboxes on your server, which is perfectly acceptable and quite > useful (e.g. an MUA allows multiple identities, and each identity uses > a separate sent-mail mailbox). You can't just blindly show the local > translation for "Sent" for all of the mailboxes, or else you've now > eliminated the user's ability to differentiate between them. On a related topic, what's the easiest way to alias various common mailbox names to one physical mailbox? For example, mapping "Trash", "Deleted Messages", "Junk" to the same mailbox? Would you use the SPECIAL-USE, or is there a better way to do this? Namescape configuration? Virtual plugin? Joseph Tam From ya.mwork at yandex.ru Wed Feb 6 08:09:12 2013 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Wed, 06 Feb 2013 10:09:12 +0400 Subject: [Dovecot] dovecot-2.2: dsync to imapc not working Message-ID: <5111F388.9030305@yandex.ru> Hello. I have dovecot installation # dovecot --version 20130205 (03a0af22100d+) built with imapc backend. I'm tried to sync mailboxes from another server after clean mail directory localy: # dsync -v -o imapc_user="user at example.org" -o imapc_password="pass" -o imapc_host=imap.example.org -o imapc_features=rfc822.size -o mailbox_list_index=no backup -R -f -u "user at example.org" imapc: and get this message dsync(user at example.org): Error: Exporting mailbox INBOX failed: Backend doesn't support GUIDs, sync with header hashes instead Repeated command returns this one message: dsync(user at example.org): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. What wrong with this build? Maybe there is another way to do it? For example create backup in local temporary directory and synchronize this one and working storage. From skdovecot at smail.inf.fh-brs.de Wed Feb 6 10:10:47 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 Feb 2013 09:10:47 +0100 (CET) Subject: [Dovecot] Dovecot main process killed by KILL signal In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 5 Feb 2013, For at ll wrote: > I have a dovecot 2.0.7 installed on ubuntu 12.10 and often in dmesg I see > "init: dovecot main process (7104) killed by KILL signal", I must restarted Your Dovecot gets killed with a sure kill (aka kill -9). Somebody or some process wants your Dovecot stopped for sure. Do you have something in your syslog and/or Dovecot log? Does the message comes up, _because_ you stop or restart Dovecot? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURIQB13r2wJMiz2NAQIRGAf/ei4soM0f4wn7UFb0tsQsTJUJ8YfD1cAN OcSviw9BfmN7RiqUL8H0sICRINHWBnz8oRmHf/2zGGWZGQTOlbVNoUefZ423I5nH HaE73A1vhbi+wmKh5DvStn6NB0WxVs6U8LFXPzryzV1haeS3pM2KOgf1YVRaagfN VVcgz1in81NUhtqdmVTEzyauI0mbquCclTuQ9EmynYlncc9T48UIt0Q7ApMiA3Kb VZ/ms5xLxdTbD9gS+SAbWfALA15kU9+EMgdwMqGvPWTlSc9CD/I0tzyHYkmFbSVJ fdru3g1RTZGUd4yJwOcrevoo1j7CW8/+fuEPrCVU7D4WRmRujTDhig== =dayi -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Wed Feb 6 12:10:25 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 Feb 2013 11:10:25 +0100 (CET) Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510FF702.4030208@mclien.de> References: <510BEBDC.6020802@mclien.de> <510FF702.4030208@mclien.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 4 Feb 2013, Frank wrote: > Steffen Kaiser wrote: >> On Fri, 1 Feb 2013, Frank Lienhard wrote: >> >>> My first attempt was tho simply rsync the Maildirs along with the homes. >>> Both deovecot versions are set up with the maildir format, but this >>> results in ~30% doublicated mails on the new server. >> >> Er, a) you got 30% more files on the new server (aka target), than you have >> on the old server (aka source)?? >> >> b) Or do you see them, when you login via IMAP to the the new server? >> >> === >> >> If a), did you ran rsync, while the source host is in production still? Do >> you use "H" option to preserve hardlinks and/or "--delete" if you run rsync >> multiple times? >> >> If b), did you verified the duplication with a IMAP client, that does _no_ >> caching on its own or with a fresh and empty local cache / profile / ... . >> > I used -avu, where: > -a equals: -rlptgoD (no -H,-A,-X) > -u skip file that are newer on the receiver > > I stopped the getmail cronjob to ensure that no new messages get fetched > while sync. After that I created a new account in Icedove and at first all > went well, but after a while I got duoplicates of abou 900 old messages. > I admit I only looked that up from Icedove and not in the dir from cli. When you rsync'ed multiple times, you _must_ use "--delete" as well, because in Maildir status changes, such as "read", "seen", "replied", lables a.s.o., are reflected by changing the filename of the messages. Hence, without "--delete" you get the same message multiple times with different status or labels. I tried it myself and had 1:1 copies of the mailboxes, regardless if I use the same or a fresh profile in Thunderbird. I use: rsync -essh -vaH --delete from/ target:/.../to/ - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURIsEV3r2wJMiz2NAQLNQwgAwRiTbnrPlgmKIYUMwDVWo5tztighb8+b WoYjr4XkK5SkxJH1ZUnsaNzoKYC0j/Zf+PE/kK9C32X+itkIhH8f3L8jITCzj3RR kl6/j/OE3hnMDJNSfoYabmdpzc9knXabj04GVbZNW0cwXX33eU3CQd9yHbno3Raq DVsnCIe23L7sO7giNORUIpZJuhzfri8YpvAdZsypaK0ZfMXbBxYN6lPaoEPkYcKH T91EvjbeGKJGpAKokjtyCIrBbvYPW6ARyD/y7G7qrGFvUqqZTzhEUbEPuZ6kCkFS Zc5bmv90BcQ/O1U6U2jH72A3UeCsCRBl9BWp9Cvs5P5Pbi1kyzvZrA== =GtYM -----END PGP SIGNATURE----- From h.reindl at thelounge.net Wed Feb 6 12:27:50 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 06 Feb 2013 11:27:50 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: References: <510BEBDC.6020802@mclien.de> <510FF702.4030208@mclien.de> Message-ID: <51123026.2000400@thelounge.net> Am 06.02.2013 11:10, schrieb Steffen Kaiser: > On Mon, 4 Feb 2013, Frank wrote: > >> Steffen Kaiser wrote: >>> On Fri, 1 Feb 2013, Frank Lienhard wrote: >>> >>>> My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with >>>> the maildir format, but this results in ~30% doublicated mails on the new server. >>> >>> Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)?? >>> >>> b) Or do you see them, when you login via IMAP to the the new server? >>> >>> === >>> >>> If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve >>> hardlinks and/or "--delete" if you run rsync multiple times? >>> >>> If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh >>> and empty local cache / profile / ... . >>> >> I used -avu, where: >> -a equals: -rlptgoD (no -H,-A,-X) >> -u skip file that are newer on the receiver > >> I stopped the getmail cronjob to ensure that no new messages get fetched while sync. After that I created a new >> account in Icedove and at first all went well, but after a while I got duoplicates of abou 900 old messages. >> I admit I only looked that up from Icedove and not in the dir from cli. > > When you rsync'ed multiple times, you _must_ use "--delete" as well, because in Maildir status changes, such as > "read", "seen", "replied", lables a.s.o., are reflected by changing the filename of the messages. Hence, without > "--delete" you get the same message multiple times with different status or labels. > > I tried it myself and had 1:1 copies of the mailboxes, regardless if I use the same or a fresh profile in Thunderbird. > > I use: rsync -essh -vaH --delete from/ target:/.../to/ generally you should use options to REALLY make a folder a 1:1 copy, the params below are used by me since many years for ANY sort of folder sync --delete-after -tPrlHpogEAX # -z compress # -t timestamps # -P progress # -r recursive # -l links # -H hard-links # -p permissions # -o owner # -g group # -E executability # -A acls # -X xtended attributes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From ya.mwork at yandex.ru Wed Feb 6 14:07:42 2013 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Wed, 06 Feb 2013 16:07:42 +0400 Subject: [Dovecot] dovecot-2.2: broken domain quota calculation Message-ID: <5112478E.2030003@yandex.ru> Hello. Used next version of dovecot: # dovecot --version 20130205 (03a0af22100d+) Quota configuration: plugin { quota = dict:Domain quota:%d:redis:host=127.0.0.1:prefix=domain/ quota_rule = *:storage=0 quota_rule2 = Trash:storage=+5%% quota2 = dict:User quota::redis:host=127.0.0.1:prefix=user/ quota2_rule = *:storage=0 quota2_rule2 = Trash:storage=+5%% } Problem: incorrect quota calculations. Dovecot gets size and messages count of all domain from last user in list. In 2.1 works fine. Steps to reproduce problem: 1. install dovecot 2. create over 50 users with e-mails 3. execute: doveadm quota recalc -A 4. execute: doveadm quota get -A 5. get count of all users message: sum in some application result of execution doveadm quota get -A | grep MESSAGE | grep 'User quota' | awk '{print $5}' (139443 messages) 6. get size of all users message: sum in some application result of execution doveadm quota get -A | grep STORAGE | grep 'User quota' | awk '{print $5}' (2304399 Kb) 7. look at some user user at example.org Domain quota STORAGE 82408 51200000 0 user at example.org Domain quota MESSAGE 1078 - 0 user at example.org User quota STORAGE 0 209715200 0 user at example.org User quota MESSAGE 0 - 0 and see 1078 messages and 82408 Kb in domain quota 8. look at last user in list z at example.org Domain quota STORAGE 82408 51200000 0 z at example.org Domain quota MESSAGE 1078 - 0 z at example.org User quota STORAGE 82408 209715200 0 z at example.org User quota MESSAGE 1078 - 0 and see matching values. From gsansone at eumus.edu.uy Wed Feb 6 15:19:56 2013 From: gsansone at eumus.edu.uy (Gustavo Sansone) Date: Wed, 06 Feb 2013 11:19:56 -0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> <1360030140.10326.54.camel@innu> <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> Message-ID: <565beb939e973e6fbf6057cafbc27daf@eumus.edu.uy> El 2013-02-05 12:10, Gustavo Sansone escribi?: > El 2013-02-05 0:09, Timo Sirainen escribi?: >> On Tue, 2013-01-29 at 16:29 -0200, Gustavo Sansone wrote: >>> hello everyone, i'm new to this list >>> im running dovecot as imap+pop (2.1.12) and roundcube (0.8.4) as >>> client. >>> >>> having problems on saving to imap folders, with errors on some >>> accounts >>> >>> my folder structure is as follow: >>> >>> /home/user/.maildir >>> /home/user/.maildir/.Drafts >>> /home/user/.maildir/.Junk >>> /home/user/.maildir/.Sent >>> /home/user/.maildir/.Trash >>> /home/user/.maildir/.Archives >>> /home/user/.maildir/.Archives.2010 >>> ... >>> >>> i'm very confused with the namespace settings. i don't understand >>> what >>> is the correct way to configure it. >>> don't need public or shared folders. >> >> Easiest to use the default namespace settings. There you wouldn't >> have >> any namespace prefix. > > if i just remove the namespace 'INBOX.' , on 10-mail.conf then there > is something that creates new Send and Draft folders into my maildir > folder like this INBOX.Send and INBOX.Drafts. and Roundcube shows me > these new folders under INBOX (screenshot attached). > can't figure out why it happens. > sorry, forgot to mention that when INBOX.Send and INBOX.Drafts are created, Roundcube subscribe to these folders as special folders. the screenshot was taken after i added INBOX. prefix again to 10-mail.conf another important thing (i guess), is when installed dovecot, i migrated from Courier following the wiki2 documentation. >>> on dovecot.log i can see this error with my account (and all the >>> others), but it seems to work ok using roundcube: >>> imap(gsansone): Error: autocreate: No namespace found for mailbox: >>> Junk >> >> In your current setup you have namespace prefix=INBOX. so the >> mailbox >> name would be INBOX.Junk. >> >>> with a few accounts, i have this on roundcube log: >>> IMAP Error: Could not save message in Drafts in >>> /var/www/localhost/htdocs/mail/program/steps/mail/sendmail.inc >>> with error on saving to imap folder in roundcube UI >> >> Roundcube also isn't configured to have INBOX. prefix. Sounds like >> you >> should just remove it. From aoster at novanetwork.de Wed Feb 6 15:26:39 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Wed, 06 Feb 2013 14:26:39 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <510F81F2.1020809@novanetwork.de> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> <510F81F2.1020809@novanetwork.de> Message-ID: <51125A0F.4040502@novanetwork.de> Am 04.02.2013 10:40, schrieb Andreas Oster: > Am 22.01.2013 12:54, schrieb Timo Sirainen: >> On 17.1.2013, at 16.58, Andreas Oster wrote: >> >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. >> >> It should work for all namespaces. >> >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota >> >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota >> >> That should work? Worked at least in latest v2.1 hg. >> >> > Hello Timo, > > can you please confirm, that it is actually possible to have multiple > quota roots with separate/different quota values. > > Currently I have this config: > > namespace { > disabled = no > hidden = no > ignore_on_failure = no > inbox = no > list = children > location = maildir:/var/vmail/archives/%n/Maildir > prefix = Archives/ > separator = / > subscriptions = yes > type = private > } > namespace inbox { > disabled = no > hidden = no > ignore_on_failure = no > inbox = yes > list = yes > location = > prefix = > separator = / > subscriptions = yes > type = private > } > plugin { > quota = maildir:User quota:ns= > quota2 = maildir:Archives quota:ns=Archives/ > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+200M > quota_rule3 = ns=Archives/:storage=5G > } > > > With this setup I get the following error: > > > Debug: Quota root: name=User quota backend=maildir args=ns= > Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 > Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 > Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete > configuration for rule 'storage=5G' should be changed to 'storage=+5G' > Debug: Quota rule: root=User quota mailbox=ns=Archives/ > bytes=+5368709120 messages=0 > Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ > > > Thank you for your kind help. > > best regards > Andreas > Hi all, my quota issue is solved. It was a stupid configuration error and it seems that I have been blind :-( plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota2_rule = *:storage=5G } this confiurationg works like a charm. Thank you for your kind help. best regards Andreas From aoster at novanetwork.de Wed Feb 6 15:26:39 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Wed, 06 Feb 2013 14:26:39 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <510F81F2.1020809@novanetwork.de> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> <510F81F2.1020809@novanetwork.de> Message-ID: <51125A0F.4040502@novanetwork.de> Am 04.02.2013 10:40, schrieb Andreas Oster: > Am 22.01.2013 12:54, schrieb Timo Sirainen: >> On 17.1.2013, at 16.58, Andreas Oster wrote: >> >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. >> >> It should work for all namespaces. >> >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota >> >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota >> >> That should work? Worked at least in latest v2.1 hg. >> >> > Hello Timo, > > can you please confirm, that it is actually possible to have multiple > quota roots with separate/different quota values. > > Currently I have this config: > > namespace { > disabled = no > hidden = no > ignore_on_failure = no > inbox = no > list = children > location = maildir:/var/vmail/archives/%n/Maildir > prefix = Archives/ > separator = / > subscriptions = yes > type = private > } > namespace inbox { > disabled = no > hidden = no > ignore_on_failure = no > inbox = yes > list = yes > location = > prefix = > separator = / > subscriptions = yes > type = private > } > plugin { > quota = maildir:User quota:ns= > quota2 = maildir:Archives quota:ns=Archives/ > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+200M > quota_rule3 = ns=Archives/:storage=5G > } > > > With this setup I get the following error: > > > Debug: Quota root: name=User quota backend=maildir args=ns= > Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 > Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 > Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete > configuration for rule 'storage=5G' should be changed to 'storage=+5G' > Debug: Quota rule: root=User quota mailbox=ns=Archives/ > bytes=+5368709120 messages=0 > Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ > > > Thank you for your kind help. > > best regards > Andreas > Hi all, my quota issue is solved. It was a stupid configuration error and it seems that I have been blind :-( plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota2_rule = *:storage=5G } this confiurationg works like a charm. Thank you for your kind help. best regards Andreas From marcio.merlone at a1.ind.br Wed Feb 6 15:36:18 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 11:36:18 -0200 Subject: [Dovecot] Sieve also filter outgoing messages Message-ID: <51125C52.9040007@a1.ind.br> Greetings, A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver on postfix, perhaps the sieve script could be executed when saving to the Sent folder? Regards, -- Marcio Merlone From h.reindl at thelounge.net Wed Feb 6 15:53:02 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 06 Feb 2013 14:53:02 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51125C52.9040007@a1.ind.br> References: <51125C52.9040007@a1.ind.br> Message-ID: <5112603E.90805@thelounge.net> Am 06.02.2013 14:36, schrieb Marcio Merlone: > A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver > on postfix, perhaps the sieve script could be executed when saving to the Sent folder? there are complete different things happening for incoming messages and save sent messages you do NOT want get sieve in action if you drag&drop messages between folders and do this for sent messages would trigger the same since the client acts the same way -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From rafaelvolpeti at gmail.com Wed Feb 6 16:09:26 2013 From: rafaelvolpeti at gmail.com (Rafael VOlpe TI) Date: Wed, 6 Feb 2013 12:09:26 -0200 Subject: [Dovecot] Trouble when set mail_nfs_index = yes Message-ID: Hi Buddies, dovecot-sql.conf driver = mysql connect = host=127.0.0.1 dbname=postfix user=postfix password=secret default_pass_scheme = plain password_query = SELECT username as user, password \ FROM mailbox WHERE username = '%u' AND active = 1 user_query = SELECT concat("/var/vmail/", maildir) as home, 125 as uid, 125 as gid FROM mailbox WHERE username = '%u' Version: 2.0.19: /etc/dovecot/dovecot.conf doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/conf.d/10-mail.conf:157: fsync_disable has been renamed to mail_fsync # OS: Linux 3.2.0-37-generic x86_64 Ubuntu 12.04.2 LTS auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 125 first_valid_uid = 125 last_valid_gid = 125 last_valid_uid = 125 log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/%Lh/:INDEX=/%Lh/ mail_nfs_storage = yes mmap_disable = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = Hello, I installed a few weeks ago the Dovecot package on a Debian stable along with postfix, everything works fine except the sieve filter. I followed the howto found on the internet (google: debian postfix dovecot sieve). I'm able to view and modify sieve rules with roundcube or thunderbird plugins, but the rules are not applied when a message arrives and all messages arrive in INBOX. Here is the content of my configuration files concerning sieve : dovecot.conf protocols = imap imaps managesieve protocol managesieve { listen = *:2000 } protocol managesieve { mail_executable = /usr/lib/dovecot/managesieve managesieve_max_line_length = 65536 managesieve_logout_format = bytes=%i/%o } protocol lda { postmaster_address = postmaster at babelouest.org mail_plugins = sieve mail_plugin_dir = /usr/lib/dovecot/modules/lda log_path = info_log_path = syslog_facility = mail } plugin { sieve_dir=~/Maildir/sieve } And the postfix/main.cf config file contains for dovecot sieve the following maibox_command = /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT" dovecot_destination_recipient_limit = 1 virtual_transport = dovecot I also tried the command mailbox_command = /usr/lib/dovecot/deliver it didn't work either... Is there somthing so obvious that I didn't see it ? Thanks in advance From rs at sys4.de Wed Feb 6 17:19:26 2013 From: rs at sys4.de (Robert Schetterer) Date: Wed, 06 Feb 2013 16:19:26 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <5112603E.90805@thelounge.net> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> Message-ID: <5112747E.7030800@sys4.de> Am 06.02.2013 14:53, schrieb Reindl Harald: > > > Am 06.02.2013 14:36, schrieb Marcio Merlone: >> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver >> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? > > there are complete different things happening > for incoming messages and save sent messages > > you do NOT want get sieve in action if you drag&drop > messages between folders and do this for sent messages > would trigger the same since the client acts the same way > > Sorry Harald, i dont agree, this would be a nice feature to have in fact we internally disussed this allready, it might be used for workaround imap bugs of new outlook, and for auto flagging and auto sorting mails, anyway at recent i see no way to get it real on the tec side , but that might change in the future with i.e relation to rfc4550 Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tlx at leuxner.net Wed Feb 6 17:25:05 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 6 Feb 2013 16:25:05 +0100 Subject: [Dovecot] Sieve not filtering In-Reply-To: References: Message-ID: <20130206152505.GA9635@nihlus.leuxner.net> * Nicolas Mora 2013.02.06 16:06: > plugin { > sieve_dir=~/Maildir/sieve > } Better to post 'dovecot -n' which is the active config. http://wiki.dovecot.org/LDA/Sieve/Dovecot Not sure what the default was for the active script in the old Dovecot versions as 'sieve_dir' is the upload directory for scripts to be referenced (see Wiki). Active config will tell. You may raise logging levels and it will tell you what happens when the LDA tries to spawn sieve: http://wiki.dovecot.org/Logging Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From skdovecot at smail.inf.fh-brs.de Wed Feb 6 17:45:02 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 Feb 2013 16:45:02 +0100 (CET) Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <5112747E.7030800@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 6 Feb 2013, Robert Schetterer wrote: > Am 06.02.2013 14:53, schrieb Reindl Harald: >> Am 06.02.2013 14:36, schrieb Marcio Merlone: >>> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver >>> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? >> >> there are complete different things happening >> for incoming messages and save sent messages >> >> you do NOT want get sieve in action if you drag&drop >> messages between folders and do this for sent messages >> would trigger the same since the client acts the same way > > Sorry Harald, i dont agree, this would be a nice feature to have > in fact we internally disussed this allready, it might be used for > workaround imap bugs of new outlook, and for auto flagging and auto > sorting mails, anyway at recent i see no way to get it real on the tec > side , but that might change in the future with i.e relation to rfc4550 A combination of BCC (instead of IMAP-Append to Sent) and a stable detection of "outgoing" will work. E.g. BCC: myaccount+outgoing at mydomain Then Sieve must be able to detect subaddressing or default mailbox to fileinto. First is not availble for all installations. Last is not available at all, I think. Maybe in combination with a "From" check and/or check of last recieved header to prevent misuse, when an insider (needs to know the subaddress) mails to myaccount+outgoing at mydomain from an external MTA. Using BCC instead of IMAP-Append-to-Sent also has the advantages that the message is transferred only once and that the Recieved header has the queueid for later trouble shooting. Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURJ6f13r2wJMiz2NAQImZAf+PWNSGGA5rlep9Onf2GXMpGYpvkDpGj1G DcWRykGEO9kGhQwbO/B7AIQ21bCI9PrYg2pRN9XMGpH4RF9ippsICjlOW9Eo2n5X Lt/0dd/Jdyj5UhbvdcxzI1VLfDItbl2N6mNBTClo9V/DDKjo5W89gzIaxOQoTIvz LTGj1EFn17UUtP67/uE4zUcJ7/+rc2BhLcBL2ZpJXbKT4WFvjT/THuJ2UagYTdhZ 9+nvtNyxq54iGQzja5dR8CEXbcuyVtIgY5MZXBP3PMIfGrkfcTP8Iz4dTV0caO4/ QqrSJfe/K50h7pnYr+PlnjMtKu3vbDwkr/Qk8o4I4lBYlAwZtKKyjg== =YsoM -----END PGP SIGNATURE----- From marcio.merlone at a1.ind.br Wed Feb 6 17:57:41 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 13:57:41 -0200 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <5112747E.7030800@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> Message-ID: <51127D75.8090100@a1.ind.br> Em 06-02-2013 13:19, Robert Schetterer escreveu: > Am 06.02.2013 14:53, schrieb Reindl Harald: >> Am 06.02.2013 14:36, schrieb Marcio Merlone: >>> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver >>> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? >> there are complete different things happening >> for incoming messages and save sent messages >> >> you do NOT want get sieve in action if you drag&drop >> messages between folders and do this for sent messages >> would trigger the same since the client acts the same way > Sorry Harald, i dont agree, this would be a nice feature to have > in fact we internally disussed this allready, it might be used for > workaround imap bugs of new outlook, and for auto flagging and auto > sorting mails, anyway at recent i see no way to get it real on the tec > side , but that might change in the future with i.e relation to rfc4550 My specific need in this case is that we must forward copy all mails with a specific subject and/or destination to a mail account for auditing and registry purposes. It already works for messages which has at least on inside destination (multiple exact same copies when multiple inside destination, but what the heck...), but if a message is sent with only outside destinations I get no copy. It should work similar to postfix always_bcc, but only when a criteria matches. Regards. -- Marcio Merlone From tss at iki.fi Wed Feb 6 18:23:47 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 18:23:47 +0200 Subject: [Dovecot] Trouble when set mail_nfs_index = yes In-Reply-To: References: Message-ID: <36C41FF4-BD63-4BE8-AFA6-25E410D1E11E@iki.fi> On 6.2.2013, at 16.09, Rafael VOlpe TI wrote: > I use mailbox in nfs share. > When i set the option "mail_nfs_index = yes" the dovecot fails to start. > > Looking at logs, only prompted this: > > "Feb 6 11:07:25 picapau kernel: [ 1436.376023] init: dovecot main process > (4177) terminated with status 89" without more info. You're not looking into the correct log file. Dovecot logs another error message about it. http://wiki2.dovecot.org/Logging Anyway, better not use mail_nfs_index anyway. Use http://wiki2.dovecot.org/Director instead. From p at sys4.de Wed Feb 6 18:24:45 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Wed, 6 Feb 2013 17:24:45 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> Message-ID: <20130206162444.GB4645@sys4.de> * Steffen Kaiser : > On Wed, 6 Feb 2013, Robert Schetterer wrote: > > > Am 06.02.2013 14:53, schrieb Reindl Harald: > >> Am 06.02.2013 14:36, schrieb Marcio Merlone: > >>> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver > >>> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? Barry (Leiba) wrote . AFAIK it awaits further discussion. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Wed Feb 6 18:25:59 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 18:25:59 +0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> <1360030140.10326.54.camel@innu> <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> Message-ID: <0CAD42D4-FB5E-4BED-A5C7-345278F68CAB@iki.fi> On 5.2.2013, at 16.10, Gustavo Sansone wrote: >>> i'm very confused with the namespace settings. i don't understand what >>> is the correct way to configure it. >>> don't need public or shared folders. >> >> Easiest to use the default namespace settings. There you wouldn't have >> any namespace prefix. > > if i just remove the namespace 'INBOX.' , on 10-mail.conf then there is something that creates new Send and Draft folders into my maildir folder like this INBOX.Send and INBOX.Drafts. and Roundcube shows me these new folders under INBOX (screenshot attached). > can't figure out why it happens. Because different clients you're using have been configured to use different namespace settings. This is unfortunately how IMAP protocol still works. From Dovecot's side either set the prefix to INBOX. or to empty, and from all of the clients' sides make sure they use the matching configuration. From gfinch at ldmltd.ca Wed Feb 6 18:35:44 2013 From: gfinch at ldmltd.ca (Gregory Finch) Date: Wed, 06 Feb 2013 08:35:44 -0800 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51127D75.8090100@a1.ind.br> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> Message-ID: <51128660.4040800@ldmltd.ca> On 2013-02-06 7:57 AM, Marcio Merlone wrote: > Em 06-02-2013 13:19, Robert Schetterer escreveu: >> Am 06.02.2013 14:53, schrieb Reindl Harald: >>> Am 06.02.2013 14:36, schrieb Marcio Merlone: >>>> A probably simple question and answer: can a sieve script be >>>> executed on outgoing messages? I already use deliver >>>> on postfix, perhaps the sieve script could be executed when saving >>>> to the Sent folder? >>> there are complete different things happening >>> for incoming messages and save sent messages >>> >>> you do NOT want get sieve in action if you drag&drop >>> messages between folders and do this for sent messages >>> would trigger the same since the client acts the same way >> Sorry Harald, i dont agree, this would be a nice feature to have >> in fact we internally disussed this allready, it might be used for >> workaround imap bugs of new outlook, and for auto flagging and auto >> sorting mails, anyway at recent i see no way to get it real on the tec >> side , but that might change in the future with i.e relation to rfc4550 > My specific need in this case is that we must forward copy all mails > with a specific subject and/or destination to a mail account for > auditing and registry purposes. It already works for messages which > has at least on inside destination (multiple exact same copies when > multiple inside destination, but what the heck...), but if a message > is sent with only outside destinations I get no copy. It should work > similar to postfix always_bcc, but only when a criteria matches. > > Regards. > Would sender_bcc_maps or recipient_bcc_maps be able to work on the postfix side? Otherwise I would suggest BCC like Steffen did and filter out what you want with sieve. Heck, with the *_bcc_maps on the postfix side you can still filter what you want with sieve. -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Wed Feb 6 18:42:39 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 18:42:39 +0200 Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On 6.2.2013, at 6.51, Joseph Tam wrote: > On a related topic, what's the easiest way to alias various common > mailbox names to one physical mailbox? For example, mapping "Trash", > "Deleted Messages", "Junk" to the same mailbox? > > Would you use the SPECIAL-USE, or is there a better way to do this? > Namescape configuration? Virtual plugin? Ideally SPECIAL-USE will take care of it in future. For now you should configure it, but also realize that it won't work for all clients. There's also mailbox_alias plugin in v2.1.10+: http://wiki2.dovecot.org/Plugins/MailboxAlias (the link will work in a few minutes). From stephan at rename-it.nl Wed Feb 6 19:01:42 2013 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 06 Feb 2013 18:01:42 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <20130206162444.GB4645@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <20130206162444.GB4645@sys4.de> Message-ID: <51128C76.4030501@rename-it.nl> On 2/6/2013 5:24 PM, Patrick Ben Koetter wrote: > Barry (Leiba) wrote . AFAIK it > awaits further discussion. p at rick This is on my TODO list, but Dovecot will need to gain IMAP METADATA support first. Regards, Stephan. From marcio.merlone at a1.ind.br Wed Feb 6 19:12:34 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 15:12:34 -0200 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51128660.4040800@ldmltd.ca> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> <51128660.4040800@ldmltd.ca> Message-ID: <51128F02.5090102@a1.ind.br> Em 06-02-2013 14:35, Gregory Finch escreveu: > Would sender_bcc_maps or recipient_bcc_maps be able to work on the > postfix side? Otherwise I would suggest BCC like Steffen did and > filter out what you want with sieve. Heck, with the *_bcc_maps on the > postfix side you can still filter what you want with sieve. I've been poking around postfix now, a subject_bcc_maps with a pcre map would do :), and more realistic, a pcre header_checks with a milter may be the way to go, must read about milter first. Steffen cited "outgoing detection" - I only asked about outgoing because incoming is already done with sieve, outgoing is the missing part. A single solution for both is the ideal, so you can forget about outgoing detection. IMAP-append-to-send also were a blind shot from me, forget it also. In sum, when a subject matches a pcre, bcc it to another account, no matter what direction. Regarding Steffen suggestion, I am unable to figure what he meant. :P -- Marcio Merlone From tss at iki.fi Wed Feb 6 19:34:16 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 19:34:16 +0200 Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On 5.2.2013, at 21.52, Radek Novotn? wrote: > let me ask a question, please. Is it possible in dovecot to set up per user special-use folder names? > > Imagine situation with two users where first prefere another language that second. > > mailbox Sent { > special_use = \Sent > } > > for english speaking users and > > mailbox "Odeslan? po?ta" { > special_use = \Sent > } > > for czech speaking users. Like Patrick mentioned this isn't how it should be used.. But it would be possible anyway by returning the proper settings from userdb lookup. Easier probably set both of these in the namespace, then only configure which one will be autocreated (only if both mailboxes are created the client sees two \Sent flags). The autocreation can be made per-user by returning namespace/inbox/Sent/auto=create or namespace/inbox/Odeslan? po?ta/auto=create field from userdb. From rs at sys4.de Wed Feb 6 19:40:56 2013 From: rs at sys4.de (Robert Schetterer) Date: Wed, 06 Feb 2013 18:40:56 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51127D75.8090100@a1.ind.br> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> Message-ID: <511295A8.8020302@sys4.de> Am 06.02.2013 16:57, schrieb Marcio Merlone: > It should work similar to postfix always_bcc, but only when a criteria > matches. incomming mails can be filtered through sieve, always_bcc etc and + adressing maybe be a way to solve this, i tested somthing like this for mail archiving, however the filter rules may get complex Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From marcio.merlone at a1.ind.br Wed Feb 6 19:46:20 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 15:46:20 -0200 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <511295A8.8020302@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> <511295A8.8020302@sys4.de> Message-ID: <511296EC.4070201@a1.ind.br> Em 06-02-2013 15:40, Robert Schetterer escreveu: > Am 06.02.2013 16:57, schrieb Marcio Merlone: >> It should work similar to postfix always_bcc, but only when a criteria >> matches. > incomming mails can be filtered through sieve, always_bcc etc and + > adressing maybe be a way to solve this, i tested somthing like this > for mail archiving, however the filter rules may get complex Incoming is done with sieve, the problem comes to outgoing without any rcpt to inside, in which case it becomes incoming too. It must happen only on a specific subject.... -- Marcio Merlone From tss at iki.fi Wed Feb 6 20:30:05 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 20:30:05 +0200 Subject: [Dovecot] Subfolders problem In-Reply-To: References: <1360029796.10326.49.camel@innu> Message-ID: On 5.2.2013, at 10.00, Adam Maciejewski wrote: > # 2.1.10: /etc/dovecot/dovecot.conf v2.1.14 has some bugfix related to this.. Also I couldn't reproduce it anymore, so probably fixed. From jtam.home at gmail.com Thu Feb 7 03:59:09 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 6 Feb 2013 17:59:09 -0800 (PST) Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On Wed, 6 Feb 2013, Timo Sirainen wrote: >> On a related topic, what's the easiest way to alias various common >> mailbox names to one physical mailbox? For example, mapping "Trash", >> "Deleted Messages", "Junk" to the same mailbox? >> >> Would you use the SPECIAL-USE, or is there a better way to do this? >> Namescape configuration? Virtual plugin? > > Ideally SPECIAL-USE will take care of it in future. For now you should > configure it, but also realize that it won't work for all clients. > There's also mailbox_alias plugin in v2.1.10+: > http://wiki2.dovecot.org/Plugins/MailboxAlias (the link will work in a > few minutes). Perfect. I added a link to this page to the parent Plugins page Joseph Tam From f.biermann at eye-on.de Thu Feb 7 12:04:33 2013 From: f.biermann at eye-on.de (Frederic Biermann) Date: Thu, 7 Feb 2013 11:04:33 +0100 Subject: [Dovecot] fts_solr search in subfolders? Message-ID: Hello, i am using dovecot 2.1.x with Solr 4.x and it is all working fine. Search in current selected folder is extremly fast. But i want to ask if it is possible that i have selected inbox, setup a search and get results from subfolders. This is not working for me. I have to select the subfolder and then i get results. Is there any solution to fix this? best regards lexusburn From raabe at froglogic.com Thu Feb 7 12:22:24 2013 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 07 Feb 2013 11:22:24 +0100 Subject: [Dovecot] File permissions used for automatically created mailbox Message-ID: <51138060.5040404@froglogic.com> Hi, I'm running Dovecot 2.1.12 on FreeBSD (quite successfully so, thanks for this nice piece of software!). One thing which is slightly annoying though is that automatically created mailboxes (I have lda_mailbox_autocreate set) don't have the file permissions I'd like them to have. I'm using a vmail-based system, i.e. all mail is owned by vmail:vmail; another member of the vmail group is called 'backup', which has read access to all mail in order to create backups. All mail is stored beneath /home/vmail, e.g. /home/vmail/frerich/Maildir Right now, newly created mailbox directories (e.g. /home/vmail/bob) have 0700 permissions, but I'd like to have 0750 for all directories and 0640 for all files so that all files and directories are group-readable for backup purposes. Does anybody know whether this is configurable somehow? Right now, my workaround consists running this cron script every night: #!/bin/sh chown -R vmail:vmail /home/vmail/ find /home/vmail/ -type d -print0 | xargs -0 chmod 0750 find /home/vmail/ -type f -print0 | xargs -0 chmod 0640 -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From kavish.karkera at yahoo.com Thu Feb 7 13:19:31 2013 From: kavish.karkera at yahoo.com (Kavish Karkera) Date: Thu, 7 Feb 2013 19:19:31 +0800 (SGT) Subject: [Dovecot] fts_solr search in subfolders? In-Reply-To: References: Message-ID: <1360235971.78313.YahooMailNeo@web193505.mail.sg3.yahoo.com> Hi lexusburn, We are too trying to setting up the same scenario, would like know what configuration / settings you have used in apache solr, Help me? :) Regards, Kavish Karkera ________________________________ From: Frederic Biermann To: dovecot at dovecot.org Sent: Thursday, 7 February 2013 3:34 PM Subject: [Dovecot] fts_solr search in subfolders? Hello, i am using dovecot 2.1.x with Solr 4.x and it is all working fine. Search in current selected folder is extremly fast. But i want to ask if it is possible that i have selected inbox, setup a search and get results from subfolders. This is not working for me. I have to select the subfolder and then i get results. Is there any solution to fix this? best regards lexusburn From skdovecot at smail.inf.fh-brs.de Thu Feb 7 13:34:32 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 7 Feb 2013 12:34:32 +0100 (CET) Subject: [Dovecot] File permissions used for automatically created mailbox In-Reply-To: <51138060.5040404@froglogic.com> References: <51138060.5040404@froglogic.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 7 Feb 2013, Frerich Raabe wrote: > I'm running Dovecot 2.1.12 on FreeBSD (quite successfully so, thanks for this > nice piece of software!). One thing which is slightly annoying though is that > automatically created mailboxes (I have lda_mailbox_autocreate set) don't > have the file permissions I'd like them to have. > > Right now, newly created mailbox directories (e.g. /home/vmail/bob) have 0700 > permissions, but I'd like to have 0750 for all directories and 0640 for all > files so that all files and directories are group-readable for backup > purposes. > > Does anybody know whether this is configurable somehow? As far as I know and in my 2.1.14 it is like that: + the .XYZ directory inherits its permission to any new .XYZ.ABC mailbox (second level and down). + the Maildir/ directory inherits its permission to any new .XYZ mailbox (top level). + Files get the permission of the directory minus "x". This is true for "CREATE" command and Dovecot deliver. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURORSF3r2wJMiz2NAQKCKwf+Oz98Fwub9G+QKwEJWIY5O5EtdFJlEIlH rzILIgsM9TRwuzdIFC+ZvAwY8EjOBd+EEWAnp2Ie3PEnP0mVloWUGFh6jyo1CYM3 lvROgjqXpo3++6+6KvHVCHUP21hK9nfJrYiGf5ESj0PMTlKPS4kBgdOhGnMQayM3 3Tn8RcpngzG0IG/ZNfW780uSAfmK0e7YOek5BumsRtt5J7cpozL2X6ce91ymOi4A fzyKTit+KLpwiglUj+68ZB39eX1SmHqrv8UeEXmxKCCjxoxHnk+pq/30MJd3a7lF xOLW8yJx6qFUbs0J3TymN+rSKTDkkaaNU3QM9rsQjy2W5XWYF+5FKg== =ONer -----END PGP SIGNATURE----- From mark at msapiro.net Thu Feb 7 21:06:28 2013 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 7 Feb 2013 11:06:28 -0800 Subject: [Dovecot] Dovecot 2.2.beta1 Message-ID: Just an FYI, I have been running 2.2.beta1 for a few days now with no problems, but I'm not using any of the new features ;) -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From kharford at constantcontact.com Fri Feb 8 00:12:39 2013 From: kharford at constantcontact.com (Harford, Ken) Date: Thu, 7 Feb 2013 22:12:39 +0000 Subject: [Dovecot] user uknown Message-ID: Hi, I upgraded to version 2.1.1 from version 1.0.7 and I am having issues with postfix and dovecot reading my userdb file. Here is my dovecot config: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.1.1.el5 x86_64 CentOS release 5.7 (Final) auth_debug = yes auth_socket_path = /var/run/dovecot/auth-userdb default_internal_user = vmail default_login_user = dovecot disable_plaintext_auth = no auth_mechanisms = plain log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir mbox_write_locks = fcntl namespace inbox { inbox = yes location = prefix = } passdb { args = username_format=%u /etc/dovecot.ctctmon/%d/userdb driver = passwd-file } postmaster_address = postmaster at example.com protocols = imap lmtp service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap { process_limit = 1024 } ssl = required ssl_cert = service=lda Feb 07 17:05:27 auth: Debug: passwd-file(ctctsc9 at p2-ctctmon.constantcontact.com): lookup: user=ctctsc9 at p2-ctctmon. file=/etc/dovecot.ctctmon/p2-ctctmon./userdb Feb 07 17:05:27 auth: Info: passwd-file(ctctsc9 at p2-ctctmon.): unknown user Feb 07 17:05:27 lda: Debug: auth input: Feb 07 17:05:27 auth: Debug: master out: NOTFOUND 1 Postfix: Feb 7 17:07:51 p2-ctctmon101 postfix/pipe[17969]: C91831C8168: to=, relay=dovecot, delay=0.1, delays=0.09/0/0/0.02, dsn=5.1.1, status=bounced (user unknown) Userdb: ctctsc1:$1$e3k3Xp.u$QdSaJMh79dhivHUuaToDO0:::ctctsc1 Any ideas would be greatly appreciated!! Ken From user+dovecot at localhost.localdomain.org Fri Feb 8 01:26:19 2013 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Fri, 08 Feb 2013 00:26:19 +0100 Subject: [Dovecot] dsync: Invalid server handshake In-Reply-To: <36C41C6B-F529-4787-A12E-C55975B44629@ct.heise.de> References: <36C41C6B-F529-4787-A12E-C55975B44629@ct.heise.de> Message-ID: <5114381B.5060209@localhost.localdomain.org> On 02/05/2013 07:36 PM Dusan Zivadinovic wrote: > ? > and I get this error: > > dsync-local(dz): Error: Invalid server handshake: dsync-server 2 > dsync-remote(dz): Error: Invalid client handshake: dsync-client 1 > > # lokal machine: dovecot 2.0.19apple1 running on Mac OS X 10.8.2 Server > using standard Apple-Dovecot-Configuration > > # remote machine: 2.1.12 running on Mac OS X 10.4 using PAM authentication > > > I cant seem to find any documentation to this error, nor do I find any dsync entries > in the logs of both machines. Does anyone have a hint? You are using different minor-versions of Dovecot; v2.0 v/s v2.1. Dovecot v2.0 uses dsync 'version 1', and v2.1 uses dsync 'version 2'. If there aren't to much big mailboxes, I would stop imap/pop3 on the old server, move the mailboxes to the new server and start iamp/pop3 on the new server. Or you have to update Dovecot to version 2.1 on the older server. Then you should be able to dsync between both servers. Regards, Pascal -- The trapper recommends today: c01dcafe.1303900 at localdomain.org From tlx at leuxner.net Fri Feb 8 09:25:33 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 8 Feb 2013 08:25:33 +0100 Subject: [Dovecot] user uknown In-Reply-To: References: Message-ID: <20130208072533.GA31594@nihlus.leuxner.net> * Harford, Ken 2013.02.07 23:12: > passdb { > args = username_format=%u /etc/dovecot.ctctmon/%d/userdb > driver = passwd-file http://wiki2.dovecot.org/AuthDatabase/PasswdFile %u expects user at domain in the file > Userdb: > ctctsc1:$1$e3k3Xp.u$QdSaJMh79dhivHUuaToDO0:::ctctsc1 Add the domain component to ctctsc1 in this file or set username_format=%n. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From amateo at um.es Fri Feb 8 09:29:09 2013 From: amateo at um.es (Angel L. Mateo) Date: Fri, 08 Feb 2013 08:29:09 +0100 Subject: [Dovecot] Disabling auth caching just from one passdb Message-ID: <5114A945.4090700@um.es> Hello, I'm using various password databases (one of them is a ldap database, which is the used for users connecting with email clients, and the other with pam, using a pam module integrated with our SSO that is used from our webmail). I would like to disable password caching in my pam database (because 'passwords' provided by our SSO are single use tokens, so caching them has no benefit), but not in the ldap. As far as I know, auth_cache_size is the option to enable caching in all databases. But, is there any option I could use to disable it in a specific password database? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From jtam.home at gmail.com Fri Feb 8 09:59:43 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 7 Feb 2013 23:59:43 -0800 (PST) Subject: [Dovecot] Imap process crash: assertion failed: (full_fs_access) In-Reply-To: References: Message-ID: Any guess at what would cause this? Feb 7 21:20:53 server dovecot: imap(user): Panic: file mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion failed: (full_fs_access) (Sorry, no core dump) According to my logs, this user couldn't start an IMAP session for a 2 hour stretch for hundreds of connections. Then the problem seemed to have went away. I logged back as that user using a master password, but couldn't recreate this problem. The backtrace is just hex numbers (stripped executables) and I didn't drop privileges, so it refused to drop core, so that's no help. Joseph Tam From peter.rindfuss at wzb.eu Fri Feb 8 11:24:05 2013 From: peter.rindfuss at wzb.eu (Peter Rindfuss) Date: Fri, 08 Feb 2013 10:24:05 +0100 Subject: [Dovecot] sieve /editheader / addheader :last crash Message-ID: <5114C435.4040105@wzb.eu> Hi Everybody, I'm using dovecot 2.1.14 from http://xi.rename-it.nl/debian/ with debian squeeze stable. In general, sieve works fine for me, but "addheader :last ..." from the editheader extension causes a crash when it processes a message. I was able to narrow this down a bit: It only happens if "addheader :last" is the first editheader command executed in a script. If it is preceded by any "deleteheader" or "addheader" without ":last", everything is fine. This is what I get from sieve-test when it crashes: sieve-test(root): Panic: file istream.c: line 133 (i_stream_read): assertion failed: (_stream->skip != _stream->pos) sieve-test(root): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4881a) [0x7f5e0b37681a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f5e0b376902] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f5e0b349f7f] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x17c) [0x7f5e0b37da9c] -> /usr/lib/dovecot/libdovecot.so.0(+0x51453) [0x7f5e0b37f453] -> /usr/lib/dovecot/libdovecot.so.0(+0x5150e) [0x7f5e0b37f50e] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7f5e0b37d989] -> /usr/lib/dovecot/libdovecot.so.0(+0x53fe1) [0x7f5e0b381fe1] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7f5e0b37d989] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x52) [0x7f5e0b37dc72] -> /usr/lib/dovecot/libdovecot.so.0(io_stream_copy+0x41) [0x7f5e0b38ae21] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e1de) [0x7f5e0b38c1de] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_send_istream+0x33) [0x7f5e0b38acb3] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_continue+0x38) [0x7f5e0be0caf8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xb8) [0x7f5e0be376d8] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x56) [0x7f5e0be086c6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x4a) [0x7f5e0be3ce9a] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x3fa66) [0x7f5e0c103a66] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x366b4) [0x7f5e0c0fa6b4] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x3ff) [0x7f5e0c0fabef] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x56) [0x7f5e0c10b456] -> sieve-test(main+0x73e) [0x403a5e] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f5e0abdec8d] -> sieve-test() [0x403219] Aborted Best regards Peter From paolo.miotto at uniud.it Fri Feb 8 14:17:40 2013 From: paolo.miotto at uniud.it (Paolo Miotto) Date: Fri, 08 Feb 2013 13:17:40 +0100 Subject: [Dovecot] Moving index files on another disk: no troubles but need more assurances Message-ID: <20130208131740.39334u183vmu13ac@webmail.uniud.it> Hi, I'm using dovecot 1.2.15 on debian squeeze with maildir, and I want to move index files out of the mailboxes disk to reduce disk I/O. I need to get the assurance that the switch is completely transparent to my clients (IMAP and POP). I read the wiki documentation and have found this thread (http://dovecot.org/pipermail/dovecot/2009-September/042665.html), which makes me confident, even if it speaks of a older version and only pop3. So I set up a test server, copied a true mailbox and changed mail location from: mail_location = maildir:~/Maildir to mail_location = maildir:~/Maildir:INDEX=/srv/indexes/%1n/%n (all users under the same domain, no need to use %d). On the IMAP side all seems to work fine, new indexes are created in the new location (dovecot.index.log on 1st access, dovecot.index.cache on message manipolation). A test client (evolution) doesn't notice at all, and I have verified via a telnet imap session that UIDs don't change. But I see that when I move the indexes the IMAP HIGHESTMODSEQ response changes and is reset to 1. Can this baffle the clients? On the rfc I don't found reference to this status walking backwards. Need I to verify that all messages have the virtual size (W=nnnn) in the file name (the mailboxes are from a previous cyrus installation, then we switch to dovecot 1.2)? What other test you suggest that checking UIDL on the pop3 side? What happens if the indexes disk disappears (broken or removed from bay or all paths down for FC/iscsi)? Can dovecot 1.2 continue with INDEX=MEMORY as for a disk full? Sorry for the many questions.... Paolo ---------------------------------------------------------------------- SEMEL (SErvizio di Messaging ELettronico) - AINF, Universita' di Udine From tss at iki.fi Fri Feb 8 15:08:50 2013 From: tss at iki.fi (Timo Sirainen) Date: Fri, 08 Feb 2013 15:08:50 +0200 Subject: [Dovecot] v2.1.15 released Message-ID: <1360328930.3230.12.camel@hurina> http://dovecot.org/releases/2.1/dovecot-2.1.15.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.15.tar.gz.sig v2.1.14's dovecot.index.cache fixes caused Dovecot to use more disk I/O and memory than was necessary. From bogus@does.not.exist.com Fri Feb 8 16:31:49 2013 From: bogus@does.not.exist.com () Date: Fri, 08 Feb 2013 14:31:49 -0000 Subject: No subject Message-ID: while (rename(src, dest) < 0 && count < 2) { if (errno != EEXIST) { rename can never return EEXIST in errno. maildir-save.c, line 124. /* move the file into new/ directory - syncing will pick it up from there */ if (rename(tmp_path, new_path) == 0) failed = FALSE; Here you can lose emails if the new/ folder contains a message whose base name is equal. The only way to avoid this is: 1) use link and unlink, not rename 2) leave messages that have a time_t part equal to or higher than time(NULL). Anyway, Maildir has a strict consistency criteria which says that all messages that are linked from tmp/ into new/ _must_ use time(NULL) plus a number that is guaranteed not to lapse within one second, and which does not collide with other messages in new/. new/ is the single entry point into cur/, and messages in new/ can not be "older" than the messages in cur/. The simple reasoning for this is that you can never guarantee that there is no message in cur/ that has the same base name, but perhaps different flags. It follows that when moving a message from new/ to cur/, it is required that the server only picks messages that are older than one second. Andy -- Andreas Aardal Hanssen http://www.andreas.hanssen.name/gpg From bogus@does.not.exist.com Fri Feb 8 16:31:49 2013 From: bogus@does.not.exist.com () Date: Fri, 08 Feb 2013 14:31:49 -0000 Subject: No subject Message-ID: checking IMAP connection. Based on that experimentation, it seems that when I try to verify certificate files with openssl, all checks out, but when I try to check thing through IMAPS, things go ugly (see log below). If I try same openssl s_client command on my web server, it gets everything correctly. As result from this one, I've even tried to use certificate from my web server with IMAP and even then openssl keeps on saying that there is bad record mac. Is this bug in dovecot's SSL handling or have I managed to mess something in my setup? URLS: CA cert: http://jylitalo.homeip.net/ca/ca.crt IMAPD cert: http://jylitalo.homeip.net/ca/imapd.crt [log starts] bash-2.05a$ openssl verify -CAfile /usr/local/www/data/ca/ca.crt /etc/ssl/certs/imapd.crt /etc/ssl/certs/imapd.crt: OK bash-2.05a$ openssl s_client -host localhost -port 993 -CAfile /usr/local/www/data/ca/ca.crt -verify -debug verify depth is 0 CONNECTED(00000003) depth=1 /C=FI/ST=Finland/L=Helsinki/O=Juha Ylitalo/CN=Juha Ylitalo/Email=jylitalo at iki.fi verify return:1 depth=0 /C=FI/ST=Finland/O=Juha Ylitalo/CN=coat.st-paul/Email=jylitalo at iki.fi verify return:1 47169:error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert bad record mac:/usr/src/secure/lib/libssl/../../../crypto/openssl/crypto/../ssl/s3_pkt.c:1046:SSL alert number 20 47169:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:/usr/src/secure/lib/libssl/../../../crypto/openssl/crypto/../ssl/s23_lib.c:226: bash-2.05a$ [log ends] -- Juha Ylitalo juha.o.ylitalo at nokia.com +358 40 562 6152 http://linux.nokia.com/~jylitalo/ From bogus@does.not.exist.com Fri Feb 8 16:31:49 2013 From: bogus@does.not.exist.com () Date: Fri, 08 Feb 2013 14:31:49 -0000 Subject: No subject Message-ID: ... - maildir: change it to use '/' as hierarchy separator to allow '.' characters (for usernames in shared folders) ... Now changing the separator in the maildir-storage.c is trivial (even though it "can't" be changed :-), but fixing the resultant breakage in maildir-list.c needs a little more understanding. Timo, could you give me some pointers? I can see that I need to work out what imap_match_init() does. My motivation is to follow up on an earlier thread about reusing .mailboxlist files as .subscription files, preferably without rewriting them to replace the slashes with something else. Thanks -- Charlie From ben at morrow.me.uk Fri Feb 1 00:35:42 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 31 Jan 2013 22:35:42 +0000 Subject: [Dovecot] Userdb passwd and 'nologin' users Message-ID: <20130131223541.GC25874@anubis.morrow.me.uk> I am running Dovecot with system users (userdb passwd), but some of those users don't have shell accounts on the IMAP server so their shell on that machine is set to /usr/sbin/nologin. Currently I am using maildirs and this is not a problem, but I am in the process of switching to dbox which means I will need a cronjob running 'doveadm purge -A'. During testing I found that those users with a 'nologin' shell are not included in the list returned by the userdb iterator, and that the iterator doesn't honour the first/last_valid_uid settings. This inconsistency seems undesirable, so the attached patch - makes lookup perform the same checks as iteration, - makes the 'nologin' check configurable, - adds a new optional check that the user owns their home directory. The last check was the one performed by qmail, and seems to me to be a more reliable 'is this a real user' check than a nologin shell. If this patch is applied, the release notes for the next release should probably mention that system users with a 'nologin' shell will no longer be allowed to log in to IMAP until the 'auth_check_nologin' setting is changed from true to false. Also, there seem to be two first/last_valid_uid settings: first_valid_uid itself, which is honoured by the storage subsystem, and auth_first_valid_uid, which is honoured by the 'passwd' userdb. Is this intentional? Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: userdb-passwd-nologin.patch Type: text/x-diff Size: 4203 bytes Desc: not available URL: From ben at indietorrent.org Fri Feb 1 00:49:14 2013 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 31 Jan 2013 17:49:14 -0500 Subject: [Dovecot] Reviewing end-user ham/spam submissions before feeding them to sa-learn via Dovecot Antispam plug-in In-Reply-To: References: <50F6D7A6.50307@indietorrent.org> Message-ID: <510AF4EA.5050401@indietorrent.org> On 1/17/2013 4:31 AM, Steffen Kaiser wrote: > On Wed, 16 Jan 2013, Ben Johnson wrote: > >> Currently, I'm using the Dovecot Antispam plug-in with great >> success. Everything works as expected. > >> However, I would like to change the plug-in's behavior such that >> it simply sends a copy of a message that is moved from Inbox -> >> Junk (or Junk -> Inbox) to an administrator, instead of calling >> sa-learn (I'm using SpamAssassin) automatically. > >> Basically, I would like to be able to review messages that >> end-users submit for training before they are actually fed to >> sa-learn. > > Hmm, if you use Maildir: > > plugin { antispam_backend = spool2dir antispam_allow_append_to_spam > = true [snip] antispam_spool2dir_spam = > /path/to/admin/Maildir/.TrainingReview.spam/new/%%020lu-%%05lu > antispam_spool2dir_notspam = > /path/to/admin/Maildir/.TrainingReview.not_spam/new/%%020lu-%%05lu > } > > any of your mail users need write permission those directories, the > admin needs read permissions for the spooled files, you need some > method to pass the reviewed messages to sa-learn. > > IMHO, for 3rd step: I would either flag messages to be learned or > move them into another folder. And a cron job feeds them to > sa-learn, so they do not get into the message queue again and are > probably re-filterred or modified before sa-learn picks them up. > > If you do not have Maildir to drop the files to or get problems, > because the messages appear in "new" before they are spooled on > disk and hence they may get indexed wrongly, because their content > changes (which is forbidden in IMAP), you could spool them into > > antispam_spool2dir_spam = /tmp/spamspool/spam/%%020lu-%%05lu-%u > antispam_spool2dir_notspam = /tmp/spamspool/ham/%%020lu-%%05lu-%u > -or- antispam_spool2dir_spam = /tmp/spamspool/%%020lu-%%05lu-%u-S > antispam_spool2dir_notspam = /tmp/spamspool/%%020lu-%%05lu-%u-H > > and have another cron job or inotify-wrapper move the messages to > the reviewer's mailbox. In this case, you can make use of the "%u" > component, which is expanded by the username, who spooled the > message. So you could ignore messages of some users and/or ... . > > Kind regards, > > -- Steffen Kaiser > Steffen, thank you very much for the thorough reply. I apologize for taking so long to digest it and respond. I am using the Maildir format indeed. >> any of your mail users need write permission those directories, >> the admin needs read permissions for the spooled files, By "mail users", do you mean, e.g., the "vmail" user account (I'm on Debian/Ubuntu)? My understanding is that the "vmail" user account handles all IMAP transactions; if this is true, then are you saying that the only requisite to your suggestions is that the "vmail" user has read/write access to the following two directories? /path/to/admin/Maildir/.TrainingReview.spam/new/ and /path/to/admin/Maildir/.TrainingReview.not_spam/new/ >> you need some method to pass the reviewed messages to sa-learn. In the past, I have simply sorted the messages into "Ham" and "Spam" sub-folders of the admin's training Inbox, and called sa-learn, with the appropriate --ham/--spam switch on each, using a cron job. It sounds as though this is what you are suggesting, and I can continue with this approach. I went ahead and tried to reconfigure Dovecot's Antispam plug-in to use the spool2dir backend, but I'm receiving a less-than-helpful message from the plug-in when I try to move a message from Inbox to Junk or vice versa: "CANNOT: antispam plugin not configured". Initially, I was using paths to the admin's mailbox, as demonstrated in your initial response, but I simplified the spool location in order to eliminate permission problems. Please note that I am using Dovecot 1.2.9 in Ubuntu 10.04 LTS. By extension, I am using the Antispam plug-in for Dovecot 1 (not 2), the manpage for which is at http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html . So, the configuration option names and expected values differ slightly from those in your example. These are the only configuration directives that I am using: ------------------------------------- plugin { # Unrelated plugin options here [...] antispam_spam_pattern_ignorecase = SPAM;JUNK # Is this next directive actually required for this approach? antispam_allow_append_to_spam = yes antispam_spool2dir_spam = /tmp/spamspool/%%020lu-%u-%%05lus antispam_spool2dir_notspam = /tmp/spamspool/%%020lu-%u-%%05luh } ------------------------------------- The spool directory that is specified exists and, for testing purposes, has 0777 permissions: # ls -lah /tmp | grep "spamspool" drwxrwxrwx 2 root root 4.0K Jan 31 14:22 spamspool Based on the above-cited manpage, these are be the only options that are required (perhaps with the exception of antispam_allow_append_to_spam). What have I overlooked here? Thanks again for all your help! -Ben From tss at iki.fi Fri Feb 1 01:03:38 2013 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Feb 2013 01:03:38 +0200 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: <20130131223541.GC25874@anubis.morrow.me.uk> References: <20130131223541.GC25874@anubis.morrow.me.uk> Message-ID: On 1.2.2013, at 0.35, Ben Morrow wrote: > I am running Dovecot with system users (userdb passwd), but some of > those users don't have shell accounts on the IMAP server so their shell > on that machine is set to /usr/sbin/nologin. Currently I am using > maildirs and this is not a problem, but I am in the process of switching > to dbox which means I will need a cronjob running 'doveadm purge -A'. > > During testing I found that those users with a 'nologin' shell are not > included in the list returned by the userdb iterator, and that the > iterator doesn't honour the first/last_valid_uid settings. This > inconsistency seems undesirable, so the attached patch > > - makes lookup perform the same checks as iteration, Hmmh. You could also just have them aliased to other users, so this wouldn't be necessary.. > - makes the 'nologin' check configurable, > - adds a new optional check that the user owns their home directory. These settings are passwd-specific, so they would have to something like: userdb { driver = passwd args = check-nologin=n check-home=y } > The last check was the one performed by qmail, and seems to me to be a > more reliable 'is this a real user' check than a nologin shell. It also performs disk I/O, slowing down the lookup. > If this patch is applied, the release notes for the next release should > probably mention that system users with a 'nologin' shell will no longer > be allowed to log in to IMAP until the 'auth_check_nologin' setting is > changed from true to false. The default will in any case be the same as it is now. > Also, there seem to be two first/last_valid_uid settings: > first_valid_uid itself, which is honoured by the storage subsystem, and > auth_first_valid_uid, which is honoured by the 'passwd' userdb. Is this > intentional? Nope, that's a bug. Fixed that in v2.2: http://hg.dovecot.org/dovecot-2.2/rev/18661d1d6ed0 From daniel.parthey at informatik.tu-chemnitz.de Fri Feb 1 05:17:42 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 1 Feb 2013 04:17:42 +0100 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: <20130131223541.GC25874@anubis.morrow.me.uk> References: <20130131223541.GC25874@anubis.morrow.me.uk> Message-ID: <20130201031742.GA18862@daniel.localdomain> Hi Ben, Ben Morrow wrote: > + if (set->check_nologin) { > + /* skip entries that don't have a valid shell. > + they're again probably not real users. */ > + if (strcmp(pw->pw_shell, "/bin/false") == 0 || > + strcmp(pw->pw_shell, "/sbin/nologin") == 0 || > + strcmp(pw->pw_shell, "/usr/sbin/nologin") == 0) > + return FALSE; > + } Valid shells are defined in /etc/shells and "locked" users, I would strongly discourage from hardcoding a list of no-login shells here. Users locked with "passwd -l" can also be detected by a ! at the beginning of the password hash. Regards Daniel -- https://plus.google.com/103021802792276734820 From dovecot at lists.wgwh.ch Fri Feb 1 14:28:25 2013 From: dovecot at lists.wgwh.ch (Oli Schacher) Date: Fri, 1 Feb 2013 13:28:25 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <1359663448.3230.0.camel@hurina> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> Message-ID: <20130201132825.5363b1f4@boscos> On Thu, 31 Jan 2013 22:17:28 +0200 Timo Sirainen wrote: > On Thu, 2013-01-31 at 21:51 +0200, Timo Sirainen wrote: > > On 31.1.2013, at 19.41, Oli Schacher wrote: > > > > >>> Jan 31 17:13:11 doco1 dovecot: dsync-local(user1): Error: > > >>> Mailbox INBOX: Remote didn't send mail > > >>> GUID=33dabe0f11980a51200c0000960042f4 (UID=104) > > > > I guess there's some bug that causes this to happen in some > > situations.. But the reason for mail duplication should be fixed > > by: http://hg.dovecot.org/dovecot-2.2/rev/138f1c76c0ec > > > > Except that shouldn't have been necessary. doveadm-server returns > > success before it has finished running dsync. Not sure why, need to > > debug it further. > > Fixed with a bit of a kludge: > http://hg.dovecot.org/dovecot-2.2/rev/e9e6a95cea21 > > I can confirm that it has become significantly harder to produce errors with the latest patches. There still seems to be a problem when changes to both mailboxes at the same time are involved, however, today I didn't have time to test "scientifically", i just updated to latest hg and clicked around, so this report probably won't be of much use to you,sorry. I'll try to make reproducible tests again next week. I'll post the errors from my clicking session anyway, maybe it helps you figuring out what went wrong even without knowing how to reproduce. At least the "Operation not permitted" error below when killing the dsync process sounds unintended? Logoutput is from changeset 78bdcb6642c7 running on both servers. Server 1: Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=211) Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=205) Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=208) Feb 1 07:12:54 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=205: 7a30ff22af5b0b510f0c0000960042f4 != 8230ff22af5b0b510f0c0000960042f4 Feb 1 07:12:54 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=228) [...] Feb 1 07:12:55 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Importing mailbox INBOX failed Feb 1 07:12:56 doco1 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.62) failed: EOF Feb 1 07:12:56 doco1 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.62) failed: Broken pipe Feb 1 07:12:56 doco1 dovecot: dsync-local(user1): Error: Remote command returned error 75 [...] Feb 1 07:12:57 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=291: 7b30ff22af5b0b510f0c0000960042f4 != 8d30ff22af5b0b510f0c0000960042f4 Feb 1 07:12:57 doco1 dovecot: doveadm: Error: dsync-remote(user1): Panic: file dsync-mailbox-import.c: line 1112 (dsync_mailbox_import_change): assertion failed: (change->type == DSYNC_MAIL_CHANGE_TYPE_SAVE) Feb 1 07:12:57 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x5d4ea) [0x7f19cf5954ea] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f19cf5955d2] -> /usr/lib64/dovecot/libdovecot.so.0(+0x1f6ca) [0x7f19cf5576ca] -> /usr/bin/doveadm(dsync_mailbox_import_change+0x501) [0x42c881] -> /usr/bin/doveadm(dsync_brain_sync_mails+0x3a2) [0x4290c2] -> /usr/bin/doveadm(dsync_brain_run+0x169) [0x425e29] -> /usr/bin/doveadm() [0x426380] -> /usr/bin/doveadm() [0x434aa0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f19cf5a4076] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f19cf5a5107] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f19cf5a4018] -> /usr/bin/doveadm() [0x424134] -> /usr/bin/doveadm() [0x40fe4f] -> /usr/bin/doveadm() [0x41067d] -> /usr/bin/doveadm(doveadm_mail_try_run+0x141) [0x410ba1] -> /usr/bin/doveadm(main+0x3f1) [0x417bc1] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f19cf1c3cdd] -> /usr/bin/doveadm() [0x40f839] Feb 1 07:12:57 doco1 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.62) failed: EOF Server 2: Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=205: 7a30ff22af5b0b510f0c0000960042f4 != 8230ff22af5b0b510f0c0000960042f4 Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=228) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=234) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7b30ff22af5b0b510f0c0000960042f4 (UID=238) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7c30ff22af5b0b510f0c0000960042f4 (UID=256) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7c30ff22af5b0b510f0c0000960042f4 (UID=235) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7c30ff22af5b0b510f0c0000960042f4 (UID=239) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=255) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=226) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=237) Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Importing mailbox INBOX failed Feb 1 07:13:24 doco2 dovecot: dsync-local(user1): Error: Remote command process isn't dying, killing it Feb 1 07:13:24 doco2 dovecot: dsync-local(user1): Error: kill() failed: Operation not permitted -- message transmitted on 100% recycled electrons From skdovecot at smail.inf.fh-brs.de Fri Feb 1 15:45:50 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 1 Feb 2013 14:45:50 +0100 (CET) Subject: [Dovecot] Reviewing end-user ham/spam submissions before feeding them to sa-learn via Dovecot Antispam plug-in In-Reply-To: <510AF4EA.5050401@indietorrent.org> References: <50F6D7A6.50307@indietorrent.org> <510AF4EA.5050401@indietorrent.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 31 Jan 2013, Ben Johnson wrote: > On 1/17/2013 4:31 AM, Steffen Kaiser wrote: >> On Wed, 16 Jan 2013, Ben Johnson wrote: >> > I am using the Maildir format indeed. > >>> any of your mail users need write permission those directories, >>> the admin needs read permissions for the spooled files, > > By "mail users", do you mean, e.g., the "vmail" user account (I'm on > Debian/Ubuntu)? My understanding is that the "vmail" user account Yes, I mean that Unix account, Dovecot accesses the fils with. In a setup with virtual users "vmail" makes sense. > handles all IMAP transactions; if this is true, then are you saying > that the only requisite to your suggestions is that the "vmail" user > has read/write access to the following two directories? > > /path/to/admin/Maildir/.TrainingReview.spam/new/ > > and > > /path/to/admin/Maildir/.TrainingReview.not_spam/new/ Yes. >>> you need some method to pass the reviewed messages to sa-learn. > > In the past, I have simply sorted the messages into "Ham" and "Spam" > sub-folders of the admin's training Inbox, and called sa-learn, with > the appropriate --ham/--spam switch on each, using a cron job. It > sounds as though this is what you are suggesting, and I can continue That's what I mean. > with this approach. > > I went ahead and tried to reconfigure Dovecot's Antispam plug-in to > use the spool2dir backend, but I'm receiving a less-than-helpful > message from the plug-in when I try to move a message from Inbox to > Junk or vice versa: "CANNOT: antispam plugin not configured". > > Please note that I am using Dovecot 1.2.9 in Ubuntu 10.04 LTS. By Oh, I have no experience with Dovecot v1.2; in v1.0 you have to compile one particular backend into antispam-plugin. Maybe, distributors have another, non-Dovecot way to select between the backends. > extension, I am using the Antispam plug-in for Dovecot 1 (not 2), the > manpage for which is at > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > . So, the configuration option names and expected values differ > slightly from those in your example. The man-page contains: "INSTALLATION First copy the ?defconfig? file to ?.config? and edit it as necessary. You need to have the dovecot headers installed and possibly other things depending on the backend you choose. Then, assuming you have configured the INSTALLDIR correctly, simply run ?make install?. " - -and- "BACKENDS The plugin supports multiple backends, there are currently two working backends included in the distribution: " This suggests that my guess is true - although I don't understand the "there are currently two backends included" part. ====== So you could re-compile another antispam-plugin with spool2dir backend, or - - you are using these settings, right? #===================== # mail sending plugin # # Because of the way this plugin works, you can also use it # to train via an arbitrary program that receives the message # on standard input, in that case you can use the config # options antispam_mail_spam and antispam_mail_notspam for # the argument that distinguishes between ham and spam. # For example: # antispam_mail_sendmail = /path/to/mailtrain # antispam_mail_sendmail_args = --for;%u # antispam_mail_spam = --spam # antispam_mail_notspam = --ham change antispam_mail_sendmail into a script, that drops the message into the correct mail folder, e.g.: #!/bin/bash mode= for opt; do if test "x$opt" == x--ham; then mode=HAM break elif test "x$opt" == x--ham; then mode=SPAM break fi done if test -n "$mode"; then # options from http://wiki1.dovecot.org/LDA /path/to/dovecot-deliver -d spamadmin -m Training.$mode fi This sends the message bypassing a MTA to the spamadmin user. Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUQvHDl3r2wJMiz2NAQJEeAf/XmxSzh+cPqviAax/ucThVaYfygrARz6G qXRLbea/8fnhlRfO2seL75tElDmRsirVXGPu5awpf0WUEzFD96HWrmcrKMMRPfyE uFylqzVB2dnmc+KOLolGb08hKRooMOaTPQt1Y9eVDGAplQM8PoNu+K3QE+rqkCbf OJiL2pxJrEbiTOxzVhFSgUY/VdJVYLUBY4BpC5iZp7nNvXvub4scNlcd7OX0T1Kj nlPnjpw2eNWX+UBCmjbfuVQKVFLBIFQFL9gnxZMphCMzjjYYgPaGHpSBlO00C+aM ddiR46SrcjJIP4pXZsJyf5xw5aOCIUk2PXGr4aQFj409rcVJaK3CsQ== =NgZB -----END PGP SIGNATURE----- From micah at riseup.net Fri Feb 1 18:09:28 2013 From: micah at riseup.net (micah anderson) Date: Fri, 01 Feb 2013 11:09:28 -0500 Subject: [Dovecot] dsync timeout? In-Reply-To: <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> References: <87libacola.fsf@minnow.riseup.net> <3B764B67-F536-4BB3-BF2A-4777F4B6B3F0@iki.fi> <87halyck05.fsf@minnow.riseup.net> <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> Message-ID: <87sj5gaudj.fsf@minnow.riseup.net> Sean Kamath writes: > On Jan 30, 2013, at 3:46 PM, micah anderson wrote: >> Seems that only the above process was still around and no other dsync >> processes. I have three machines that all have this happening it seems. >> >> I wonder if there is a ssh configuration option I could set to make >> these die off. > > If the ssh process isn't sending anything, and just waiting for read()s, and keepalives are turned off, the SSH session might never know the remote side is long gone. . . > > If any data were transmitted, it would discover the remote side is turned off. > > See man ssh_config and the option TCPKeepAlive. > > BTW: Since it's not on the command line, it's likely in /etc/ssh_config or /etc/ssh/ssh_config. Or ~/.ssh/config. In /etc/ssh/sshd_config on the server I'm sending to, TCPKeepAlive yes is set. The default on this system, according to the man page, seems to be to have TCPKeepAlive set. Perhaps I should set ServerAliveInterval? micah From ben at indietorrent.org Fri Feb 1 18:18:12 2013 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 01 Feb 2013 11:18:12 -0500 Subject: [Dovecot] Reviewing end-user ham/spam submissions before feeding them to sa-learn via Dovecot Antispam plug-in In-Reply-To: References: <50F6D7A6.50307@indietorrent.org> <510AF4EA.5050401@indietorrent.org> Message-ID: <510BEAC4.8080207@indietorrent.org> On 2/1/2013 8:45 AM, Steffen Kaiser wrote: > On Thu, 31 Jan 2013, Ben Johnson wrote: > >> On 1/17/2013 4:31 AM, Steffen Kaiser wrote: >>> On Wed, 16 Jan 2013, Ben Johnson wrote: >>> >> I am using the Maildir format indeed. > >>>> any of your mail users need write permission those >>>> directories, the admin needs read permissions for the spooled >>>> files, > >> By "mail users", do you mean, e.g., the "vmail" user account (I'm >> on Debian/Ubuntu)? My understanding is that the "vmail" user >> account > > Yes, I mean that Unix account, Dovecot accesses the fils with. In > a setup with virtual users "vmail" makes sense. > >> handles all IMAP transactions; if this is true, then are you >> saying that the only requisite to your suggestions is that the >> "vmail" user has read/write access to the following two >> directories? > >> /path/to/admin/Maildir/.TrainingReview.spam/new/ > >> and > >> /path/to/admin/Maildir/.TrainingReview.not_spam/new/ > > Yes. > >>>> you need some method to pass the reviewed messages to >>>> sa-learn. > >> In the past, I have simply sorted the messages into "Ham" and >> "Spam" sub-folders of the admin's training Inbox, and called >> sa-learn, with the appropriate --ham/--spam switch on each, using >> a cron job. It sounds as though this is what you are suggesting, >> and I can continue > > That's what I mean. > >> with this approach. > >> I went ahead and tried to reconfigure Dovecot's Antispam plug-in >> to use the spool2dir backend, but I'm receiving a >> less-than-helpful message from the plug-in when I try to move a >> message from Inbox to Junk or vice versa: "CANNOT: antispam >> plugin not configured". > >> Please note that I am using Dovecot 1.2.9 in Ubuntu 10.04 LTS. >> By > > Oh, I have no experience with Dovecot v1.2; in v1.0 you have to > compile one particular backend into antispam-plugin. Maybe, > distributors have another, non-Dovecot way to select between the > backends. > >> extension, I am using the Antispam plug-in for Dovecot 1 (not 2), >> the manpage for which is at >> http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html >> >> . So, the configuration option names and expected values differ >> slightly from those in your example. > > The man-page contains: > > "INSTALLATION > > First copy the ?defconfig? file to ?.config? and edit it as > necessary. You need to have the dovecot headers installed and > possibly other things depending on the backend you choose. Then, > assuming you have configured the INSTALLDIR correctly, simply run > ?make install?. " > > -and- > > "BACKENDS > > The plugin supports multiple backends, there are currently two > working backends included in the distribution: " > > This suggests that my guess is true - although I don't understand > the "there are currently two backends included" part. > > ====== > > So you could re-compile another antispam-plugin with spool2dir > backend, or - - you are using these settings, right? > > #===================== # mail sending plugin # # Because of the way > this plugin works, you can also use it # to train via an arbitrary > program that receives the message # on standard input, in that case > you can use the config # options antispam_mail_spam and > antispam_mail_notspam for # the argument that distinguishes between > ham and spam. # For example: # antispam_mail_sendmail = > /path/to/mailtrain # antispam_mail_sendmail_args = --for;%u # > antispam_mail_spam = --spam # antispam_mail_notspam = --ham > > change antispam_mail_sendmail into a script, that drops the > message into the correct mail folder, e.g.: > > #!/bin/bash > > mode= for opt; do if test "x$opt" == x--ham; then mode=HAM break > elif test "x$opt" == x--ham; then mode=SPAM break fi done > > if test -n "$mode"; then # options from > http://wiki1.dovecot.org/LDA /path/to/dovecot-deliver -d spamadmin > -m Training.$mode fi > > This sends the message bypassing a MTA to the spamadmin user. > > Kind regards, > > -- Steffen Kaiser Steffen, It seems you're correct in that the spool2dir back-end isn't included with version 1 of the plug-in. I like the idea of using a pipe script better than changing the back-end. Thank you for providing a solid example; it works beautifully with a couple of small changes. Here's the final script: --------------------------------------- #!/bin/bash mode= for opt; do if test "x$*" == "x--ham"; then mode=HAM break elif test "x$*" == "x--spam"; then mode=SPAM break fi done if test -n "$mode"; then # options from http://wiki1.dovecot.org/LDA /usr/lib/dovecot/deliver -d user at example.com -m Training.$mode fi exit 0 --------------------------------------- For anyone who is curious, here are the Antispam plug-in options: --------------------------------------- # For Dovecot < 2.0. antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_mail_tmpdir = /tmp antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham --------------------------------------- Looks like we're in business! Thanks again for sharing your valuable insights; your help is greatly appreciated. -Ben From frank at mclien.de Fri Feb 1 18:22:52 2013 From: frank at mclien.de (Frank Lienhard) Date: Fri, 01 Feb 2013 17:22:52 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? Message-ID: <510BEBDC.6020802@mclien.de> I'm about to replace my old server. Both servers are on my internal network. The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. I set up the users on both systems identically (same gid uid). My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the maildir format, but this results in ~30% doublicated mails on the new server. I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. So which could be the most painless way to solve that? From h.reindl at thelounge.net Fri Feb 1 18:25:35 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 01 Feb 2013 17:25:35 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BEBDC.6020802@mclien.de> References: <510BEBDC.6020802@mclien.de> Message-ID: <510BEC7F.20409@thelounge.net> Am 01.02.2013 17:22, schrieb Frank Lienhard: > I'm about to replace my old server. Both servers are on my internal network. > The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. > I set up the users on both systems identically (same gid uid). > My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the > maildir format, but this results in ~30% doublicated mails on the new server. > I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. > So which could be the most painless way to solve that? imapsync is version agnostic and if you have a database with the user-logins it should be easy to geenrate a shellscript i did migration of around 200000 messages and some hundret users from Apple OSX EIMS to dbmail/dovecot in 2009 this way -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From janfrode at tanso.net Fri Feb 1 19:00:07 2013 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 1 Feb 2013 18:00:07 +0100 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? Message-ID: We upgraded our two dovecot directors from v2.0.14 to dovecot-ee 2.1.10.3 this week, and after that mail seems to be flowing a lot slower than before. The backend mailstores are untouched, on v2.0.14 still. After the upgrade we've been hitting process_limit for lmtp a lot, and we're struggeling with large queues in the incoming mailservers that are using LMTP virtual transport towards our two directors. I seem to remember 2.1 should have a new lmtp-proxying code. Is there anything in this that maybe needs to be tuned that's different from v2.0 ? I'm a bit scheptical to just increasing the process_limit for LMTP proxying, as I doubt running many hundreds of simultaneous deliveries should work that much better against the backend storage.. ###### doveconf -n ########## # 2.1.10.3: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.32.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) default_client_limit = 4000 director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 disable_plaintext_auth = no listen = * lmtp_proxy = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = proxy=y nopassword=y driver = static } protocols = imap pop3 lmtp sieve service anvil { client_limit = 6247 } service auth { client_limit = 8292 unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 4096 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { address = * port = 24 } process_limit = 100 } service managesieve-login { executable = managesieve-login director inet_listener sieve { address = * port = 4190 } process_limit = 50 } service pop3-login { executable = pop3-login director process_limit = 2048 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } ssl_cert = References: <510BEBDC.6020802@mclien.de> <510BEC7F.20409@thelounge.net> Message-ID: <510BF90D.4000709@mclien.de> Reindl Harald wrote: > Am 01.02.2013 17:22, schrieb Frank Lienhard: > >> I'm about to replace my old server. Both servers are on my internal network. >> The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. >> I set up the users on both systems identically (same gid uid). >> My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the >> maildir format, but this results in ~30% doublicated mails on the new server. >> I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't work. >> So which could be the most painless way to solve that? >> > > imapsync is version agnostic and if you have a database > with the user-logins it should be easy to geenrate a shellscript > > i did migration of around 200000 messages and some hundret users > from Apple OSX EIMS to dbmail/dovecot in 2009 this way > > sadly, this is not availble from the debian repos. I searched a bit around: syncmaildir, imapsync: only available in in Lenny OR Wheezy, but needed on both servers isync, mailsync: mentioned to hold two maildirs in sync with a more or less difficult to configure setup which leaves me one option (from programms I found); offlineimap: mentioned to backup mails from a remote/ISP mailserver in a local directory of a user. Wonder if that works , when the local dir is one of an imap server....... From h.reindl at thelounge.net Fri Feb 1 19:22:12 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 01 Feb 2013 18:22:12 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BF90D.4000709@mclien.de> References: <510BEBDC.6020802@mclien.de> <510BEC7F.20409@thelounge.net> <510BF90D.4000709@mclien.de> Message-ID: <510BF9C4.4070907@thelounge.net> Am 01.02.2013 18:19, schrieb Frank Lienhard: > Reindl Harald wrote: >> Am 01.02.2013 17:22, schrieb Frank Lienhard: >> >>> I'm about to replace my old server. Both servers are on my internal network. >>> The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. >>> I set up the users on both systems identically (same gid uid). >>> My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with the >>> maildir format, but this results in ~30% doublicated mails on the new server. >>> I then learned about doveadm/dsync, only to figure out that this isn't available in v 1.0.15 and therefor won't >>> work. >>> So which could be the most painless way to solve that? >>> >> >> imapsync is version agnostic and if you have a database >> with the user-logins it should be easy to geenrate a shellscript >> >> i did migration of around 200000 messages and some hundret users >> from Apple OSX EIMS to dbmail/dovecot in 2009 this way >> >> > sadly, this is not availble from the debian repos. > I searched a bit around: > syncmaildir, imapsync: only available in in Lenny OR Wheezy, but needed on both servers needed on both servers is simply not true i have even a template in my admin-backend to generate imapsync scripts out of a db imapsync --host1 [source] --user1 [user1] --password1 [pwd1] --authmech1 LOGIN --host2 [target] --user2 [user2] --password2 [pwd2] --authmech2 LOGIN --skipsize -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From ben at morrow.me.uk Fri Feb 1 20:31:27 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 1 Feb 2013 18:31:27 +0000 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: References: <20130131223541.GC25874@anubis.morrow.me.uk> Message-ID: <20130201183126.GD25874@anubis.morrow.me.uk> At 1AM +0200 on 1/02/13 you (Timo Sirainen) wrote: > On 1.2.2013, at 0.35, Ben Morrow wrote: > > > I am running Dovecot with system users (userdb passwd), but some of > > those users don't have shell accounts on the IMAP server so their shell > > on that machine is set to /usr/sbin/nologin. Currently I am using > > maildirs and this is not a problem, but I am in the process of switching > > to dbox which means I will need a cronjob running 'doveadm purge -A'. > > > > During testing I found that those users with a 'nologin' shell are not > > included in the list returned by the userdb iterator, and that the > > iterator doesn't honour the first/last_valid_uid settings. This > > inconsistency seems undesirable, so the attached patch > > > > - makes lookup perform the same checks as iteration, > > Hmmh. You could also just have them aliased to other users, so this > wouldn't be necessary.. I don't understand what you mean. Alias them where? > > - makes the 'nologin' check configurable, > > - adds a new optional check that the user owns their home directory. > > These settings are passwd-specific, so they would have to something like: > > userdb { > driver = passwd > args = check-nologin=n check-home=y > } OK. New patch attached. > > The last check was the one performed by qmail, and seems to me to be a > > more reliable 'is this a real user' check than a nologin shell. > > It also performs disk I/O, slowing down the lookup. Hmm. OK, I've left that part out: my real users are segregated by UID anyway, so all I really care about is getting rid of the nologin check. (I would be perfectly happy if the check were just removed altogether.) > > If this patch is applied, the release notes for the next release should > > probably mention that system users with a 'nologin' shell will no longer > > be allowed to log in to IMAP until the 'auth_check_nologin' setting is > > changed from true to false. > > The default will in any case be the same as it is now. Well, yes; but authentication will now check for a nologin shell by default, which it didn't before, so the visible behaviour will have changed. > > Also, there seem to be two first/last_valid_uid settings: > > first_valid_uid itself, which is honoured by the storage subsystem, and > > auth_first_valid_uid, which is honoured by the 'passwd' userdb. Is this > > intentional? > > Nope, that's a bug. Fixed that in v2.2: > http://hg.dovecot.org/dovecot-2.2/rev/18661d1d6ed0 Cool. Will that be backported to 2.1 at some point? Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: userdb-passwd-nologin2.patch Type: text/x-diff Size: 4142 bytes Desc: not available URL: From ben at morrow.me.uk Fri Feb 1 20:37:46 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 1 Feb 2013 18:37:46 +0000 Subject: [Dovecot] Userdb passwd and 'nologin' users In-Reply-To: <20130201031742.GA18862@daniel.localdomain> References: <20130131223541.GC25874@anubis.morrow.me.uk> <20130201031742.GA18862@daniel.localdomain> Message-ID: <20130201183746.GE25874@anubis.morrow.me.uk> At 4AM +0100 on 1/02/13 you (Daniel Parthey) wrote: > Hi Ben, > > Ben Morrow wrote: > > + if (set->check_nologin) { > > + /* skip entries that don't have a valid shell. > > + they're again probably not real users. */ > > + if (strcmp(pw->pw_shell, "/bin/false") == 0 || > > + strcmp(pw->pw_shell, "/sbin/nologin") == 0 || > > + strcmp(pw->pw_shell, "/usr/sbin/nologin") == 0) > > + return FALSE; > > + } > > Valid shells are defined in /etc/shells and "locked" users, I would > strongly discourage from hardcoding a list of no-login shells here. That list isn't mine, my patch just moves that code from one part of the file to another and makes it conditional. Personally I don't think checking the shell is sensible at all, which is why I'm trying to make it optional. > Users locked with "passwd -l" can also be detected by a ! at > the beginning of the password hash. That is system-specific, and in any case you have to be root (and on non-BSD systems you have to make a shadow password call) to see the password field. The userdb shouldn't be doing that: locked users will already be prevented from logging in with a password because the password won't match. (Of course, this doesn't cover e.g. Kerberos logins, though I would usually lock a Kerberos user by telling the KDC not to issue tickets rather than by locking the passwd account.) Ben From trashcan at odo.in-berlin.de Fri Feb 1 22:39:09 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 1 Feb 2013 20:39:09 +0000 (UTC) Subject: [Dovecot] cmsg cancel Message-ID: ignore Article cancelled by slrn 1.0.1 From trashcan at odo.in-berlin.de Fri Feb 1 22:53:55 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 1 Feb 2013 21:53:55 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <20130201132825.5363b1f4@boscos> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> Message-ID: [Sorry Oli for my previous mail to your address, only. Resent here] Oli Schacher wrote: > There still seems to be a problem when changes to both mailboxes at > the same time are involved I can confirm your observation, although triggered by a different test scenario, similar to the one I did use with 2.1 replicator before (http://www.dovecot.org/list/dovecot/2012-March/064354.html). This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test" at both servers "mx1" and "mx2", and replicator uses ssh for remote access. Both servers run a recent postfix, use lmtp for local delivery, and "test" is a virtual user. Test script to produce local testmails of equal size at mx1: | #!/bin/csh | set INDEX = 101 | set endINDEX = 200 | while ( $INDEX <= $endINDEX ) | echo $INDEX | echo "test" | mail -s $INDEX test at mx1 | if ( $INDEX % 1000 == 0 ) then | sleep 1 | endif | @ INDEX = $INDEX + 1 |end |exit 0 Test script to produce testmails of equal size at mx2: | #!/bin/csh | set INDEX = 1101 | set endINDEX = 1200 | while ( $INDEX <= $endINDEX ) | echo $INDEX | echo "test" | mail -s $INDEX test at mx2 | if ( $INDEX % 1000 == 0 ) then | sleep 1 | endif | @ INDEX = $INDEX + 1 |end |exit 0 All tests are run with vanilla mailboxes, after restarting dovecot, and without imap connections by MUA: 1) Simultaneous mailbomb approach: run both scripts simultaneously, and you'll end up with numerous duplicates in mailboxes "test". Very often you'll find multiples. 2) Mailbomb approach: run one script at one server only, and all mails will become perfectly well synchronised. 3) Mofify both scripts to "( $INDEX % 1 == 0 )" to add a second waiting between every mail injection, and run them simultaneously at both servers, and you'll end up with significantly less duplicates and no more multiples. > Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=211) > Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Importing mailbox INBOX failed > Feb 1 07:13:24 doco2 dovecot: dsync-local(user1): Error: Remote command process isn't dying, killing it I do see those error messages as well, and in addition numerous of those: | dovecot: dsync-local(test): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=7153: 82c5df0a4ffa0b5141e300006a0d5a02 != 29cc9f284ffa0b5141c2000036abecbd | doveadm: Error: dsync-remote(test): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=7153: 82c5df0a4ffa0b5141e300006a0d5a02 != 29cc9f284ffa0b5141c2000036abecbd | dovecot: lmtp(49752, test): Error: Corrupted index cache file /.../test/mailboxes/INBOX/dbox-Mails/dovecot.index.cache: File too small | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: mdbox /.../test/mailboxes/INBOX/dbox-Mails: Storage keeps breaking | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Disconnected: Logged out in=425 out=1100 JFYI, and regards, Michael From tss at iki.fi Sat Feb 2 00:00:56 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 2 Feb 2013 00:00:56 +0200 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: Message-ID: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> On 1.2.2013, at 19.00, Jan-Frode Myklebust wrote: > We upgraded our two dovecot directors from v2.0.14 to dovecot-ee > 2.1.10.3 this week, and after that mail seems to be flowing a lot > slower than before. The backend mailstores are untouched, on v2.0.14 > still. After the upgrade we've been hitting process_limit for lmtp a > lot, and we're struggeling with large queues in the incoming > mailservers that are using LMTP virtual transport towards our two > directors. > > I seem to remember 2.1 should have a new lmtp-proxying code. Is there > anything in this that maybe needs to be tuned that's different from > v2.0 ? I'm a bit scheptical to just increasing the process_limit for > LMTP proxying, as I doubt running many hundreds of simultaneous > deliveries should work that much better against the backend storage.. Hmm. The main difference is that v2.1 writes temporary files to mail_temp_dir. If that's in tmpfs (and probably even if it isn't), it should still be pretty fast.. Have you checked if there's an increase in disk I/O usage, or system cpu usage? Or actually .. It could simply be that in v2.0.15 service lmtp { client_limit } default was changed to 1 (from default_client_limit=1000). This is important with the backend, because writing to message store can be slow, but proxying should be able to handle more than 1 client per process, even with the new temporary file writing. So you could see if it helps to set lmtp { client_limit = 100 } or something. From daniel.parthey at informatik.tu-chemnitz.de Sat Feb 2 04:32:20 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 2 Feb 2013 03:32:20 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BF90D.4000709@mclien.de> References: <510BEBDC.6020802@mclien.de> <510BEC7F.20409@thelounge.net> <510BF90D.4000709@mclien.de> Message-ID: <20130202023220.GA11927@daniel.localdomain> Hi Frank, Frank Lienhard wrote: > Reindl Harald wrote: > >Am 01.02.2013 17:22, schrieb Frank Lienhard: > >>I'm about to replace my old server. Both servers are on my internal network. > >>The old one ist running debian lenny(32bit) with dovecot 1.0.15 and the new one debian Wheezy with dovecot 2.1.7. > > > > imapsync is version agnostic and if you have a database > > with the user-logins it should be easy to generate a shellscript > > > sadly, this is not availble from the debian repos. imapsync is written in perl and available on github: https://github.com/imapsync/imapsync This is IMHO the best tool where dsync cannot be applied (as in your case). The only drawback of imapsync is that it cannot preserve UIDs since it uploads the messages via IMAP and the server will assign new message UIDs. Regards Daniel -- https://plus.google.com/103021802792276734820 From kamath at moltingpenguin.com Sat Feb 2 10:07:31 2013 From: kamath at moltingpenguin.com (Sean Kamath) Date: Sat, 2 Feb 2013 00:07:31 -0800 Subject: [Dovecot] dsync timeout? In-Reply-To: <87sj5gaudj.fsf@minnow.riseup.net> References: <87libacola.fsf@minnow.riseup.net> <3B764B67-F536-4BB3-BF2A-4777F4B6B3F0@iki.fi> <87halyck05.fsf@minnow.riseup.net> <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> <87sj5gaudj.fsf@minnow.riseup.net> Message-ID: <762DC7CC-1CF5-404E-AF36-41BA08B7C74B@moltingpenguin.com> On Feb 1, 2013, at 8:09 AM, micah anderson wrote: > Sean Kamath writes: > >> On Jan 30, 2013, at 3:46 PM, micah anderson wrote: >>> Seems that only the above process was still around and no other dsync >>> processes. I have three machines that all have this happening it seems. >>> >>> I wonder if there is a ssh configuration option I could set to make >>> these die off. >> >> If the ssh process isn't sending anything, and just waiting for read()s, and keepalives are turned off, the SSH session might never know the remote side is long gone. . . >> >> If any data were transmitted, it would discover the remote side is turned off. >> >> See man ssh_config and the option TCPKeepAlive. >> >> BTW: Since it's not on the command line, it's likely in /etc/ssh_config or /etc/ssh/ssh_config. Or ~/.ssh/config. > > In /etc/ssh/sshd_config on the server I'm sending to, TCPKeepAlive yes > is set. Did you check ~/.ssh/config for the user running the dsync? > The default on this system, according to the man page, seems to be to > have TCPKeepAlive set. > > Perhaps I should set ServerAliveInterval? Perhaps. That states how long to send the KeepAlive packet. There are many settings that can affect this, including ServerAliveCountMax ServerAliveInterval TCPKeepAlive There is also the sshd_config settings ClientAliveCountMax ClientAliveInterval TCPKeepAlive At this point, I think you need to see what's happening on both sides of the SSH connection. I don't recall what system you're on, but for linux you can use netstat -anp (as root) to find out what process is connected to which port, and on linux and other systems you can use lsof to find out what is connected to ports. Maybe the TCP port is open and valid and there's no data coming through? This can happen if, for example, you have any port forwarding or X session forwarding through SSH (i.e., if ssh -X is the default) and something accidentally is holding that port open (this can happen in your regular shell if, for example, you have something open an X application and you forget (because you backgrounded it) -- you're logout of the server will hang until the X applications are closed. Note that it isn't always a visible client that will do this. :-(). Sean From janfrode at tanso.net Sat Feb 2 12:59:04 2013 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sat, 2 Feb 2013 11:59:04 +0100 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: On Fri, Feb 1, 2013 at 11:00 PM, Timo Sirainen wrote: > On 1.2.2013, at 19.00, Jan-Frode Myklebust wrote: > > Have you checked if there's an increase in disk I/O usage, or system cpu usage? > On the directors, cpu usage, and load averages seems to have gone down by about 50% since the upgrade. On the backend mailstores running 2.0.14 I see no effect (but these are quite busy, so less LMTP might just have lead to better response on other services). > Or actually .. It could simply be that in v2.0.15 service lmtp { client_limit } default was changed to 1 (from default_client_limit=1000). This is important with the backend, because writing to message store can be slow, but proxying should be able to handle more than 1 client per process, even with the new temporary file writing. So you could see if it helps to set lmtp { client_limit = 100 } or something. > My backend lmtp services are configured with client_limit = 1, process_limit = 25, and there are 6 backends I.e. max 150 backend LMTP processes if all lmtp is spread evenly between the backends, which it woun't be since backends are weighted differently (2x 50, 2x75 and 2x100). I assume each director will max proxy process_limit*client_limit to my backends. Will it be OK to have a much higher process_limit*client_limit on the directors than on the backends? It will not be a problem if directors are configured to seemingly handle a lot more simultaneous connections than the backends? -jf From tss at iki.fi Sat Feb 2 18:12:07 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 2 Feb 2013 18:12:07 +0200 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: On 2.2.2013, at 12.59, Jan-Frode Myklebust wrote: >> Or actually .. It could simply be that in v2.0.15 service lmtp { client_limit } default was changed to 1 (from default_client_limit=1000). This is important with the backend, because writing to message store can be slow, but proxying should be able to handle more than 1 client per process, even with the new temporary file writing. So you could see if it helps to set lmtp { client_limit = 100 } or something. >> > > My backend lmtp services are configured with client_limit = 1, > process_limit = 25, and there are 6 backends I.e. max 150 backend LMTP > processes if all lmtp is spread evenly between the backends, which it > woun't be since backends are weighted differently (2x 50, 2x75 and > 2x100). > > I assume each director will max proxy process_limit*client_limit to my > backends. Will it be OK to have a much higher > process_limit*client_limit on the directors than on the backends? It > will not be a problem if directors are configured to seemingly handle > a lot more simultaneous connections than the backends? Best to keep the bottleneck closest to MTA. If director can handle more connections than backend, then MTA is uselessly waiting on the extra LMTP connections to timeout. So I'd keep the director's process_limit*client_limit somewhat close to what backends can handle (somewhat more is probably ok too). Anyway, if backend reaches the limit it logs a warning about it and then just doesn't accept the connections until one of the existing ones finish. From joe.beaubien at gmail.com Mon Feb 4 01:07:59 2013 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Sun, 3 Feb 2013 18:07:59 -0500 Subject: [Dovecot] dsync backup questions Message-ID: Hi, I'm currently trying to setup remote backups of my emails but i'm running into issues (mdbox format, indexes and storage in the same folder hierarchy). Local backup command: dsync -u "my_user" backup /backups/my_user (1) Recently, I noticed that the local backup takes up twice the size as the original mail location (8gb vs. 4gb). I purged alot of emails from the original location, so the size shrunk, but the local backup just keeps on getting bigger. I couldn't find any dsync option that would delete extra emails. - Question: Why isn't the local backup synced properly and remove the extra emails? (2) What is the best why to copy this local backup to a remote location that does NOT have the possibility to run dsync. - Question 1: is rsync safe to use and will this data work for restore? - Question 2: Would it be safe to simply rsync the original mail_location to the remote server? I searched the net, but it seems that mdbox is tricky to backup. Thanks in advance, -Joe From ben at morrow.me.uk Mon Feb 4 02:57:29 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 4 Feb 2013 00:57:29 +0000 Subject: [Dovecot] dsync backup questions In-Reply-To: References: Message-ID: <20130204005729.GA68109@anubis.morrow.me.uk> I can't give authoratitive answers to either of these, but... At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: > > I'm currently trying to setup remote backups of my emails but i'm running > into issues (mdbox format, indexes and storage in the same folder > hierarchy). > > Local backup command: dsync -u "my_user" backup /backups/my_user > > (1) Recently, I noticed that the local backup takes up twice the size as > the original mail location (8gb vs. 4gb). I purged alot of emails from the > original location, so the size shrunk, but the local backup just keeps on > getting bigger. I couldn't find any dsync option that would delete extra > emails. > > - Question: Why isn't the local backup synced properly and remove the extra > emails? Are you running 'doveadm purge' on the backed-up dbox? It looks to me as though dsync doesn't do that. I don't know if there's any (simple) way to do that without a running Dovecot instance attached the dbox directory: it's not entirely clear to me whether doveadm will run locally without contacting a doveadm-server instance running under Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. It might be easier to dsync to a Maildir instead. This should preserve all the Dovecot-visible metadata, and dsyncing back to the dbox for restore should put it all back. > (2) What is the best why to copy this local backup to a remote location > that does NOT have the possibility to run dsync. > > - Question 1: is rsync safe to use and will this data work for restore? > > - Question 2: Would it be safe to simply rsync the original mail_location > to the remote server? AFAICT, if Dovecot is stopped on both sides of the transfer it should be safe. If either side has a currently running Dovecot instance (or any other Dovecot tools, like a concurrent dsync) using that dbox, it's likely rsync will copy an inconsistent snapshot of the data, and the result will be corrupted. If you have truly atomic filesystem snapshots, rsyncing out of a snapshot also *ought* to be safe, since the situation is the same as if Dovecot crashed at the moment the snapshot was taken, though it might be a good idea to run doveadm force-resync to make sure the result is consistent from Dovecot's point of view. Ben From dave at dawoodfall.net Mon Feb 4 04:01:05 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 4 Feb 2013 02:01:05 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc Message-ID: <20130204020105.GA15960@Blackswan> Version 2.1.10 For some reason Mutt doesn't list any info about IMAP folders and seems to ignore folder_format. My questions is does anyone have any workarounds for this? From dave at dawoodfall.net Mon Feb 4 04:01:05 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 4 Feb 2013 02:01:05 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc Message-ID: <20130204020105.GA15960@Blackswan> Version 2.1.10 For some reason Mutt doesn't list any info about IMAP folders and seems to ignore folder_format. My questions is does anyone have any workarounds for this? From dave at dawoodfall.net Mon Feb 4 04:01:05 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 4 Feb 2013 02:01:05 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc Message-ID: <20130204020105.GA15960@Blackswan> Version 2.1.10 For some reason Mutt doesn't list any info about IMAP folders and seems to ignore folder_format. My questions is does anyone have any workarounds for this? From daniel.parthey at informatik.tu-chemnitz.de Mon Feb 4 04:08:24 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 4 Feb 2013 03:08:24 +0100 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <20130204020105.GA15960@Blackswan> References: <20130204020105.GA15960@Blackswan> Message-ID: <20130204020824.GA14425@daniel.localdomain> David Woodfall wrote: > Version 2.1.10 > > For some reason Mutt doesn't list any info about IMAP folders and > seems to ignore folder_format. Doesn't have anything to with dovecot, does it? It looks as if folder_format option of mutt as only for local folders, and will not work for IMAP. Regards Daniel -- https://plus.google.com/103021802792276734820 From dave at dawoodfall.net Mon Feb 4 04:19:33 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 04 Feb 2013 02:19:33 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <20130204020824.GA14425@daniel.localdomain> References: <20130204020105.GA15960@Blackswan> <20130204020824.GA14425@daniel.localdomain> Message-ID: <510F1AB5.6080503@dawoodfall.net> On 04/02/13 02:08, Daniel Parthey wrote: > David Woodfall wrote: >> Version 2.1.10 >> >> For some reason Mutt doesn't list any info about IMAP folders and >> seems to ignore folder_format. > > Doesn't have anything to with dovecot, does it? > > It looks as if folder_format option of mutt > as only for local folders, and will not work for IMAP. I was grasping at straws and hoping someone here may know. From skdovecot at smail.inf.fh-brs.de Mon Feb 4 10:37:59 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 4 Feb 2013 09:37:59 +0100 (CET) Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510BEBDC.6020802@mclien.de> References: <510BEBDC.6020802@mclien.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 1 Feb 2013, Frank Lienhard wrote: > My first attempt was tho simply rsync the Maildirs along with the homes. Both > deovecot versions are set up with the maildir format, but this results in > ~30% doublicated mails on the new server. Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)?? b) Or do you see them, when you login via IMAP to the the new server? === If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve hardlinks and/or "--delete" if you run rsync multiple times? If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh and empty local cache / profile / ... . Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUQ9zZ13r2wJMiz2NAQI67wf/X6XVPzXOygUZWX8s6Wthf8UyHs+Pew9v QKKuwOvm4lb7mF9vzVYMtTFwSHyYUCvSgB+oSPmRpX/bGReSq1+PwpIYQOpKvU+b EpaWTyBMXDyzToPuwnd/VZgnzzVuBa5IVbvg6neLIi7yrco+VjbwQJ1B/VPLHjRO YVusoMn0ddJWNE01ix2w54kBc9eSS54hcIuzkac3fyp2vTLl3LYgBJ0lRUp2ztkX C4sn1ejOqXkfBLc20jL52RW+FW0pH0P7HTCCz+XWkJFmiMXcDEoymBL62fI1MuAh UMNPvUIVcZcT/Q5FA8QGVRnRyKH5S4snRPkHEqSP2A39tDhbM/882A== =XIiO -----END PGP SIGNATURE----- From adam at xbsd.pl Mon Feb 4 10:51:36 2013 From: adam at xbsd.pl (Adam Maciejewski) Date: Mon, 4 Feb 2013 09:51:36 +0100 Subject: [Dovecot] Subfolders problem Message-ID: I have moved from dovecot 1.x to 2.x and I have big problem with subfolders. When I'm moving subfolder with other subfolders is moving only main subfolder, without subfolders, example : mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep -e "janusz|jarek" ./jarek ./jarek/dbox-Mails ./jarek/dbox-Mails/dovecot.index.log ./jarek/jarek2 ./jarek/jarek2/dbox-Mails ./jarek/jarek2/dbox-Mails/dovecot.index.log ./jarek/jarek2/jarek3 ./jarek/jarek2/jarek3/dbox-Mails ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log ./janusz ./janusz/dbox-Mails ./janusz/dbox-Mails/dovecot.index.log ./janusz/janusz2 ./janusz/janusz2/dbox-Mails ./janusz/janusz2/dbox-Mails/dovecot.index.log ./janusz/janusz2/janusz3 ./janusz/janusz2/janusz3/dbox-Mails ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log And now I'm moving jarek2 with jarek3 as subfolder to janusz2 subfolder, result : mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep -e "janusz|jarek" ./jarek ./jarek/dbox-Mails ./jarek/dbox-Mails/dovecot.index.log ./jarek/jarek2 ./jarek/jarek2/jarek3 ./jarek/jarek2/jarek3/dbox-Mails ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log ./janusz ./janusz/dbox-Mails ./janusz/dbox-Mails/dovecot.index.log ./janusz/janusz2 ./janusz/janusz2/dbox-Mails ./janusz/janusz2/dbox-Mails/dovecot.index.log ./janusz/janusz2/dbox-Mails/dovecot.index.cache ./janusz/janusz2/janusz3 ./janusz/janusz2/janusz3/dbox-Mails ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log ./janusz/janusz2/jarek2 ./janusz/janusz2/jarek2/dbox-Mails ./janusz/janusz2/jarek2/dbox-Mails/dovecot.index.log As You can see there is jarek2 subfolder but jarek3 havent been moved. I can see this folder in Thunderbird, after clicking error message pops out saying folder dosent exist. -- Adam Maciejewski From alessio at skye.it Mon Feb 4 11:15:56 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 10:15:56 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled Message-ID: <510F7C4C.7020303@skye.it> Hi, during the weekend I upgrade my mail servers from dovecot 2.1.13 to dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). After upgrade all works fine, but today the load avarage un MXs is doubled. Mailboxes are in Maildir/ on NFS and I'm using dovecot as LDA and dict quota only for quota usage. May be a problem with the new version? This is my configuration: $ ./configure --prefix=/usr/local/dovecot-2.1 --with-vpopmail --with-mysql --with-bzlib --with-zlib # cat /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve require ["vnd.dovecot.duplicate", "fileinto", "mailbox"]; if header :contains "X-Spam-Status" "Yes" { fileinto "Spam"; } if header :contains "Subject" "TEST 123" { fileinto "Spam"; } if duplicate { fileinto "Trash"; } ##################### # 2.1.14: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf expire = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl zlib expire maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.duplicate mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=192.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl expire = Trash expire2 = Spam expire_dict = proxy::expire quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.duplicate } protocols = imap pop3 sieve service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } service quota-warning { executable = script /usr/local/dovecot-2.1/etc/dovecot/scripts/quota-warning.sh unix_listener quota-warning { user = vpopmail } user = vpopmail } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From skdovecot at smail.inf.fh-brs.de Mon Feb 4 13:08:58 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 4 Feb 2013 12:08:58 +0100 (CET) Subject: [Dovecot] SOLVED for me: how to setup different quota for multiple namespaces In-Reply-To: <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 22 Jan 2013, Timo Sirainen wrote: Hey, it works (for me) now: namespace { type = private separator = . prefix = archive. location = mdbox:/home/%u/archive #subscriptions = no #list = children } plugin { quota = dict:User quota::ns=:proxy::quota quota2 = dict:Archive quota:%u.archive:ns=archive.:proxy::quota quota2_rule = *:storage=1048576 } Note the %u.archive _and_ ns=archive. ! The "%u.archive" is used as "username" in the SQL table and you require another name (primary key) there for both namespaces, see below. Seems to work for any type of namespace. @Timo: I will going to document this setup in the Wiki unless you say the syntax is wrong and/or works because of a bug only. :-) ================================== Old response with my debugging/findings for information only. ==== all the name spaces use one entry in the quota dict table, hence, it does not work. > On 17.1.2013, at 16.58, Andreas Oster wrote: > >>> I just saw on important difference in the doc and this configuration: >>> >>> see >>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>> >>> the ns=name syntax is for _public_ namespaces only. I just tested it >>> with this setup and every message is counted for both namespaces, if >>> delivered into INBOX or a mailbox of the Archive namespace. > > It should work for all namespaces. > >>>> plugin { >>>> quota = dict:User quota::proxy::quota >>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=Archive.:proxy::quota > > That should work? Worked at least in latest v2.1 hg. I use these settings now: # 2.1.14 (ea7e45c1da72+): /usr/local/dovecot-2.1.14/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 [snip] dict { quota = pgsql:/usr/local/dovecot-2.1.14/etc/dovecot/dovecot-dict-sql.conf.ext } namespace { location = mdbox:/home/%u/archive prefix = archive. separator = . type = private } plugin { quota = dict:User quota::ns=:proxy::quota quota2 = dict:Archive quota::ns=archive.:proxy::quota quota2_rule = *:storage=1048576 quota_rule = *:storage=300MB quota_rule2 = Trash:storage=+30M } The problem is now, that there is one row in the quota dict SQL table, that holds the quota of an user, e.g. doveadm quota recalc executes these SQL statements: 2013-02-04 11:52:07 CET LOG: statement: BEGIN 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (bytes,username) VALUES ('2173894','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (messages,username) VALUES ('89','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: COMMIT 2013-02-04 11:52:07 CET LOG: statement: BEGIN 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE username = 'dvtest1' 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (bytes,username) VALUES ('2582','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota (messages,username) VALUES ('2','dvtest1') 2013-02-04 11:52:07 CET LOG: statement: COMMIT Deliveries to INBOX or a folder of "archive." yields: UPDATE quota SET bytes=bytes+1220,messages=messages+1 WHERE username = 'dvtest1' This is true if I change the type of name spaces "archive" from "private" into "shared" or "public". - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUQ+Wyl3r2wJMiz2NAQL2fwf/SV2SAl+lvzwN9TEEZfwtsHwTb9D7W1QH wfoMEJLWBFp8/i+gpjhbGNWbi9P5ogqljuJvALYVIXFNfoGllJBlAEStxBhpv5Z/ xQCY5lB9nCbnGa0g2MA0DMLF+dCmQrUdP0kl7R6moXaa4tn/C40pmkxvGSJjQsFk qMESy+cuREBSJq3SRP5g8tnlBC14dgoVfg+cWPVzOd/iQatIVQGghC5PqtLj36g3 rYJNvMZ8inGSf9pui21F9rHaN1RLeeACkAEF256Pt1lHBWKaecAhlvK+e3Ps1vru mssHT+z50T7jee8pwjMK6NMW4IbDgeU8wv9yaf4/c5LmfVPHP6E2cA== =D38F -----END PGP SIGNATURE----- From aoster at novanetwork.de Mon Feb 4 13:20:53 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 12:20:53 +0100 Subject: [Dovecot] SOLVED for me: how to setup different quota for multiple namespaces In-Reply-To: References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F9995.8060404@novanetwork.de> Am 04.02.2013 12:08, schrieb Steffen Kaiser: > On Tue, 22 Jan 2013, Timo Sirainen wrote: > > Hey, it works (for me) now: > > namespace { > type = private > separator = . > prefix = archive. > location = mdbox:/home/%u/archive > #subscriptions = no > #list = children > } > > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota:%u.archive:ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > } > > Note the %u.archive _and_ ns=archive. ! The "%u.archive" is used as > "username" in the SQL table and you require another name (primary key) > there for both namespaces, see below. Seems to work for any type of > namespace. > > @Timo: I will going to document this setup in the Wiki unless you say > the syntax is wrong and/or works because of a bug only. > > :-) > > ================================== > > Old response with my debugging/findings for information only. > > ==== > > all the name spaces use one entry in the quota dict table, hence, > it does not work. > >> On 17.1.2013, at 16.58, Andreas Oster wrote: > >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. > >> It should work for all namespaces. > >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota > >> That should work? Worked at least in latest v2.1 hg. > > I use these settings now: > > # 2.1.14 (ea7e45c1da72+): > /usr/local/dovecot-2.1.14/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 > > [snip] > > dict { > quota = > pgsql:/usr/local/dovecot-2.1.14/etc/dovecot/dovecot-dict-sql.conf.ext > } > namespace { > location = mdbox:/home/%u/archive > prefix = archive. > separator = . > type = private > } > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > quota_rule = *:storage=300MB > quota_rule2 = Trash:storage=+30M > } > > The problem is now, that there is one row in the quota dict SQL table, > that holds the quota of an user, e.g. doveadm quota recalc executes > these SQL statements: > > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2173894','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('89','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2582','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('2','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > > Deliveries to INBOX or a folder of "archive." yields: > > UPDATE quota SET bytes=bytes+1220,messages=messages+1 WHERE username = > 'dvtest1' > > This is true if I change the type of name spaces "archive" from "private" > into "shared" or "public". > > -- Steffen Kaiser > Hello Steffen, I do not use a SQL backend to store the quota information, so I guess the %u parameter is of little use with my setup. I have also tried to configure quota with dict and flat file, but this also did seem not work. Thanks best regards Andreas From aoster at novanetwork.de Mon Feb 4 13:20:53 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 12:20:53 +0100 Subject: [Dovecot] SOLVED for me: how to setup different quota for multiple namespaces In-Reply-To: References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F9995.8060404@novanetwork.de> Am 04.02.2013 12:08, schrieb Steffen Kaiser: > On Tue, 22 Jan 2013, Timo Sirainen wrote: > > Hey, it works (for me) now: > > namespace { > type = private > separator = . > prefix = archive. > location = mdbox:/home/%u/archive > #subscriptions = no > #list = children > } > > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota:%u.archive:ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > } > > Note the %u.archive _and_ ns=archive. ! The "%u.archive" is used as > "username" in the SQL table and you require another name (primary key) > there for both namespaces, see below. Seems to work for any type of > namespace. > > @Timo: I will going to document this setup in the Wiki unless you say > the syntax is wrong and/or works because of a bug only. > > :-) > > ================================== > > Old response with my debugging/findings for information only. > > ==== > > all the name spaces use one entry in the quota dict table, hence, > it does not work. > >> On 17.1.2013, at 16.58, Andreas Oster wrote: > >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. > >> It should work for all namespaces. > >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota > >> That should work? Worked at least in latest v2.1 hg. > > I use these settings now: > > # 2.1.14 (ea7e45c1da72+): > /usr/local/dovecot-2.1.14/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 > > [snip] > > dict { > quota = > pgsql:/usr/local/dovecot-2.1.14/etc/dovecot/dovecot-dict-sql.conf.ext > } > namespace { > location = mdbox:/home/%u/archive > prefix = archive. > separator = . > type = private > } > plugin { > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=archive.:proxy::quota > quota2_rule = *:storage=1048576 > quota_rule = *:storage=300MB > quota_rule2 = Trash:storage=+30M > } > > The problem is now, that there is one row in the quota dict SQL table, > that holds the quota of an user, e.g. doveadm quota recalc executes > these SQL statements: > > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2173894','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('89','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > 2013-02-04 11:52:07 CET LOG: statement: BEGIN > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: DELETE FROM quota WHERE > username = 'dvtest1' > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (bytes,username) VALUES ('2582','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: INSERT INTO quota > (messages,username) VALUES ('2','dvtest1') > 2013-02-04 11:52:07 CET LOG: statement: COMMIT > > Deliveries to INBOX or a folder of "archive." yields: > > UPDATE quota SET bytes=bytes+1220,messages=messages+1 WHERE username = > 'dvtest1' > > This is true if I change the type of name spaces "archive" from "private" > into "shared" or "public". > > -- Steffen Kaiser > Hello Steffen, I do not use a SQL backend to store the quota information, so I guess the %u parameter is of little use with my setup. I have also tried to configure quota with dict and flat file, but this also did seem not work. Thanks best regards Andreas From alessio at skye.it Mon Feb 4 15:28:01 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 14:28:01 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <510F7C4C.7020303@skye.it> References: <510F7C4C.7020303@skye.it> Message-ID: <510FB761.3040908@skye.it> Il 04/02/2013 10:15, Alessio Cecchi ha scritto: > Hi, > > during the weekend I upgrade my mail servers from dovecot 2.1.13 to > dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). > Moreover, in the log file I found these errors never seen before: Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory What do they mean? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From tss at iki.fi Mon Feb 4 15:58:16 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 15:58:16 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <510FB761.3040908@skye.it> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> Message-ID: <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> On 4.2.2013, at 15.28, Alessio Cecchi wrote: > Il 04/02/2013 10:15, Alessio Cecchi ha scritto: >> Hi, >> >> during the weekend I upgrade my mail servers from dovecot 2.1.13 to >> dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). >> > > Moreover, in the log file I found these errors never seen before: > > Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory Well .. Those errors should go away if you revert this patch: http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a But does it fix the performance? .. From tss at iki.fi Mon Feb 4 17:07:19 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 04 Feb 2013 17:07:19 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> Message-ID: <1359990439.3230.4.camel@hurina> On Mon, 2013-02-04 at 15:58 +0200, Timo Sirainen wrote: > On 4.2.2013, at 15.28, Alessio Cecchi wrote: > > > Il 04/02/2013 10:15, Alessio Cecchi ha scritto: > >> Hi, > >> > >> during the weekend I upgrade my mail servers from dovecot 2.1.13 to > >> dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). > >> > > > > Moreover, in the log file I found these errors never seen before: > > > > Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > > Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory > > Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory > > Well .. Those errors should go away if you revert this patch: > > http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a > > But does it fix the performance? .. No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f From alessio at skye.it Mon Feb 4 17:38:30 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 16:38:30 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <1359990439.3230.4.camel@hurina> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> Message-ID: <510FD5F6.60808@skye.it> Il 04/02/2013 16:07, Timo Sirainen ha scritto: > On Mon, 2013-02-04 at 15:58 +0200, Timo Sirainen wrote: >> On 4.2.2013, at 15.28, Alessio Cecchi wrote: >> >>> Il 04/02/2013 10:15, Alessio Cecchi ha scritto: >>>> Hi, >>>> >>>> during the weekend I upgrade my mail servers from dovecot 2.1.13 to >>>> dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). >>>> >>> >>> Moreover, in the log file I found these errors never seen before: >>> >>> Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>> Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>> Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory >> >> Well .. Those errors should go away if you revert this patch: >> >> http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a >> >> But does it fix the performance? .. > > No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f > > Thanks Timo, tonight I will apply the second patch to 2.1.14 for the performance problem. Should I also remove the path "2b76d357a56a" or is only optional/for test? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From tss at iki.fi Mon Feb 4 17:44:07 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 17:44:07 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <510FD5F6.60808@skye.it> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: On 4.2.2013, at 17.38, Alessio Cecchi wrote: >>>> Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>> Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>> Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory >>> >>> Well .. Those errors should go away if you revert this patch: >>> >>> http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a >>> >>> But does it fix the performance? .. >> >> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f >> >> > > Thanks Timo, > > tonight I will apply the second patch to 2.1.14 for the performance problem. > > Should I also remove the path "2b76d357a56a" or is only optional/for test? Removing it removes those errors from logs, but other than that it shouldn't make any difference. Instead of removing you could also apply these patches that should remove them more correctly: http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 From kharford at constantcontact.com Mon Feb 4 17:45:41 2013 From: kharford at constantcontact.com (Harford, Ken) Date: Mon, 4 Feb 2013 15:45:41 +0000 Subject: [Dovecot] Overloading IMAP Server In-Reply-To: Message-ID: Okay, I set a wait period of 30 seconds and I am still having issues. I am sending about 50 messages from 6 servers (300 messages) to the same IMAP mailbox at the same time and dovecot doesn't seem to be able handle that kind of message count. I am using dovecot delivery instead of postfix delivery Any other ideas? Thanks Ken On 1/22/13 4:04 AM, "Steffen Kaiser" wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Mon, 21 Jan 2013, Harford, Ken wrote: > >> I am having an issue with dovecot IMAP reading and writing >>to >> one specific mailbox. I have a check email delivery script that sends > >What's the purpose of the verification: > >+ The prompt delivery? > >+ The delivery at all? > >+ The correct delivery? > >> out a email and then logs into dovecot to check if the mailbox that the >> message has been delivered to. However, the servers that send these >> messages send them in bunches depending on how many postfix spools it >>is >> checking. So I could get 60 messages all being sent to 1 mailbox and >> then having to read that same mailbox to check if the message actually >> got delivered. >> >> I am getting a lot of errors such as the following: >> >> Could not connect to IMAP server >> imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX > >Is this an error logged by Dovecot? > >> Any ideas as to what I can tweak in dovecot to allow these messages to >>be verified. > >a) >Wait a bit before logging into the IMAP server and check the messages >sequentionally, not in parallel. E.g. your script could synchronize >itself >with other parallel instances by a shared lock. > >b) >Use mail_log and a script on the mail server monitoring Dovecot logs and >register delivered message ids and/or recipient. Then figure out a good >way to probe, if one pariticular message has been seen. >- -or- >make note of message id/recipient on the sender side, somehow push (ssh >or >something) that information on regular basis to the mail server, where a >script verifies that data. > >c) To verify the correct delivery: use some sort of checksumming. > >Kind regards, > >- -- >Steffen Kaiser >-----BEGIN PGP SIGNATURE----- >Version: GnuPG v1.4.11 (GNU/Linux) > >iQEVAwUBUP5WEF3r2wJMiz2NAQIatAgAjwZR1VGnyURuncLRNeeY0dBaqWzAUcNK >g4YoiXB6NLxP6Xr/QO6Pw+6Q3sECJ0CJdwImOz56FVFNuiOygUMQ1dCdZ+Tpy9Wu >xeAdobrkOo2uzgRnUh7+WLeZ8k32gZZztCAQA90uTvB3MBox+7J/P8/mApp+8/tr >JKyhUy8RJIYXH+yrw/ZkbjmOSEEkf/wTPnBQcMKw5/4odha4lu+yEKx766mm0YCe >3USL7eVqN0Q9FwUH+iCjnRJAdKRETP4CnfQpjVp7eiJX7HD0iNwhucDwf037Ev2N >PG173M40pMllRyesjiV8gwOu8mAHu6V3QUiFJkYAKFOotUVO6xMrNQ== >=SeaP >-----END PGP SIGNATURE----- From jerry at seibercom.net Mon Feb 4 18:01:37 2013 From: jerry at seibercom.net (Jerry) Date: Mon, 4 Feb 2013 11:01:37 -0500 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: <20130204110137.637541f3@scorpio> On Mon, 4 Feb 2013 17:44:07 +0200 Timo Sirainen articulated: [snip > Removing it removes those errors from logs, but other than that it > shouldn't make any difference. Instead of removing you could also > apply these patches that should remove them more correctly: > > http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 Looks like you are going to have to release version 2.1.15 in the very near future. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Mon Feb 4 18:05:01 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 18:05:01 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <20130204110137.637541f3@scorpio> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <20130204110137.637541f3@scorpio> Message-ID: On 4.2.2013, at 18.01, Jerry wrote: > On Mon, 4 Feb 2013 17:44:07 +0200 > Timo Sirainen articulated: > > [snip > >> Removing it removes those errors from logs, but other than that it >> shouldn't make any difference. Instead of removing you could also >> apply these patches that should remove them more correctly: >> >> http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d >> http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > > Looks like you are going to have to release version 2.1.15 in the very > near future. Yeah. :( Would be nice if people tested those new patches, so I don't have to put out 2.1.16 soon also. From alessio at skye.it Mon Feb 4 11:15:56 2013 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 04 Feb 2013 10:15:56 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled Message-ID: <510F7C4C.7020303@skye.it> Hi, during the weekend I upgrade my mail servers from dovecot 2.1.13 to dovecot 2.1.14 (and rebuild dovecot-2.1-pigeonhole-0.3.3). After upgrade all works fine, but today the load avarage un MXs is doubled. Mailboxes are in Maildir/ on NFS and I'm using dovecot as LDA and dict quota only for quota usage. May be a problem with the new version? This is my configuration: $ ./configure --prefix=/usr/local/dovecot-2.1 --with-vpopmail --with-mysql --with-bzlib --with-zlib # cat /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve require ["vnd.dovecot.duplicate", "fileinto", "mailbox"]; if header :contains "X-Spam-Status" "Yes" { fileinto "Spam"; } if header :contains "Subject" "TEST 123" { fileinto "Spam"; } if duplicate { fileinto "Trash"; } ##################### # 2.1.14: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf expire = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl zlib expire maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.duplicate mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=192.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl expire = Trash expire2 = Spam expire_dict = proxy::expire quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.duplicate } protocols = imap pop3 sieve service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } service quota-warning { executable = script /usr/local/dovecot-2.1/etc/dovecot/scripts/quota-warning.sh unix_listener quota-warning { user = vpopmail } user = vpopmail } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From aoster at novanetwork.de Mon Feb 4 11:40:02 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 10:40:02 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F81F2.1020809@novanetwork.de> Am 22.01.2013 12:54, schrieb Timo Sirainen: > On 17.1.2013, at 16.58, Andreas Oster wrote: > >>> I just saw on important difference in the doc and this configuration: >>> >>> see >>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>> >>> the ns=name syntax is for _public_ namespaces only. I just tested it >>> with this setup and every message is counted for both namespaces, if >>> delivered into INBOX or a mailbox of the Archive namespace. > > It should work for all namespaces. > >>>> plugin { >>>> quota = dict:User quota::proxy::quota >>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=Archive.:proxy::quota > > That should work? Worked at least in latest v2.1 hg. > > Hello Timo, can you please confirm, that it is actually possible to have multiple quota roots with separate/different quota values. Currently I have this config: namespace { disabled = no hidden = no ignore_on_failure = no inbox = no list = children location = maildir:/var/vmail/archives/%n/Maildir prefix = Archives/ separator = / subscriptions = yes type = private } namespace inbox { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota_rule3 = ns=Archives/:storage=5G } With this setup I get the following error: Debug: Quota root: name=User quota backend=maildir args=ns= Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete configuration for rule 'storage=5G' should be changed to 'storage=+5G' Debug: Quota rule: root=User quota mailbox=ns=Archives/ bytes=+5368709120 messages=0 Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ Thank you for your kind help. best regards Andreas From aoster at novanetwork.de Mon Feb 4 11:40:02 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Mon, 04 Feb 2013 10:40:02 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> Message-ID: <510F81F2.1020809@novanetwork.de> Am 22.01.2013 12:54, schrieb Timo Sirainen: > On 17.1.2013, at 16.58, Andreas Oster wrote: > >>> I just saw on important difference in the doc and this configuration: >>> >>> see >>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>> >>> the ns=name syntax is for _public_ namespaces only. I just tested it >>> with this setup and every message is counted for both namespaces, if >>> delivered into INBOX or a mailbox of the Archive namespace. > > It should work for all namespaces. > >>>> plugin { >>>> quota = dict:User quota::proxy::quota >>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota > > quota = dict:User quota::ns=:proxy::quota > quota2 = dict:Archive quota::ns=Archive.:proxy::quota > > That should work? Worked at least in latest v2.1 hg. > > Hello Timo, can you please confirm, that it is actually possible to have multiple quota roots with separate/different quota values. Currently I have this config: namespace { disabled = no hidden = no ignore_on_failure = no inbox = no list = children location = maildir:/var/vmail/archives/%n/Maildir prefix = Archives/ separator = / subscriptions = yes type = private } namespace inbox { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota_rule3 = ns=Archives/:storage=5G } With this setup I get the following error: Debug: Quota root: name=User quota backend=maildir args=ns= Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete configuration for rule 'storage=5G' should be changed to 'storage=+5G' Debug: Quota rule: root=User quota mailbox=ns=Archives/ bytes=+5368709120 messages=0 Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ Thank you for your kind help. best regards Andreas From tlx at leuxner.net Mon Feb 4 12:56:53 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 4 Feb 2013 11:56:53 +0100 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <510F1AB5.6080503@dawoodfall.net> References: <20130204020105.GA15960@Blackswan> <20130204020824.GA14425@daniel.localdomain> <510F1AB5.6080503@dawoodfall.net> Message-ID: <20130204105653.GA21474@nihlus.leuxner.net> * David Woodfall 2013.02.04 03:19: > >It looks as if folder_format option of mutt > >as only for local folders, and will not work for IMAP. > > I was grasping at straws and hoping someone here may know. As mentioned this may be one for the mutt list. But it should work with IMAP: set imap_user="user at domain.tld" set folder="imap://host.domain.tld/" set spoolfile="imap://host.domain.tld/INBOX" set index_format="%4C %Z %2M %[!%Y.%m.%d %H:%M] %-30.30F (%5c) %s" Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From jerry at seibercom.net Mon Feb 4 18:01:37 2013 From: jerry at seibercom.net (Jerry) Date: Mon, 4 Feb 2013 11:01:37 -0500 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: <20130204110137.637541f3@scorpio> On Mon, 4 Feb 2013 17:44:07 +0200 Timo Sirainen articulated: [snip > Removing it removes those errors from logs, but other than that it > shouldn't make any difference. Instead of removing you could also > apply these patches that should remove them more correctly: > > http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 Looks like you are going to have to release version 2.1.15 in the very near future. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Mon Feb 4 18:05:01 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Feb 2013 18:05:01 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <20130204110137.637541f3@scorpio> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <20130204110137.637541f3@scorpio> Message-ID: On 4.2.2013, at 18.01, Jerry wrote: > On Mon, 4 Feb 2013 17:44:07 +0200 > Timo Sirainen articulated: > > [snip > >> Removing it removes those errors from logs, but other than that it >> shouldn't make any difference. Instead of removing you could also >> apply these patches that should remove them more correctly: >> >> http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d >> http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > > Looks like you are going to have to release version 2.1.15 in the very > near future. Yeah. :( Would be nice if people tested those new patches, so I don't have to put out 2.1.16 soon also. From jerry at seibercom.net Mon Feb 4 18:24:26 2013 From: jerry at seibercom.net (Jerry) Date: Mon, 4 Feb 2013 11:24:26 -0500 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <20130204110137.637541f3@scorpio> Message-ID: <20130204112426.2d483478@scorpio> On Mon, 4 Feb 2013 18:05:01 +0200 Timo Sirainen articulated: > On 4.2.2013, at 18.01, Jerry wrote: > > > On Mon, 4 Feb 2013 17:44:07 +0200 > > Timo Sirainen articulated: > > > > [snip > > > >> Removing it removes those errors from logs, but other than that it > >> shouldn't make any difference. Instead of removing you could also > >> apply these patches that should remove them more correctly: > >> > >> http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > >> http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > > > > Looks like you are going to have to release version 2.1.15 in the > > very near future. > > Yeah. :( > > Would be nice if people tested those new patches, so I don't have to > put out 2.1.16 soon also. I think that most people, myself included, are hesitant about applying patches to systems that are all ready up and running satisfactory. As you will no doubt concur, there is nothing worse that updating or applying a patch to a critical application and then discovering it is now broken. Perhaps, and this is just my 2? on the matter, you could form a small group of BETA testers. I have no idea if that would work out for you or not. In any event, Dovecot rarely has a serious BUG problem. Just those pesky "nits & gnats" that have to be tended to occasionally. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From andy at xecu.net Mon Feb 4 19:08:33 2013 From: andy at xecu.net (Andy Dills) Date: Mon, 04 Feb 2013 12:08:33 -0500 Subject: [Dovecot] Errors with doveadm when using checkpassword Message-ID: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> We have a checkpassword authentication with mysql pre-fetch for the userdb lookups. When trying to do: doveadm search -u andytest at xecu.net mailbox Trash DELETED I get: doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS lookup failed doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb lookup failed (to see if user is proxied, because doveadm_proxy_port is set) And I see this in the logs: Feb 4 12:02:04 mail-out01 dovecot: auth: Error: userdb-checkpassword(andytest at xecu.net): Child 12591 exited with status 1 I'm not surprised the checkpassword lookup is failing; the password isn't being supplied. Shouldn't it just be doing the prefetch lookup instead, like the lmtp service? Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From frank at mclien.de Mon Feb 4 19:59:30 2013 From: frank at mclien.de (Frank) Date: Mon, 04 Feb 2013 18:59:30 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: References: <510BEBDC.6020802@mclien.de> Message-ID: <510FF702.4030208@mclien.de> Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 1 Feb 2013, Frank Lienhard wrote: > >> My first attempt was tho simply rsync the Maildirs along with the >> homes. Both deovecot versions are set up with the maildir format, but >> this results in ~30% doublicated mails on the new server. > > Er, a) you got 30% more files on the new server (aka target), than you > have on the old server (aka source)?? > > b) Or do you see them, when you login via IMAP to the the new server? > > === > > If a), did you ran rsync, while the source host is in production > still? Do you use "H" option to preserve hardlinks and/or "--delete" > if you run rsync multiple times? > > If b), did you verified the duplication with a IMAP client, that does > _no_ caching on its own or with a fresh and empty local cache / > profile / ... . > I used -avu, where: -a equals: -rlptgoD (no -H,-A,-X) -u skip file that are newer on the receiver I stopped the getmail cronjob to ensure that no new messages get fetched while sync. After that I created a new account in Icedove and at first all went well, but after a while I got duoplicates of abou 900 old messages. I admit I only looked that up from Icedove and not in the dir from cli. From metro_domain_admin at fastmail.fm Mon Feb 4 22:34:50 2013 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Mon, 04 Feb 2013 15:34:50 -0500 Subject: [Dovecot] dsync-local(user): Error: proxy client timed out Message-ID: <1360010090.18396.140661186767549.3BEC6B22@webmail.messagingengine.com> Hello, I have an intermittent but frequent problem using dsync backup. The command outputs: dsync-local(user): Error: proxy client timed out Then dies with exit code 75. Any subsequent attempts to re-run the backup invariably fail like this: dsync-local(user): Warning: Destination mailbox INBOX has been modified, need to recreate it before we can continue syncing dsync-local(user): Error: read() from worker server failed: EOF dsync-local(user): Error: msg-copy returned invalid input: dsync-local(user): Error: remote: dsync-remote(user at domain.com): Error: Can't delete mailbox INBOX: INBOX can't be deleted. dsync-remote(l_user at domain.com): Error: command MSG-SAVE failed dsync-local(user): Warning: Mailbox changes caused a desync. You may want to run dsync again. The syntax I am using is: dsync -u $user backup \ ssh -i \ dsync -u l_$user We have this running in a script that loops through all users. The error happens mostly on the larger mailboxes but I've seen it succeed on the largest ones then fail on smaller. One mid-sized account now fails consistently. The backup is running from a Mac OS PPC server running 2.1.12 The target is a Linux Intel box running 2.1.13, but I had the same problem sending to a Mac OS Intel on 2.1.12. Mdbox in all cases. Thanks, Tony From list at airstreamcomm.net Mon Feb 4 23:24:14 2013 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 04 Feb 2013 15:24:14 -0600 Subject: [Dovecot] METADATA / ANNOTATE extensions Message-ID: <511026FE.3090901@airstreamcomm.net> According to the roadmap http://wiki2.dovecot.org/Roadmap implementation of the METADATA / ANNOTATE extensions is fairly high on the list. I was just curious if there is a target release in mind for this functionality? From tss at iki.fi Tue Feb 5 03:53:47 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 03:53:47 +0200 Subject: [Dovecot] METADATA / ANNOTATE extensions In-Reply-To: <511026FE.3090901@airstreamcomm.net> References: <511026FE.3090901@airstreamcomm.net> Message-ID: <1360029227.10326.42.camel@innu> On Mon, 2013-02-04 at 15:24 -0600, list at airstreamcomm.net wrote: > According to the roadmap http://wiki2.dovecot.org/Roadmap implementation > of the METADATA / ANNOTATE extensions is fairly high on the list. I was > just curious if there is a target release in mind for this functionality? v2.2 has the underlying functionality to set key=value to mailboxes, so it could still be added to v2.2. It needs: - dsync needs to be able to sync them - ACL plugin needs to restrict the set/get - IMAP commands to access them The dsync change is the main reason I haven't yet added it. From tss at iki.fi Tue Feb 5 03:55:01 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 03:55:01 +0200 Subject: [Dovecot] dsync-local(user): Error: proxy client timed out In-Reply-To: <1360010090.18396.140661186767549.3BEC6B22@webmail.messagingengine.com> References: <1360010090.18396.140661186767549.3BEC6B22@webmail.messagingengine.com> Message-ID: <1360029301.10326.43.camel@innu> On Mon, 2013-02-04 at 15:34 -0500, Metro Domain Admin wrote: > I have an intermittent but frequent problem using dsync backup. The > command outputs: > > dsync-local(user): Error: proxy client timed out > > Then dies with exit code 75. Any subsequent attempts to re-run the > backup invariably fail like this: > > dsync-local(user): Warning: Destination mailbox INBOX has been modified, > need to recreate it before we can continue syncing Looks like there are two bugs. v2.2 will have a redesigned dsync that should fix these problems. Too much trouble to try to fix it for v2.1. From tss at iki.fi Tue Feb 5 03:56:47 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 03:56:47 +0200 Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> Message-ID: <1360029407.10326.45.camel@innu> On Mon, 2013-02-04 at 12:08 -0500, Andy Dills wrote: > We have a checkpassword authentication with mysql pre-fetch for the > userdb lookups. > > When trying to do: > > doveadm search -u andytest at xecu.net mailbox Trash DELETED > > I get: > > doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS > lookup failed > doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb > lookup failed (to see if user is proxied, because doveadm_proxy_port is > set) Do you have proxying enabled? If not, remove doveadm_proxy_port and this solves itself. > I'm not surprised the checkpassword lookup is failing; the password > isn't being supplied. Shouldn't it just be doing the prefetch lookup > instead, like the lmtp service? doveadm doesn't care about the password. It cares about the proxy destination host so that it can automatically connect to the correct host to run the command. From tss at iki.fi Tue Feb 5 04:01:20 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:01:20 +0200 Subject: [Dovecot] dsync backup questions In-Reply-To: <20130204005729.GA68109@anubis.morrow.me.uk> References: <20130204005729.GA68109@anubis.morrow.me.uk> Message-ID: <1360029680.10326.48.camel@innu> On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote: > I can't give authoratitive answers to either of these, but... > > At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: > > > > I'm currently trying to setup remote backups of my emails but i'm running > > into issues (mdbox format, indexes and storage in the same folder > > hierarchy). > > > > Local backup command: dsync -u "my_user" backup /backups/my_user > > > > (1) Recently, I noticed that the local backup takes up twice the size as > > the original mail location (8gb vs. 4gb). I purged alot of emails from the > > original location, so the size shrunk, but the local backup just keeps on > > getting bigger. I couldn't find any dsync option that would delete extra > > emails. > > > > - Question: Why isn't the local backup synced properly and remove the extra > > emails? > > Are you running 'doveadm purge' on the backed-up dbox? It looks to me as > though dsync doesn't do that. I don't know if there's any (simple) way > to do that without a running Dovecot instance attached the dbox > directory: it's not entirely clear to me whether doveadm will run > locally without contacting a doveadm-server instance running under > Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. Right. doveadm -o mail=mdbox:/backups/my_user purge > It might be easier to dsync to a Maildir instead. This should preserve > all the Dovecot-visible metadata, and dsyncing back to the dbox for > restore should put it all back. Better sdbox than maildir. > > (2) What is the best why to copy this local backup to a remote location > > that does NOT have the possibility to run dsync. > > > > - Question 1: is rsync safe to use and will this data work for restore? > > > > - Question 2: Would it be safe to simply rsync the original mail_location > > to the remote server? > > AFAICT, if Dovecot is stopped on both sides of the transfer it should be > safe. If either side has a currently running Dovecot instance (or any > other Dovecot tools, like a concurrent dsync) using that dbox, it's > likely rsync will copy an inconsistent snapshot of the data, and the > result will be corrupted. It won't be badly corrupted though. At worst Dovecot will rebuild the indexes, which takes a while. And most users probably won't get any corruption at all. From tss at iki.fi Tue Feb 5 04:03:16 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:03:16 +0200 Subject: [Dovecot] Subfolders problem In-Reply-To: References: Message-ID: <1360029796.10326.49.camel@innu> On Mon, 2013-02-04 at 09:51 +0100, Adam Maciejewski wrote: > I have moved from dovecot 1.x to 2.x and I have big problem with > subfolders. > When I'm moving subfolder with other subfolders is moving only main > subfolder, without subfolders, example : What v2.x exactly? Looks like a bug. Probably fixed in a newer version. If you already have a recent v2.1, show your doveconf -n. > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > -e "janusz|jarek" > ./jarek > ./jarek/dbox-Mails > ./jarek/dbox-Mails/dovecot.index.log > ./jarek/jarek2 > ./jarek/jarek2/dbox-Mails > ./jarek/jarek2/dbox-Mails/dovecot.index.log > ./jarek/jarek2/jarek3 > ./jarek/jarek2/jarek3/dbox-Mails > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > ./janusz > ./janusz/dbox-Mails > ./janusz/dbox-Mails/dovecot.index.log > ./janusz/janusz2 > ./janusz/janusz2/dbox-Mails > ./janusz/janusz2/dbox-Mails/dovecot.index.log > ./janusz/janusz2/janusz3 > ./janusz/janusz2/janusz3/dbox-Mails > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > > And now I'm moving jarek2 with jarek3 as subfolder to janusz2 subfolder, > result : > > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > -e "janusz|jarek" > ./jarek > ./jarek/dbox-Mails > ./jarek/dbox-Mails/dovecot.index.log > ./jarek/jarek2 > ./jarek/jarek2/jarek3 > ./jarek/jarek2/jarek3/dbox-Mails > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > ./janusz > ./janusz/dbox-Mails > ./janusz/dbox-Mails/dovecot.index.log > ./janusz/janusz2 > ./janusz/janusz2/dbox-Mails > ./janusz/janusz2/dbox-Mails/dovecot.index.log > ./janusz/janusz2/dbox-Mails/dovecot.index.cache > ./janusz/janusz2/janusz3 > ./janusz/janusz2/janusz3/dbox-Mails > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > ./janusz/janusz2/jarek2 > ./janusz/janusz2/jarek2/dbox-Mails > ./janusz/janusz2/jarek2/dbox-Mails/dovecot.index.log > > As You can see there is jarek2 subfolder but jarek3 havent been moved. I > can see this folder in Thunderbird, after clicking error message pops out > saying folder dosent exist. > > From tss at iki.fi Tue Feb 5 04:06:42 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:06:42 +0200 Subject: [Dovecot] Broken physical size caching in Dovecot 2.1.10 In-Reply-To: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> References: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> Message-ID: <1360030002.10326.52.camel@innu> On Tue, 2013-01-29 at 19:58 +0000, Rob Redpath wrote: > I'm encountering two, probably related, errors in my Dovecot 2.1.10 > install. The issue relates to compressed email stored in Maildir format > directories. > > In some cases, a mailbox will become inaccessible, and the following > will be logged from a doveadm fetch:- > > doveadm(info at example.com): Error: Corrupted index cache file > /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken > physical size for mail UID 2777 > doveadm(info at example.com): Error: Cached message size smaller than > expected (18996 < 64624) > doveadm(info at example.com): Error: Maildir filename has wrong S value, > renamed the file from > /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z > to > /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z > > Interestingly, the file mentioned in the error ( > /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't > exist. > > The uncompressed size of the file is 64624 bytes, the compressed size is > 18996. Dovecot's zlib plugin requires that the maildir filenames have ,S=uncompressed size. Otherwise you get those errors. http://dovecot.org/tools/maildir-size-check.sh can be used to fix the filenames. From tss at iki.fi Tue Feb 5 04:09:00 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:09:00 +0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> Message-ID: <1360030140.10326.54.camel@innu> On Tue, 2013-01-29 at 16:29 -0200, Gustavo Sansone wrote: > hello everyone, i'm new to this list > im running dovecot as imap+pop (2.1.12) and roundcube (0.8.4) as > client. > > having problems on saving to imap folders, with errors on some accounts > > my folder structure is as follow: > > /home/user/.maildir > /home/user/.maildir/.Drafts > /home/user/.maildir/.Junk > /home/user/.maildir/.Sent > /home/user/.maildir/.Trash > /home/user/.maildir/.Archives > /home/user/.maildir/.Archives.2010 > ... > > i'm very confused with the namespace settings. i don't understand what > is the correct way to configure it. > don't need public or shared folders. Easiest to use the default namespace settings. There you wouldn't have any namespace prefix. > on dovecot.log i can see this error with my account (and all the > others), but it seems to work ok using roundcube: > imap(gsansone): Error: autocreate: No namespace found for mailbox: Junk In your current setup you have namespace prefix=INBOX. so the mailbox name would be INBOX.Junk. > with a few accounts, i have this on roundcube log: > IMAP Error: Could not save message in Drafts in > /var/www/localhost/htdocs/mail/program/steps/mail/sendmail.inc > with error on saving to imap folder in roundcube UI Roundcube also isn't configured to have INBOX. prefix. Sounds like you should just remove it. From andy at xecu.net Tue Feb 5 04:16:41 2013 From: andy at xecu.net (Andy Dills) Date: Mon, 4 Feb 2013 21:16:41 -0500 Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <1360029407.10326.45.camel@innu> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> Message-ID: On Feb 4, 2013, at 8:56 PM, Timo Sirainen wrote: > On Mon, 2013-02-04 at 12:08 -0500, Andy Dills wrote: >> We have a checkpassword authentication with mysql pre-fetch for the >> userdb lookups. >> >> When trying to do: >> >> doveadm search -u andytest at xecu.net mailbox Trash DELETED >> >> I get: >> >> doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS >> lookup failed >> doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb >> lookup failed (to see if user is proxied, because doveadm_proxy_port is >> set) > > Do you have proxying enabled? If not, remove doveadm_proxy_port and this > solves itself Yes, I apologize, it appears I edited out the paragraph where I described our setup. Cluster of dovecot dual instance servers, one proxy/director, one backend. The backend config lives in dovecot.conf, the proxy config in proxy.conf (let me know if the specific config would help). Version 2.1.12 (current version in FreeBSD ports). I should mention, I have to specify the director-admin socket manually when using doveadm director, it's as if doveadm isn't quite aware of the config in the proxy instance. However, the authentication happens in the primary "dovecot.conf" instance...that's where checkpassword gets called, and where the prefetch is configured. The proxy is "dumb". >> I'm not surprised the checkpassword lookup is failing; the password >> isn't being supplied. Shouldn't it just be doing the prefetch lookup >> instead, like the lmtp service? > > doveadm doesn't care about the password. It cares about the proxy > destination host so that it can automatically connect to the correct > host to run the command. I understand doveadm doesn't care about the password; i was confused why checkpassword was being called when the password is unknown rather than using the mysql prefetch. So, I guess what you're saying is doveadm should be talking to the proxy instance rather than the backend instance? I assume I need to move the backend to a differently named config and make my proxy instances live in dovecot.conf, so that doveadm sees the "right" config? Thanks, Andy From tss at iki.fi Tue Feb 5 04:39:07 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 05 Feb 2013 04:39:07 +0200 Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> Message-ID: <1360031947.10326.67.camel@innu> On Mon, 2013-02-04 at 21:16 -0500, Andy Dills wrote: > On Feb 4, 2013, at 8:56 PM, Timo Sirainen wrote: > > > On Mon, 2013-02-04 at 12:08 -0500, Andy Dills wrote: > >> We have a checkpassword authentication with mysql pre-fetch for the > >> userdb lookups. > >> > >> When trying to do: > >> > >> doveadm search -u andytest at xecu.net mailbox Trash DELETED > >> > >> I get: > >> > >> doveadm(andytest at xecu.net): Error: user andytest at xecu.net: Auth PASS > >> lookup failed > >> doveadm(andytest at xecu.net): Fatal: /var/run/dovecot/auth-userdb: passdb > >> lookup failed (to see if user is proxied, because doveadm_proxy_port is > >> set) > > > > Do you have proxying enabled? If not, remove doveadm_proxy_port and this > > solves itself > > Yes, I apologize, it appears I edited out the paragraph where I described our setup. > > Cluster of dovecot dual instance servers, one proxy/director, one backend. The backend config lives in dovecot.conf, the proxy config in proxy.conf (let me know if the specific config would help). Version 2.1.12 (current version in FreeBSD ports). > > I should mention, I have to specify the director-admin socket manually when using doveadm director, it's as if doveadm isn't quite aware of the config in the proxy instance. > > However, the authentication happens in the primary "dovecot.conf" instance...that's where checkpassword gets called, and where the prefetch is configured. The proxy is "dumb". OK, so you are using director, so it is good that doveadm is doing a proxy lookup, otherwise it might run it on the wrong server and cause troubles with NFS caching (which is exactly what director is supposed to prevent). > >> I'm not surprised the checkpassword lookup is failing; the password > >> isn't being supplied. Shouldn't it just be doing the prefetch lookup > >> instead, like the lmtp service? > > > > doveadm doesn't care about the password. It cares about the proxy > > destination host so that it can automatically connect to the correct > > host to run the command. > > > I understand doveadm doesn't care about the password; i was confused why checkpassword was being called when the password is unknown rather than using the mysql prefetch. > > So, I guess what you're saying is doveadm should be talking to the proxy instance rather than the backend instance? I assume I need to move the backend to a differently named config and make my proxy instances live in dovecot.conf, so that doveadm sees the "right" config? I think you need to remove doveadm_proxy_port from the backend dovecot.conf. Then it doesn't perform the PASS lookup. But you also should run doveadm via the proxy instance so that it gets run in the correct server (doveadm -c /etc/dovecot/proxy.conf or doveadm -i proxy if you've given it a name). From andy at xecu.net Tue Feb 5 05:15:19 2013 From: andy at xecu.net (Andy Dills) Date: Mon, 4 Feb 2013 22:15:19 -0500 (EST) Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <1360031947.10326.67.camel@innu> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> <1360031947.10326.67.camel@innu> Message-ID: <20130204214631.Y69315@shell.xecu.net> On Tue, 5 Feb 2013, Timo Sirainen wrote: > > So, I guess what you're saying is doveadm should be talking to the > > proxy instance rather than the backend instance? I assume I need to > > move the backend to a differently named config and make my proxy > > instances live in dovecot.conf, so that doveadm sees the "right" > > config? > > I think you need to remove doveadm_proxy_port from the backend > dovecot.conf. Thanks Timo. That gave me: doveadm -c /usr/local/etc/dovecot/proxy.conf search -u andynew at xecu.net mailbox Trash DELETED doveadm(andytest at xecu.net): Error: doveadm_password not set, can't authenticate to remote server doveadm(andytest at xecu.net): Error: 10.0.0.47:30003: Internal failure for andytest at xecu.net So, on a whim, I moved the doveadm_password out from the local {} sections into the main level, and then things started to work as expected. Is that how it should be setup now? Thanks again for help getting me straightened out! This all started because I want to start purging the trash with doveadm instead of 'find' with -delete. Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From adam at xbsd.pl Tue Feb 5 10:00:52 2013 From: adam at xbsd.pl (Adam Maciejewski) Date: Tue, 5 Feb 2013 09:00:52 +0100 Subject: [Dovecot] Subfolders problem In-Reply-To: <1360029796.10326.49.camel@innu> References: <1360029796.10326.49.camel@innu> Message-ID: # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.1 auth_debug = yes auth_master_user_separator = * auth_username_format = %Ln default_vsz_limit = 2 G dict { sieve = mysql:/etc/dovecot/dict-sql-sieve.conf sieve_dir = mysql:/etc/dovecot/dict-sql-sieve_dir.conf } disable_plaintext_auth = no dotlock_use_excl = no dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l5 log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = xbsd it solutions mail_access_groups = mail mail_debug = yes mail_location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n mail_plugins = acl namespace { hidden = no list = children location = maildir:/var/vmail/public/xbsd/Maildir:INDEX=~/mdbox/public/xbsd:CONTROL=~/mdbox/public/xbsd prefix = xbsd/ separator = / subscriptions = no type = public } namespace deleted { hidden = yes list = no location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n:MAILBOXDIR=deleted:SUBSCRIPTIONS=subscriptions-deleted prefix = .DELETED/ separator = / type = private } namespace deleted_expunged { hidden = yes list = no location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n:MAILBOXDIR=deleted_expunged:SUBSCRIPTIONS=subscriptions-deleted_expunged prefix = .DELETED/.EXPUNGED/ separator = / type = private } namespace expunged { hidden = yes list = no location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n:MAILBOXDIR=expunged:SUBSCRIPTIONS=subscriptions-expunged prefix = .EXPUNGED/ separator = / type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/master.users.%s driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile lazy_expunge = .EXPUNGED/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append save mailbox_create mail_log_fields = uid box msgid size vsize flags from subject mail_replica = remote:mail-storage-2.atm sieve = dict:proxy::sieve;name=active;bindir=~/.sieve-bin sieve_dir = dict:proxy::sieve_dir;bindir=~/.sieve-bin } protocols = imap lmtp pop3 service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service auth { executable = /usr/lib/dovecot/auth unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } unix_listener auth-userdb { mode = 0666 } user = root } service dict { unix_listener dict { mode = 0666 } } service doveadm { process_min_avail = 1 service_count = 1024 } service imap-login { chroot = login executable = /usr/lib/dovecot/imap-login inet_listener imap { address = * port = 143 } service_count = 0 user = dovecot } service imap { executable = /usr/lib/dovecot/imap } service lmtp { inet_listener lmtp { port = 24 } process_limit = 48 } service pop3-login { chroot = login executable = /usr/lib/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } service_count = 0 user = dovecot } service pop3 { executable = /usr/lib/dovecot/pop3 } service replicator { process_min_avail = 1 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol doveadm { namespace deleted { list = yes location = prefix = } namespace deleted_expunged { list = yes location = prefix = } namespace expunged { list = yes location = prefix = } } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = acl lazy_expunge notify replication } protocol lmtp { mail_plugins = acl notify replication sieve } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_plugins = notify mail_log sieve acl } 2013/2/5 Timo Sirainen > On Mon, 2013-02-04 at 09:51 +0100, Adam Maciejewski wrote: > > I have moved from dovecot 1.x to 2.x and I have big problem with > > subfolders. > > When I'm moving subfolder with other subfolders is moving only main > > subfolder, without subfolders, example : > > What v2.x exactly? Looks like a bug. Probably fixed in a newer version. > If you already have a recent v2.1, show your doveconf -n. > > > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > > -e "janusz|jarek" > > ./jarek > > ./jarek/dbox-Mails > > ./jarek/dbox-Mails/dovecot.index.log > > ./jarek/jarek2 > > ./jarek/jarek2/dbox-Mails > > ./jarek/jarek2/dbox-Mails/dovecot.index.log > > ./jarek/jarek2/jarek3 > > ./jarek/jarek2/jarek3/dbox-Mails > > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > > ./janusz > > ./janusz/dbox-Mails > > ./janusz/dbox-Mails/dovecot.index.log > > ./janusz/janusz2 > > ./janusz/janusz2/dbox-Mails > > ./janusz/janusz2/dbox-Mails/dovecot.index.log > > ./janusz/janusz2/janusz3 > > ./janusz/janusz2/janusz3/dbox-Mails > > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > > > > And now I'm moving jarek2 with jarek3 as subfolder to janusz2 subfolder, > > result : > > > > mail-storage-1 /var/vmail/home/adamskitest/mdbox/mailboxes # find | egrep > > -e "janusz|jarek" > > ./jarek > > ./jarek/dbox-Mails > > ./jarek/dbox-Mails/dovecot.index.log > > ./jarek/jarek2 > > ./jarek/jarek2/jarek3 > > ./jarek/jarek2/jarek3/dbox-Mails > > ./jarek/jarek2/jarek3/dbox-Mails/dovecot.index.log > > ./janusz > > ./janusz/dbox-Mails > > ./janusz/dbox-Mails/dovecot.index.log > > ./janusz/janusz2 > > ./janusz/janusz2/dbox-Mails > > ./janusz/janusz2/dbox-Mails/dovecot.index.log > > ./janusz/janusz2/dbox-Mails/dovecot.index.cache > > ./janusz/janusz2/janusz3 > > ./janusz/janusz2/janusz3/dbox-Mails > > ./janusz/janusz2/janusz3/dbox-Mails/dovecot.index.log > > ./janusz/janusz2/jarek2 > > ./janusz/janusz2/jarek2/dbox-Mails > > ./janusz/janusz2/jarek2/dbox-Mails/dovecot.index.log > > > > As You can see there is jarek2 subfolder but jarek3 havent been moved. I > > can see this folder in Thunderbird, after clicking error message pops out > > saying folder dosent exist. > > > > > > > -- Adam Maciejewski xBSD IT Solutions ul. R??yckiego 8 - 78-100 Ko?obrzeg tel. kom. +48 665 138 884 GG: 13821 Regon: 320797083 NIP: 671-171-49-61 From alessio at skye.it Tue Feb 5 11:07:43 2013 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 05 Feb 2013 10:07:43 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> Message-ID: <5110CBDF.4030302@skye.it> Il 04/02/2013 16:44, Timo Sirainen ha scritto: > On 4.2.2013, at 17.38, Alessio Cecchi wrote: > >>>>> Feb 04 14:03:56 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>>> Feb 04 14:06:55 imap(xxxx at yyyy.com): Error: read() failed: No such file or directory >>>>> Feb 04 14:09:09 imap(zzz at cccc.it): Error: read() failed: No such file or directory >>>> >>>> Well .. Those errors should go away if you revert this patch: >>>> >>>> http://hg.dovecot.org/dovecot-2.1/raw-rev/2b76d357a56a >>>> >>>> But does it fix the performance? .. >>> >>> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f >>> >>> >> >> Thanks Timo, >> >> tonight I will apply the second patch to 2.1.14 for the performance problem. >> >> Should I also remove the path "2b76d357a56a" or is only optional/for test? > > Removing it removes those errors from logs, but other than that it shouldn't make any difference. Instead of removing you could also apply these patches that should remove them more correctly: > > http://hg.dovecot.org/dovecot-2.1/rev/93633121bc9d > http://hg.dovecot.org/dovecot-2.1/rev/b15a98fd8e15 > Also after the application of these patches the situation of the load is the same. If there are no other solutions I will return to 2.1.13 ASAP. Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From janfrode at tanso.net Tue Feb 5 11:57:02 2013 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 5 Feb 2013 10:57:02 +0100 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: I think there must be some bug I'm hitting here. One of my directors is still running with "client_limit = 1, process_limit = 100" for the lmtp service, and now it's logging: master: Warning: service(lmtp): process_limit (100) reached, client connections are being dropped Checking "sudo netstat -anp|grep ":24 " I see 287 ports in TIME_WAIT, one in CLOSE_WAIT and the listening "0.0.0.0:24". No active connections. There are 100 lmtp-processes running. When trying to connect to the lmtp-port I immediately get dropped: $ telnet localhost 24 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. Connection closed by foreign host. Is there maybe some counter that's getting out of sync, or some back off penalty algorithm that kicks in when it first hit the process limit ? -jf From andy at xecu.net Tue Feb 5 14:53:50 2013 From: andy at xecu.net (Andy Dills) Date: Tue, 5 Feb 2013 07:53:50 -0500 (EST) Subject: [Dovecot] Errors with doveadm when using checkpassword In-Reply-To: <1360031947.10326.67.camel@innu> References: <46b12a568d2347fe5fbfe1bf24651655@xecu.net> <1360029407.10326.45.camel@innu> <1360031947.10326.67.camel@innu> Message-ID: <20130205072304.C88642@shell.xecu.net> On Tue, 5 Feb 2013, Timo Sirainen wrote: > I think you need to remove doveadm_proxy_port from the backend > dovecot.conf. Then it doesn't perform the PASS lookup. But you also > should run doveadm via the proxy instance so that it gets run in the > correct server (doveadm -c /etc/dovecot/proxy.conf or doveadm -i proxy > if you've given it a name). On a seperate note I'm sure a lot of people would benefit from -c/-i being mentioned on http://wiki2.dovecot.org/Tools/Doveadm. You are one man with only so much time so I tried registering on the wiki to propose an edit for you, but I'm not allowed. I think all we need is to know that -c and -i exist, and a note about how people in proxy/director configurations need to make sure to tell doveadm to communicate with the instance that is running director. For some reason, my intuition would be that since doveadm is aware of both instances, that it should be aware of which one's config to use for connecting to director for proxy information. Thanks, Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 --- From valerius at afterlogic.com Tue Feb 5 15:58:14 2013 From: valerius at afterlogic.com (Valery V. Sedletski) Date: Tue, 5 Feb 2013 13:58:14 +0000 Subject: [Dovecot] "Header is huge" in fts-solr Message-ID: Hi, Timo and all! I am trying to index mail in a test mailbox using fts_solr plugin for full-text search. On most mailboxes, it works fine, but on some big messages I get warnings like the following, and then I get an Out of memory error from Solr, then the indexer-worker process (or doveadm) crashes with "assertion failed" error and the backtrace: ========================================================== doveadm(valerius at test.afterlogic.com): Warning: fts-solr(valerius at test.afterlogic.com): Mailbox gmail.com UID=48 header size is huge doveadm(valerius at test.afterlogic.com): Warning: fts-solr(valerius at test.afterlogic.com): Mailbox gmail.com UID=49 header size is huge doveadm(valerius at test.afterlogic.com): Panic: file ../../../../src/plugins/fts-solr/solr-connection.c: line 548 (solr_connection_post_more): assertion failed: (maxfd >= 0) doveadm(valerius at test.afterlogic.com): Error: Raw backtrace: /usr/mailsuite/lib/dovecot/libdovecot.so.0(+0x58f04) [0x7fe8a908af04] -> /usr/mailsuite/lib/dovecot/libdovecot.so.0(default_error_handler+0) [0x7fe8a908af93] -> /usr/mailsuite/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fe8a908b274] -> /usr/mailsuite/lib/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x2d2) [0x7fe8a75fe973] -> /usr/mailsuite/lib/dovecot/lib21_fts_solr_plugin.so(+0x4d03) [0x7fe8a75f9d03] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_build_more+0x77) [0x7fe8a7c1d401] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0x7fe2) [0x7fe8a7c1dfe2] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0x80d5) [0x7fe8a7c1e0d5] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0x89e4) [0x7fe8a7c1e9e4] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2b) [0x7fe8a7c1ebf5] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0xe7cf) [0x7fe8a7c247cf] -> /usr/mailsuite/lib/dovecot/lib20_fts_plugin.so(+0xe8ba) [0x7fe8a7c248ba] -> /usr/mailsuite/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x25) [0x7fe8a9379bc9] -> /usr/mailsuite/bin/doveadm() [0x4139de] -> /usr/mailsuite/bin/doveadm() [0x413c17] -> /usr/mailsuite/bin/doveadm() [0x413f18] -> /usr/mailsuite/bin/doveadm() [0x40fea6] -> /usr/mailsuite/bin/doveadm(doveadm_mail_single_user+0x154) [0x410069] -> /usr/mailsuite/bin/doveadm() [0x41090a] -> /usr/mailsuite/bin/doveadm(doveadm_mail_try_run+0xac) [0x410b81] -> /usr/mailsuite/bin/doveadm(main+0x28d) [0x41a92c] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fe8a8cc9ead] -> /usr/mailsuite/bin/doveadm() [0x40f499] ========================================================== And Solr log, at the same time: ========================================================== 2013-02-01 18:03:53.342:INFO::Logging to STDERR via org.mortbay.log.StdErrLog 2013-02-01 18:03:53.425:INFO::jetty-6.1-SNAPSHOT 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader INFO: new SolrResourceLoader for deduced Solr Home: 'solr/' 01.02.2013 18:03:53 org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:53 org.apache.solr.core.CoreContainerSInitializer initialize INFO: looking for solr.xml: /home/valerius/apache-solr-3.6.2/example/solr/solr.xml 01.02.2013 18:03:53 org.apache.solr.core.CoreContainer load INFO: Loading CoreContainer using Solr Home: 'solr/' 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader INFO: new SolrResourceLoader for directory: 'solr/' 01.02.2013 18:03:53 org.apache.solr.core.CoreContainer create INFO: Creating SolrCore '' using instanceDir: solr/. 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader INFO: new SolrResourceLoader for directory: 'solr/./' 01.02.2013 18:03:53 org.apache.solr.core.SolrConfig initLibs INFO: Adding specified lib dirs to ClassLoader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-cell-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-ooxml-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/jdom-1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/bcprov-jdk15-1.45.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/xmlbeans-2.3.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/boilerpipe-1.1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/xercesImpl-2.8.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/tika-core-1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/metadata-extractor-2.4.0-beta-1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/apache-mime4j-core-0.7.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/fontbox-1.6.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/pdfbox-1.6.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/dom4j-1.6.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/asm-3.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/jempbox-1.6.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-ooxml-schemas-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/rome-0.9.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/bcmail-jdk15-1.45.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/tagsoup-1.2.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/icu4j-4.8.1.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/tika-parsers-1.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/poi-scratchpad-3.8-beta4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/commons-compress-1.3.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/xml-apis-1.0.b2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/extraction/lib/apache-mime4j-dom-0.7.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-clustering-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/jackson-mapper-asl-1.7.4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/mahout-collections-0.3.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/mahout-math-0.3.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/hppc-0.3.4-jdk15.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/carrot2-core-3.5.0.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/jackson-core-asl-1.7.4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/clustering/lib/simple-xml-2.4.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-dataimporthandler-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/dataimporthandler/lib/mail-1.4.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/dataimporthandler/lib/activation-1.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-langid-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/langid/lib/langdetect-1.1-20120112.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/langid/lib/jsonic-1.2.7.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/dist/apache-solr-velocity-3.6.2.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/velocity-1.6.4.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/commons-collections-3.2.1.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/commons-beanutils-1.7.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/valerius/apache-solr-3.6.2/contrib/velocity/lib/velocity-tools-2.0.jar' to classloader 01.02.2013 18:03:53 org.apache.solr.core.SolrConfig INFO: Using Lucene MatchVersion: LUCENE_36 01.02.2013 18:03:54 org.apache.solr.core.SolrConfig INFO: Loaded SolrConfig: solrconfig.xml 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: Reading Solr Schema 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: Schema name=dovecot 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created string: org.apache.solr.schema.StrField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created long: org.apache.solr.schema.LongField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created slong: org.apache.solr.schema.SortableLongField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created float: org.apache.solr.schema.FloatField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created boolean: org.apache.solr.schema.BoolField 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WhitespaceTokenizerFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.SynonymFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.StopFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WordDelimiterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.LowerCaseFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.SnowballPorterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.RemoveDuplicatesTokenFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WhitespaceTokenizerFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.StopFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.WordDelimiterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.LowerCaseFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.SnowballPorterFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.NGramFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created : org.apache.solr.analysis.RemoveDuplicatesTokenFilterFactory 01.02.2013 18:03:54 org.apache.solr.util.plugin.AbstractPluginLoader load INFO: created text: org.apache.solr.schema.TextField 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: default search field in schema is body 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: query parser default operator is AND 01.02.2013 18:03:54 org.apache.solr.schema.IndexSchema readSchema INFO: unique key field: id 01.02.2013 18:03:54 org.apache.solr.core.SolrCore INFO: [] Opening new SolrCore at solr/./, dataDir=solr/./data/ 01.02.2013 18:03:54 org.apache.solr.core.JmxMonitoredMap INFO: JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer at 27b4c1d7 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initListeners INFO: [] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]} 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initListeners INFO: [] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]} 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: created json: solr.JSONResponseWriter 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: adding lazy queryResponseWriter: solr.VelocityResponseWriter 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: created velocity: solr.VelocityResponseWriter 01.02.2013 18:03:54 org.apache.solr.core.SolrCore initWriters INFO: created xslt: solr.XSLTResponseWriter 01.02.2013 18:03:54 org.apache.solr.response.XSLTResponseWriter init INFO: xsltCacheLifetimeSeconds=5 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /select: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /browse: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update: solr.XmlUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/javabin: solr.BinaryUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.CSVRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/csv: solr.CSVRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.JsonUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/json: solr.JsonUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.extraction.ExtractingRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/extract: solr.extraction.ExtractingRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.XsltUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /update/xslt: solr.XsltUpdateRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.FieldAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /analysis/field: solr.FieldAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.DocumentAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /analysis/document: solr.DocumentAnalysisRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /admin/: solr.admin.AdminHandlers 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /admin/ping: solr.PingRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /debug/dump: solr.DumpRequestHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /spell: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /tvrh: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /terms: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: adding lazy requestHandler: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.core.RequestHandlers initHandlersFromConfig INFO: created /elevate: solr.SearchHandler 01.02.2013 18:03:54 org.apache.solr.search.SolrIndexSearcher INFO: Opening Searcher at 5c29ea31 main 01.02.2013 18:03:54 org.apache.solr.update.CommitTracker INFO: commitTracker AutoCommit: disabled 01.02.2013 18:03:54 org.apache.solr.handler.component.SpellCheckComponent inform INFO: Initializing spell checkers 01.02.2013 18:03:54 org.apache.solr.handler.component.SpellCheckComponent inform INFO: No queryConverter defined, using default converter 01.02.2013 18:03:54 org.apache.solr.handler.component.QueryElevationComponent inform INFO: Loading QueryElevation from: /home/valerius/apache-solr-3.6.2/example/solr/./conf/elevate.xml 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.QueryComponent at 4d16318b 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.FacetComponent at 6c0ec436 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.MoreLikeThisComponent at 3d73eca6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.HighlightComponent at 5b5a5cf 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.StatsComponent at 5898bbf6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding debug component:org.apache.solr.handler.component.DebugComponent at 135c40eb 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting socketTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting urlScheme to: http:// 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting connTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxConnectionsPerHost to: 20 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting corePoolSize to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maximumPoolSize to: 2147483647 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxThreadIdleTime to: 5 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting sizeOfQueue to: -1 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting fairnessPolicy to: false 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.QueryComponent at 4d16318b 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.FacetComponent at 6c0ec436 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.MoreLikeThisComponent at 3d73eca6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.HighlightComponent at 5b5a5cf 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.StatsComponent at 5898bbf6 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding component:org.apache.solr.handler.component.SpellCheckComponent at 18f2225f 01.02.2013 18:03:54 org.apache.solr.handler.component.SearchHandler inform INFO: Adding debug component:org.apache.solr.handler.component.DebugComponent at 135c40eb 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting socketTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting urlScheme to: http:// 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting connTimeout to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxConnectionsPerHost to: 20 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting corePoolSize to: 0 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maximumPoolSize to: 2147483647 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting maxThreadIdleTime to: 5 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting sizeOfQueue to: -1 01.02.2013 18:03:54 org.apache.solr.handler.component.HttpShardHandlerFactory getParameter INFO: Setting fairnessPolicy to: false 01.02.2013 18:03:54 org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener sending requests to Searcher at 5c29ea31 main 01.02.2013 18:03:54 org.apache.solr.core.CoreContainer register INFO: registering core: 01.02.2013 18:03:54 org.apache.solr.servlet.SolrDispatchFilter init INFO: user.dir=/home/valerius/apache-solr-3.6.2/example 01.02.2013 18:03:54 org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() done 01.02.2013 18:03:54 org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1330) at org.apache.solr.schema.IndexSchemaSSolrQueryAnalyzer.getAnalyzer(IndexSchema.java:408) at org.apache.solr.schema.IndexSchemaSSolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:383) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:574) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:206) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1436) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1319) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1234) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79) at org.apache.solr.search.QParser.getQuery(QParser.java:143) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:105) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:59) at org.apache.solr.core.SolrCoreS3.call(SolrCore.java:1182) at java.util.concurrent.FutureTaskSSync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutorSWorker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 01.02.2013 18:03:54 org.apache.solr.core.SolrCore execute INFO: [] webapp=null path=null params={event=firstSearcher&q=static+firstSearcher+warming+in+solrconfig.xml} status=400 QTime=22 01.02.2013 18:03:54 org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener done. 01.02.2013 18:03:54 org.apache.solr.handler.component.SpellCheckComponentSSpellCheckerListener newSearcher INFO: Loading spell index for spellchecker: default 01.02.2013 18:03:54 org.apache.solr.core.SolrCore registerSearcher INFO: [] Registered new searcher Searcher at 5c29ea31 main 01.02.2013 18:03:54 org.apache.solr.servlet.SolrServlet init INFO: SolrServlet.init() 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:54 org.apache.solr.servlet.SolrServlet init INFO: SolrServlet.init() done 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 01.02.2013 18:03:54 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: solr home defaulted to 'solr/' (could not find system property or JNDI) 01.02.2013 18:03:54 org.apache.solr.servlet.SolrUpdateServlet init INFO: SolrUpdateServlet.init() done 2013-02-01 18:03:54.449:INFO::Started SocketConnector at 0.0.0.0:8983 01.02.2013 18:04:40 org.apache.solr.core.SolrDeletionPolicy onInit INFO: SolrDeletionPolicy.onInit: commits:num=1 commit{dir=/home/valerius/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1,version=1359741540635,generation=1,filenames=[segments_1] 01.02.2013 18:04:40 org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: newest commit = 1359741540635 01.02.2013 18:05:05 org.apache.solr.update.processor.LogUpdateProcessor finish INFO: {add=[1/5cc6513b4dc6de5093660000b304ded5/valerius at test.afterlogic.com, 48/5cc6513b4dc6de5093660000b304ded5/valerius at test.afterlogic.com]} 0 24629 01.02.2013 18:05:05 org.apache.solr.common.SolrException log SEVERE: java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.index.FreqProxTermsWriterPerFieldSFreqProxPostingsArray.(FreqProxTermsWriterPerField.java:193) at org.apache.lucene.index.FreqProxTermsWriterPerFieldSFreqProxPostingsArray.newInstance(FreqProxTermsWriterPerField.java:204) at org.apache.lucene.index.ParallelPostingsArray.grow(ParallelPostingsArray.java:48) at org.apache.lucene.index.TermsHashPerField.growParallelPostingsArray(TermsHashPerField.java:157) at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:460) at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:189) at org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocFieldProcessorPerThread.java:278) at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:766) at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:2328) at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:2300) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:240) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:157) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.mortbay.jetty.servlet.ServletHandlerSCachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnectionSRequestHandler.content(HttpConnection.java:945) 2013-02-01 18:05:42.595:INFO::Shutdown hook executing 2013-02-01 18:05:42.596:INFO::Graceful shutdown SocketConnector at 0.0.0.0:8983 2013-02-01 18:05:42.596:INFO::Graceful shutdown org.mortbay.jetty.webapp.WebAppContext at 5484ff20{/solr,file:/home/valerius/apache-solr-3.6.2/example/webapps/solr.war} 2013-02-01 18:05:43.596:INFO::Stopped SocketConnector at 0.0.0.0:8983 2013-02-01 18:05:43.784:INFO::Shutdown hook complete ========================================================== So, it seems that Dovecot tries to parse messages in the mailbox, and can't correctly determine where the message header ends. So, it thinks that the message header is big, and passes very big data to Solr. When trying to index it, Solr exhausts the available memory (though, I have 8 Gb of RAM on my machine, and java eats more than 2 Gb when indexing). Then connections to Solr get closed, and maxfd is invalid, hence the assertion is failed. Note also the following error ========================================================== SEVERE: org.apache.solr.common.SolrException: undefined field text ========================================================== before an out of memory error. I also tried to tweak the decode2text.sh script to ignore all attachments bigger than 1 Mb (just test if the file is bigger than 1 Mb, and if so, return "1"). This won't help. As I understood, this is because of big header, so attachments doesn't matter. I separated the set of messages which cause this error (by their UID's). So, I can give them as a testcase, the size of them all in archive is about 40 Mb. The error can be reproduced if put all these messages into an empty mailbox, and do reindexing, via IMAP search, or via "doveadm index -u ". I use Dovecot version 2.1.13. My doveconf -n : ========================================================== # 2.1.13: /usr/mailsuite/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian wheezy/sid ext4 auth_debug = yes auth_debug_passwords = yes auth_default_realm = test.afterlogic.com auth_mechanisms = plain login auth_socket_path = /usr/mailsuite/var/run/dovecot/auth-master base_dir = /usr/mailsuite/var/run/dovecot/ debug_log_path = /usr/mailsuite/var/log/dovecot/debug.log default_internal_user = exim dict { domainquotadict = mysql:/usr/mailsuite/etc/dovecot/dovecot-domain-quota-dict.conf realmquotadict = mysql:/usr/mailsuite/etc/dovecot/dovecot-realm-quota-dict.conf userquotadict = mysql:/usr/mailsuite/etc/dovecot/dovecot-user-quota-dict.conf } disable_plaintext_auth = no info_log_path = /usr/mailsuite/var/log/dovecot/info.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /usr/mailsuite/var/log/dovecot/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_gid = exim mail_home = /usr/mailsuite/data/settings/%d/%n mail_location = maildir:/usr/mailsuite/data/%d/%n mail_plugins = listescape fts fts_solr virtual mail_uid = exim managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave spamtest spamtestplus mbox_write_locks = fcntl namespace { inbox = yes location = maildir:/usr/mailsuite/data/%d/%n prefix = separator = / type = private } namespace { hidden = yes inbox = no list = no location = virtual:/usr/mailsuite/data/virtual/:LAYOUT=maildir++:INDEX=/usr/mailsuite/data/indexes/ prefix = All/ separator = / type = private } passdb { args = /usr/mailsuite/etc/dovecot/dovecot-sql.conf driver = sql } passdb { args = /usr/mailsuite/etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_allow_append_to_spam = no antispam_backend = mailtrain antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/mailsuite/scripts/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = error antispam_spam = SPAM antispam_spam_pattern = SPAM antispam_spam_pattern_ignorecase = SPAM antispam_trash = trash;Trash;Deleted Items; Deleted Messages antispam_trash_pattern = trash;Trash;Deleted * antispam_trash_pattern_ignorecase = trash;Deleted * antispam_unsure = antispam_unsure_pattern = antispam_unsure_pattern_ignorecase = fts = solr fts_decoder = decode2text fts_solr = break-imap-search url=http://localhost:8983/solr/ quota = dict:User quota::proxy::userquotadict quota2 = dict::%d:proxy::domainquotadict quota_rule = *:storage=100G quota_rule2 = Trash:storage=+20%% quota_rule3 = Spam:ignore quota_warning = storage=99%% quota-warning user 99 %d %n quota_warning2 = storage=95%% quota-warning user 95 %d %n quota_warning3 = storage=80%% quota-warning user 80 %d %n sieve = /usr/mailsuite/data/%d/%n/sieve.sieve sieve_before = /usr/mailsuite/etc/sieve/system.sieve sieve_dir = /usr/mailsuite/data/%d/%n/filters/ sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric sieve_global_dir = /usr/mailsuite/etc/sieve/ } protocols = imap pop3 lmtp sieve sendmail_path = /usr/mailsuite/bin/sendmail service auth { unix_listener auth-client { group = exim mode = 0666 user = exim } unix_listener auth-master { group = exim mode = 0666 user = exim } user = Sdefault_internal_user } service decode2text { executable = script /usr/mailsuite/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 } user = exim } service dict { unix_listener dict { group = exim mode = 0600 user = exim } } service imap-login { inet_listener imap { port = 143 } } service imap-postlogin { executable = script-login /usr/mailsuite/scripts/dovecot-postlogin.sh user = root } service imap { executable = imap imap-postlogin } service indexer-worker { user = root vsz_limit = 768 M } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } service_count = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/mailsuite/scripts/send_mail_warnquota.sh unix_listener quota-warning { user = exim } user = exim } ssl_cert = From gsansone at eumus.edu.uy Tue Feb 5 16:10:44 2013 From: gsansone at eumus.edu.uy (Gustavo Sansone) Date: Tue, 05 Feb 2013 12:10:44 -0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <1360030140.10326.54.camel@innu> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> <1360030140.10326.54.camel@innu> Message-ID: <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> El 2013-02-05 0:09, Timo Sirainen escribi?: > On Tue, 2013-01-29 at 16:29 -0200, Gustavo Sansone wrote: >> hello everyone, i'm new to this list >> im running dovecot as imap+pop (2.1.12) and roundcube (0.8.4) as >> client. >> >> having problems on saving to imap folders, with errors on some >> accounts >> >> my folder structure is as follow: >> >> /home/user/.maildir >> /home/user/.maildir/.Drafts >> /home/user/.maildir/.Junk >> /home/user/.maildir/.Sent >> /home/user/.maildir/.Trash >> /home/user/.maildir/.Archives >> /home/user/.maildir/.Archives.2010 >> ... >> >> i'm very confused with the namespace settings. i don't understand >> what >> is the correct way to configure it. >> don't need public or shared folders. > > Easiest to use the default namespace settings. There you wouldn't > have > any namespace prefix. if i just remove the namespace 'INBOX.' , on 10-mail.conf then there is something that creates new Send and Draft folders into my maildir folder like this INBOX.Send and INBOX.Drafts. and Roundcube shows me these new folders under INBOX (screenshot attached). can't figure out why it happens. >> on dovecot.log i can see this error with my account (and all the >> others), but it seems to work ok using roundcube: >> imap(gsansone): Error: autocreate: No namespace found for mailbox: >> Junk > > In your current setup you have namespace prefix=INBOX. so the mailbox > name would be INBOX.Junk. > >> with a few accounts, i have this on roundcube log: >> IMAP Error: Could not save message in Drafts in >> /var/www/localhost/htdocs/mail/program/steps/mail/sendmail.inc >> with error on saving to imap folder in roundcube UI > > Roundcube also isn't configured to have INBOX. prefix. Sounds like > you > should just remove it. -------------- next part -------------- A non-text attachment was scrubbed... Name: maildir.png Type: image/png Size: 49918 bytes Desc: not available URL: From tss at iki.fi Tue Feb 5 16:54:21 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 5 Feb 2013 16:54:21 +0200 Subject: [Dovecot] lmtp-proxying in 2.1 slower than in 2.0.14 ? In-Reply-To: References: <714F2F1B-D5FB-40C0-B720-EF0252B0696D@iki.fi> Message-ID: <3E83F6B6-0DDA-4B7B-B796-00E831F39784@iki.fi> On 5.2.2013, at 11.57, Jan-Frode Myklebust wrote: > I think there must be some bug I'm hitting here. One of my directors > is still running with "client_limit = 1, process_limit = 100" for the > lmtp service, and now it's logging: > > master: Warning: service(lmtp): process_limit (100) reached, client > connections are being dropped > > Checking "sudo netstat -anp|grep ":24 " I see 287 ports in TIME_WAIT, > one in CLOSE_WAIT and the listening "0.0.0.0:24". No active > connections. There are 100 lmtp-processes running. Sounds like the LMTP processes are hanging for some reason.. http://hg.dovecot.org/dovecot-2.1/rev/63117ab893dc might show something interesting, although I'm pretty sure it will just say that the processes are hanging in DATA command. Other interesting things to check: gdb -p bt full strace -tt -p (for a few seconds to see if anything is happening) If lmtp proxy is hanging, it should have a timeout (default 30 secs) and it should log about it if it triggers. (Although maybe not to error log.) > When trying to > connect to the lmtp-port I immediately get dropped: > > $ telnet localhost 24 > Trying 127.0.0.1... > Connected to localhost.localdomain (127.0.0.1). > Escape character is '^]'. > Connection closed by foreign host. This happens when the master process notices that all the service processes are full. > Is there maybe some counter that's getting out of sync, or some back > off penalty algorithm that kicks in when it first hit the process > limit ? Shouldn't be, but the proctitle patch should make it clearer. Strange anyway, I haven't heard of anything like this happening before. From tss at iki.fi Tue Feb 5 17:46:43 2013 From: tss at iki.fi (Timo Sirainen) Date: Tue, 5 Feb 2013 17:46:43 +0200 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <5110CBDF.4030302@skye.it> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <5110CBDF.4030302@skye.it> Message-ID: <5C275EA9-B689-4845-AE30-CF6880FFBB8E@iki.fi> On 5.2.2013, at 11.07, Alessio Cecchi wrote: >>>>> But does it fix the performance? .. >>>> >>>> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f > > Also after the application of these patches the situation of the load is the same. > > If there are no other solutions I will return to 2.1.13 ASAP. You definitely included the above patch? That's the important one. And it really should have fixed the performance problems.. From alessio at skye.it Tue Feb 5 18:02:15 2013 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 05 Feb 2013 17:02:15 +0100 Subject: [Dovecot] Upgrade from 2.1.13 to 2.1.14 and load doubled In-Reply-To: <5C275EA9-B689-4845-AE30-CF6880FFBB8E@iki.fi> References: <510F7C4C.7020303@skye.it> <510FB761.3040908@skye.it> <1C270E44-D98F-4011-8E99-EE941B949100@iki.fi> <1359990439.3230.4.camel@hurina> <510FD5F6.60808@skye.it> <5110CBDF.4030302@skye.it> <5C275EA9-B689-4845-AE30-CF6880FFBB8E@iki.fi> Message-ID: <51112D07.9090506@skye.it> Il 05/02/2013 16:46, Timo Sirainen ha scritto: > On 5.2.2013, at 11.07, Alessio Cecchi wrote: > >>>>>> But does it fix the performance? .. >>>>> >>>>> No, but this should: http://hg.dovecot.org/dovecot-2.1/rev/443ff272317f >> >> Also after the application of these patches the situation of the load is the same. >> >> If there are no other solutions I will return to 2.1.13 ASAP. > > You definitely included the above patch? That's the important one. And it really should have fixed the performance problems.. > Sure Timo, during in the afternoon the situations seems better, wait a few days for tests. Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From arnaud.abelard at univ-nantes.fr Tue Feb 5 19:33:16 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 18:33:16 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 Message-ID: <5111425C.4000902@univ-nantes.fr> Hello, I've upgraded our servers and now dovecot seems to be running out of memory when accessing my own mailbox (and only mine, which is in a way pretty fortunate): dovecot: imap(abelard-a): Error: mmap() failed with file /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is 72% full of 8GB. any idea of what is happening? Thanks in advance, Arnaud -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From arnaud.abelard at univ-nantes.fr Tue Feb 5 19:38:51 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 18:38:51 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <5111425C.4000902@univ-nantes.fr> References: <5111425C.4000902@univ-nantes.fr> Message-ID: <511143AB.3060501@univ-nantes.fr> Hrm... exactly when I pressed sent, dovecot started giving me more information: master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it)) Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were running...). Arnaud On 02/05/2013 06:33 PM, Arnaud Ab?lard wrote: > Hello, > > I've upgraded our servers and now dovecot seems to be running out of > memory when accessing my own mailbox (and only mine, which is in a way > pretty fortunate): > > dovecot: imap(abelard-a): Error: mmap() failed with file > /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory > > imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory > > /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is > 72% full of 8GB. > > any idea of what is happening? > > Thanks in advance, > > Arnaud > > -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From h.reindl at thelounge.net Tue Feb 5 19:44:25 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 05 Feb 2013 18:44:25 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <511143AB.3060501@univ-nantes.fr> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> Message-ID: <511144F9.30408@thelounge.net> hard to say without config-details shared prcoess or one per connection? in case of shared 256 MB is really really low Am 05.02.2013 18:38, schrieb Arnaud Ab?lard: > Hrm... exactly when I pressed sent, dovecot started giving me more information: > > master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you > may need to increase it)) > > Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How > come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were > running...). > > Arnaud > > On 02/05/2013 06:33 PM, Arnaud Ab?lard wrote: >> Hello, >> >> I've upgraded our servers and now dovecot seems to be running out of >> memory when accessing my own mailbox (and only mine, which is in a way >> pretty fortunate): >> >> dovecot: imap(abelard-a): Error: mmap() failed with file >> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >> >> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory >> >> /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is >> 72% full of 8GB. >> >> any idea of what is happening? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From arnaud.abelard at univ-nantes.fr Tue Feb 5 20:15:28 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 19:15:28 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <511144F9.30408@thelounge.net> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> <511144F9.30408@thelounge.net> Message-ID: <51114C40.3050801@univ-nantes.fr> On 02/05/2013 06:44 PM, Reindl Harald wrote: > hard to say without config-details > > shared prcoess or one per connection? One per connection, around 1000 connections right now. > in case of shared 256 MB is really really low I changed the vsz_limit to 512MB and it seems better, but I'm still surprised my mailbox actually hit the memory limit since I doubt it's most used one. Arnaud > > Am 05.02.2013 18:38, schrieb Arnaud Ab?lard: >> Hrm... exactly when I pressed sent, dovecot started giving me more information: >> >> master: Error: service(imap): child 22324 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you >> may need to increase it)) >> >> Is that why the imap processes were running out of memory or is that just another symptom of the same problem? How >> come we never had that problem with the older version of dovecot 2.0 (sorry, i can't remember which one we were >> running...). >> >> Arnaud >> >> On 02/05/2013 06:33 PM, Arnaud Ab?lard wrote: >>> Hello, >>> >>> I've upgraded our servers and now dovecot seems to be running out of >>> memory when accessing my own mailbox (and only mine, which is in a way >>> pretty fortunate): >>> >>> dovecot: imap(abelard-a): Error: mmap() failed with file >>> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >>> >>> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory >>> >>> /vmail/a/b/abelard-a/dovecot.index.cache is 224MB big and my mailbox is >>> 72% full of 8GB. >>> >>> any idea of what is happening? > -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From dz at ct.heise.de Tue Feb 5 20:36:29 2013 From: dz at ct.heise.de (Dusan Zivadinovic) Date: Tue, 5 Feb 2013 19:36:29 +0100 Subject: [Dovecot] dsync: Invalid server handshake Message-ID: <36C41C6B-F529-4787-A12E-C55975B44629@ct.heise.de> Hi list, I recently tried to backup mailboxes from an older server machine to a new one in order to move the service to the new machine. Both machines are in the same LAN, I used this command: dsync -R -u username backup ssh -i .ssh/id_rsa username at 192.168.1.11 /opt/local/bin/dsync and I get this error: dsync-local(dz): Error: Invalid server handshake: dsync-server 2 dsync-remote(dz): Error: Invalid client handshake: dsync-client 1 # lokal machine: dovecot 2.0.19apple1 running on Mac OS X 10.8.2 Server using standard Apple-Dovecot-Configuration # remote machine: 2.1.12 running on Mac OS X 10.4 using PAM authentication I cant seem to find any documentation to this error, nor do I find any dsync entries in the logs of both machines. Does anyone have a hint? Thank you, Dusan From radikn at seznam.cz Tue Feb 5 21:52:15 2013 From: radikn at seznam.cz (=?UTF-8?Q?Radek_Novotn=C3=BD?=) Date: Tue, 05 Feb 2013 20:52:15 +0100 Subject: [Dovecot] Per user special-use folder names Message-ID: Hi all, let me ask a question, please. Is it possible in dovecot to set up per user special-use folder names? Imagine situation with two users where first prefere another language that second. mailbox Sent { special_use = \Sent } for english speaking users and mailbox "Odeslan? po?ta" { special_use = \Sent } for czech speaking users. Thanks for your answers. Radek From dave at dawoodfall.net Tue Feb 5 22:14:51 2013 From: dave at dawoodfall.net (David Woodfall) Date: Tue, 5 Feb 2013 20:14:51 +0000 Subject: [Dovecot] Using Mutt - folder atime/mtime etc In-Reply-To: <20130204105653.GA21474@nihlus.leuxner.net> References: <20130204020105.GA15960@Blackswan> <20130204020824.GA14425@daniel.localdomain> <510F1AB5.6080503@dawoodfall.net> <20130204105653.GA21474@nihlus.leuxner.net> Message-ID: <20130205201451.GA26648@Blackswan> On Mon, Feb 04, 2013 at 11:56:53AM +0100, Thomas Leuxner put forth the proposition: >* David Woodfall 2013.02.04 03:19: > >> >It looks as if folder_format option of mutt >> >as only for local folders, and will not work for IMAP. >> >> I was grasping at straws and hoping someone here may know. > >As mentioned this may be one for the mutt list. But it should work with IMAP: > >set imap_user="user at domain.tld" >set folder="imap://host.domain.tld/" >set spoolfile="imap://host.domain.tld/INBOX" >set index_format="%4C %Z %2M %[!%Y.%m.%d %H:%M] %-30.30F (%5c) %s" After much messing about with mutt and suggestions on the list, it seems that when mutt displays the imap mailbox list it works, but the normal folder browser doesn't work with %N. Thanks for you help From stan at hardwarefreak.com Tue Feb 5 22:25:35 2013 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 05 Feb 2013 14:25:35 -0600 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <51114C40.3050801@univ-nantes.fr> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> <511144F9.30408@thelounge.net> <51114C40.3050801@univ-nantes.fr> Message-ID: <51116ABF.2080204@hardwarefreak.com> On 2/5/2013 12:15 PM, Arnaud Ab?lard wrote: > I changed the vsz_limit to 512MB and it seems better, but I'm still > surprised my mailbox actually hit the memory limit since I doubt it's > most used one. According to the wiki, vsz_limit only affects login processes, not IMAP processes, which is odd given the second error you posted, which seems to indicate a relationship between vsz_limit and imap. >>>> dovecot: imap(abelard-a): Error: mmap() failed with file >>>> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >>>> >>>> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory This error suggests your mail_process_size variable is set too low. If it is currently set to 128MB, increase it to 512MB. The bulk of this memory is used for mmap()ing files, and is virtual, not physical. Thus you don't actually need 51GB of RAM to support 100 users. Also, a dovecot.index.cache file of size 224MB seems rather large. My largest list mail folder is 150MB, contains 17,647 msgs, and has a dovecot.index.cache file of only 16MB. I use mbox. This would seem to suggest you may have a single folder with tens of thousands of msgs in it. Or maybe indexes are handled differently for your mailbox format--I've not used any others. If the former, I suggest you cull your large folders to get them down to manageable size. -- Stan From p at sys4.de Tue Feb 5 22:28:24 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Tue, 5 Feb 2013 21:28:24 +0100 Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: <20130205202824.GA23270@sys4.de> * Radek Novotn? : > Hi all, > > let me ask a question, please. Is it possible in dovecot to set up > per user special-use folder names? > > Imagine situation with two users where first prefere another > language that second. You don't need per-user folder SPECIAL-USE names, because the client must take care of the correct mapping. If the client runs in an German environment it might mount the "special_use = \Sent" mailbox as "Gesendete Objekte" and if it is Czechian it might call it "Odeslan? po?ta". That's part of what makes SPECIAL-USE so sexy. It is language independent. All it does is say "This mailbox is reserved for that particular usage." How you call it, is up to you (client). p at rick > > mailbox Sent { > special_use = \Sent > } > > for english speaking users and > > mailbox "Odeslan? po?ta" { > special_use = \Sent > } > > for czech speaking users. > > > > Thanks for your answers. Radek -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From slusarz at curecanti.org Tue Feb 5 22:50:12 2013 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 05 Feb 2013 13:50:12 -0700 Subject: [Dovecot] Per user special-use folder names In-Reply-To: <20130205202824.GA23270@sys4.de> References: <20130205202824.GA23270@sys4.de> Message-ID: <20130205135012.Horde.mB4kE_oo2esQ-uGfpIN3rg2@bigworm.curecanti.org> Quoting Patrick Ben Koetter

: > That's part of what makes SPECIAL-USE so sexy. It is language independent. > All it does is say "This mailbox is reserved for that particular usage." How > you call it, is up to you (client). Well.... not quite. The problem comes when you have *multiple* sent mailboxes on your server, which is perfectly acceptable and quite useful (e.g. an MUA allows multiple identities, and each identity uses a separate sent-mail mailbox). You can't just blindly show the local translation for "Sent" for all of the mailboxes, or else you've now eliminated the user's ability to differentiate between them. In practical use, SPECIAL-USE isn't tremendously helpful for "auto-configuration" of an MUA because of these kind of vagaries. michael From arnaud.abelard at univ-nantes.fr Tue Feb 5 23:16:53 2013 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 05 Feb 2013 22:16:53 +0100 Subject: [Dovecot] Out of memory after upgrading to dovecot 2.0.21 In-Reply-To: <51116ABF.2080204@hardwarefreak.com> References: <5111425C.4000902@univ-nantes.fr> <511143AB.3060501@univ-nantes.fr> <511144F9.30408@thelounge.net> <51114C40.3050801@univ-nantes.fr> <51116ABF.2080204@hardwarefreak.com> Message-ID: <511176C5.8010501@univ-nantes.fr> On 02/05/2013 09:25 PM, Stan Hoeppner wrote: > On 2/5/2013 12:15 PM, Arnaud Ab?lard wrote: > >> I changed the vsz_limit to 512MB and it seems better, but I'm still >> surprised my mailbox actually hit the memory limit since I doubt it's >> most used one. > > According to the wiki, vsz_limit only affects login processes, not IMAP > processes, which is odd given the second error you posted, which seems > to indicate a relationship between vsz_limit and imap. Actually, service { vsz_limit } replaced mail_process_size (at least according to what dovecot said upon restart...) so service imap { vsz_limit=512MB } which i added earlier actually does what you suggested. > >>>>> dovecot: imap(abelard-a): Error: mmap() failed with file >>>>> /vmail/a/b/abelard-a/dovecot.index.cache: Cannot allocate memory >>>>> >>>>> imap(abelard-a): Fatal: pool_system_realloc(131072): Out of memory > > This error suggests your mail_process_size variable is set too low. If > it is currently set to 128MB, increase it to 512MB. The bulk of this > memory is used for mmap()ing files, and is virtual, not physical. Thus > you don't actually need 51GB of RAM to support 100 users. > > Also, a dovecot.index.cache file of size 224MB seems rather large. My > largest list mail folder is 150MB, contains 17,647 msgs, and has a > dovecot.index.cache file of only 16MB. I use mbox. This would seem to > suggest you may have a single folder with tens of thousands of msgs in > it. Or maybe indexes are handled differently for your mailbox > format--I've not used any others. If the former, I suggest you cull > your large folders to get them down to manageable size. Yep I have a huge INBOX folder right now, I haven't archived my 2011 mail yet (I keep the previous year in my INBOX). Thanks, Arnaud > -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From forall at stalowka.info Tue Feb 5 23:49:18 2013 From: forall at stalowka.info (For@ll) Date: Tue, 05 Feb 2013 22:49:18 +0100 Subject: [Dovecot] Dovecot main process killed by KILL signal Message-ID: Hi, I have a dovecot 2.0.7 installed on ubuntu 12.10 and often in dmesg I see "init: dovecot main process (7104) killed by KILL signal", I must restarted dovecot, because I can't access to mailbox. Server hardware: 2x1TB disks 4GB memory 1xCPU 2,4GHz HARDWARE RAID 1 Mailbox ~ 1k dovecot.conf # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-22-generic x86_64 Ubuntu 12.10 xfs auth_mechanisms = plain login auth_verbose = yes default_client_limit = 1500 info_log_path = /var/log/dovecot.info log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/mail/virtual/%d/%n/ maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } passdb { driver = pam } plugin { sieve_dir = /var/mail/virtual/%d/%n/sieve sieve_global_dir = /var/mail/virtual/sieve } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = root } service imap-login { inet_listener imap { port = 0 } process_limit = 512 } service imap { process_limit = 512 service_count = 0 } service managesieve-login { inet_listener sieve { port = 33919 } } service pop3-login { inet_listener pop3 { port = 0 } } ssl_cert = References: Message-ID: W dniu 2013-02-05 22:49, For at ll pisze: > Hi, > > I have a dovecot 2.0.7 installed on ubuntu 12.10 and often in dmesg I > see "init: dovecot main process (7104) killed by KILL signal", I must > restarted dovecot, because I can't access to mailbox. Sorry, I have dovecot 2.1.7. From slusarz at curecanti.org Wed Feb 6 01:02:38 2013 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 05 Feb 2013 16:02:38 -0700 Subject: [Dovecot] Possible sort optimization (?) Message-ID: <20130205160238.Horde.lxh7MCMj2ohf5zU-z9VJug6@bigworm.curecanti.org> Maybe this is just noise... but I can reproduce this fairly reliably. Mailbox with 21,000+ messages This query: a UID SORT RETURN (ALL COUNT) (DATE) UTF-8 SUBJECT "foo" is always about 10 percent slower than this split query (I've done this 4-5 times, and the numbers are similar): a UID SEARCH RETURN (SAVE) CHARSET UTF-8 SUBJECT "foo" b UID SORT RETURN (ALL COUNT) (DATE) UTF-8 UID $ (The particular query I used matched 5 messages out of the 21,000+) My not-very-scientific benchmarking process: 1.) Stop dovecot process 2.) Delete all dovecot index files for that mailbox 3.) Flush linux paging cache (sync && echo 3 > /proc/sys/vm/drop_caches) 4.) Restart dovecot 5.) Access dovecot via command-line (PREAUTH) 6.) SELECT mailbox 7.) Issue command(s) Could be a potential area for performance improvement or could simply be lazy benchmarking. michael From joe.beaubien at gmail.com Wed Feb 6 05:41:59 2013 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Tue, 5 Feb 2013 22:41:59 -0500 Subject: [Dovecot] dsync backup questions In-Reply-To: <1360029680.10326.48.camel@innu> References: <20130204005729.GA68109@anubis.morrow.me.uk> <1360029680.10326.48.camel@innu> Message-ID: On Mon, Feb 4, 2013 at 9:01 PM, Timo Sirainen wrote: > On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote: > > I can't give authoratitive answers to either of these, but... > > > > At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: > > > > > > I'm currently trying to setup remote backups of my emails but i'm > running > > > into issues (mdbox format, indexes and storage in the same folder > > > hierarchy). > > > > > > Local backup command: dsync -u "my_user" backup /backups/my_user > > > > > > (1) Recently, I noticed that the local backup takes up twice the size > as > > > the original mail location (8gb vs. 4gb). I purged alot of emails from > the > > > original location, so the size shrunk, but the local backup just keeps > on > > > getting bigger. I couldn't find any dsync option that would delete > extra > > > emails. > > > > > > - Question: Why isn't the local backup synced properly and remove the > extra > > > emails? > > > > Are you running 'doveadm purge' on the backed-up dbox? It looks to me as > > though dsync doesn't do that. I don't know if there's any (simple) way > > to do that without a running Dovecot instance attached the dbox > > directory: it's not entirely clear to me whether doveadm will run > > locally without contacting a doveadm-server instance running under > > Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. > > Right. doveadm -o mail=mdbox:/backups/my_user purge > This worked (at least it seems to, the source and destination are roughly the same size).. However, if the the original email location has already been purge, does the backup email location also need to be purged? > > It might be easier to dsync to a Maildir instead. This should preserve > > all the Dovecot-visible metadata, and dsyncing back to the dbox for > > restore should put it all back. > > Better sdbox than maildir. > I'd rather stick to mdbox for my remote backups. I have a single email account with over 1.5 million emails in it. With a 1-file-per-message storage, this would be slow/hell to run. Unless there is a better way. > > > > (2) What is the best why to copy this local backup to a remote location > > > that does NOT have the possibility to run dsync. > > > > > > - Question 1: is rsync safe to use and will this data work for restore? > > > > > > - Question 2: Would it be safe to simply rsync the original > mail_location > > > to the remote server? > > > > AFAICT, if Dovecot is stopped on both sides of the transfer it should be > > safe. If either side has a currently running Dovecot instance (or any > > other Dovecot tools, like a concurrent dsync) using that dbox, it's > > likely rsync will copy an inconsistent snapshot of the data, and the > > result will be corrupted. > > It won't be badly corrupted though. At worst Dovecot will rebuild the > indexes, which takes a while. And most users probably won't get any > corruption at all. > > > I think there was a misunderstanding of the setup. dsync is only running on the local side, the remote side is a "dumb" rsync server that I don't fully control. From joe.beaubien at gmail.com Wed Feb 6 06:00:59 2013 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Tue, 5 Feb 2013 23:00:59 -0500 Subject: [Dovecot] dsync backup questions In-Reply-To: References: <20130204005729.GA68109@anubis.morrow.me.uk> <1360029680.10326.48.camel@innu> Message-ID: On Tue, Feb 5, 2013 at 10:41 PM, Joe Beaubien wrote: > > > > On Mon, Feb 4, 2013 at 9:01 PM, Timo Sirainen wrote: > >> On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote: >> > I can't give authoratitive answers to either of these, but... >> > >> > At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: >> > > >> > > I'm currently trying to setup remote backups of my emails but i'm >> running >> > > into issues (mdbox format, indexes and storage in the same folder >> > > hierarchy). >> > > >> > > Local backup command: dsync -u "my_user" backup /backups/my_user >> > > >> > > (1) Recently, I noticed that the local backup takes up twice the size >> as >> > > the original mail location (8gb vs. 4gb). I purged alot of emails >> from the >> > > original location, so the size shrunk, but the local backup just >> keeps on >> > > getting bigger. I couldn't find any dsync option that would delete >> extra >> > > emails. >> > > >> > > - Question: Why isn't the local backup synced properly and remove the >> extra >> > > emails? >> > >> > Are you running 'doveadm purge' on the backed-up dbox? It looks to me as >> > though dsync doesn't do that. I don't know if there's any (simple) way >> > to do that without a running Dovecot instance attached the dbox >> > directory: it's not entirely clear to me whether doveadm will run >> > locally without contacting a doveadm-server instance running under >> > Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. >> >> Right. doveadm -o mail=mdbox:/backups/my_user purge >> > > This worked (at least it seems to, the source and destination are roughly > the same size).. > > However, if the the original email location has already been purge, does > the backup email location also need to be purged? > > >> > It might be easier to dsync to a Maildir instead. This should preserve >> > all the Dovecot-visible metadata, and dsyncing back to the dbox for >> > restore should put it all back. >> >> Better sdbox than maildir. >> > > I'd rather stick to mdbox for my remote backups. I have a single email > account with over 1.5 million emails in it. With a 1-file-per-message > storage, this would be slow/hell to run. Unless there is a better way. > > >> >> > > (2) What is the best why to copy this local backup to a remote >> location >> > > that does NOT have the possibility to run dsync. >> > > >> > > - Question 1: is rsync safe to use and will this data work for >> restore? >> > > >> > > - Question 2: Would it be safe to simply rsync the original >> mail_location >> > > to the remote server? >> > >> > AFAICT, if Dovecot is stopped on both sides of the transfer it should be >> > safe. If either side has a currently running Dovecot instance (or any >> > other Dovecot tools, like a concurrent dsync) using that dbox, it's >> > likely rsync will copy an inconsistent snapshot of the data, and the >> > result will be corrupted. >> >> It won't be badly corrupted though. At worst Dovecot will rebuild the >> indexes, which takes a while. And most users probably won't get any >> corruption at all. >> >> >> > I think there was a misunderstanding of the setup. dsync is only running > on the local side, the remote side is a "dumb" rsync server that I don't > fully control. > > > what i was trying to ask with my last question is the following: I'm trying to do remote backups of an mdbox setup, and considering that I only have dsync running on the local side (not on the remote side), is it safe to simply do an rsync of the mail_location to the remote server, or should I first do a dsync (make a local duplicate) and then rsync the duplcate out to the remote server? (wish i had a whiteboard right about now). Thanks, -Joe From jtam.home at gmail.com Wed Feb 6 06:51:44 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 5 Feb 2013 20:51:44 -0800 (PST) Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On Wed, 6 Feb 2013, Michael M Slusarz wrote: > Quoting Patrick Ben Koetter

: > >> That's part of what makes SPECIAL-USE so sexy. It is language independent. >> All it does is say "This mailbox is reserved for that particular usage." How >> you call it, is up to you (client). > > Well.... not quite. The problem comes when you have *multiple* sent > mailboxes on your server, which is perfectly acceptable and quite > useful (e.g. an MUA allows multiple identities, and each identity uses > a separate sent-mail mailbox). You can't just blindly show the local > translation for "Sent" for all of the mailboxes, or else you've now > eliminated the user's ability to differentiate between them. On a related topic, what's the easiest way to alias various common mailbox names to one physical mailbox? For example, mapping "Trash", "Deleted Messages", "Junk" to the same mailbox? Would you use the SPECIAL-USE, or is there a better way to do this? Namescape configuration? Virtual plugin? Joseph Tam From ya.mwork at yandex.ru Wed Feb 6 08:09:12 2013 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Wed, 06 Feb 2013 10:09:12 +0400 Subject: [Dovecot] dovecot-2.2: dsync to imapc not working Message-ID: <5111F388.9030305@yandex.ru> Hello. I have dovecot installation # dovecot --version 20130205 (03a0af22100d+) built with imapc backend. I'm tried to sync mailboxes from another server after clean mail directory localy: # dsync -v -o imapc_user="user at example.org" -o imapc_password="pass" -o imapc_host=imap.example.org -o imapc_features=rfc822.size -o mailbox_list_index=no backup -R -f -u "user at example.org" imapc: and get this message dsync(user at example.org): Error: Exporting mailbox INBOX failed: Backend doesn't support GUIDs, sync with header hashes instead Repeated command returns this one message: dsync(user at example.org): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. What wrong with this build? Maybe there is another way to do it? For example create backup in local temporary directory and synchronize this one and working storage. From skdovecot at smail.inf.fh-brs.de Wed Feb 6 10:10:47 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 Feb 2013 09:10:47 +0100 (CET) Subject: [Dovecot] Dovecot main process killed by KILL signal In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 5 Feb 2013, For at ll wrote: > I have a dovecot 2.0.7 installed on ubuntu 12.10 and often in dmesg I see > "init: dovecot main process (7104) killed by KILL signal", I must restarted Your Dovecot gets killed with a sure kill (aka kill -9). Somebody or some process wants your Dovecot stopped for sure. Do you have something in your syslog and/or Dovecot log? Does the message comes up, _because_ you stop or restart Dovecot? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURIQB13r2wJMiz2NAQIRGAf/ei4soM0f4wn7UFb0tsQsTJUJ8YfD1cAN OcSviw9BfmN7RiqUL8H0sICRINHWBnz8oRmHf/2zGGWZGQTOlbVNoUefZ423I5nH HaE73A1vhbi+wmKh5DvStn6NB0WxVs6U8LFXPzryzV1haeS3pM2KOgf1YVRaagfN VVcgz1in81NUhtqdmVTEzyauI0mbquCclTuQ9EmynYlncc9T48UIt0Q7ApMiA3Kb VZ/ms5xLxdTbD9gS+SAbWfALA15kU9+EMgdwMqGvPWTlSc9CD/I0tzyHYkmFbSVJ fdru3g1RTZGUd4yJwOcrevoo1j7CW8/+fuEPrCVU7D4WRmRujTDhig== =dayi -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Wed Feb 6 12:10:25 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 Feb 2013 11:10:25 +0100 (CET) Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: <510FF702.4030208@mclien.de> References: <510BEBDC.6020802@mclien.de> <510FF702.4030208@mclien.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 4 Feb 2013, Frank wrote: > Steffen Kaiser wrote: >> On Fri, 1 Feb 2013, Frank Lienhard wrote: >> >>> My first attempt was tho simply rsync the Maildirs along with the homes. >>> Both deovecot versions are set up with the maildir format, but this >>> results in ~30% doublicated mails on the new server. >> >> Er, a) you got 30% more files on the new server (aka target), than you have >> on the old server (aka source)?? >> >> b) Or do you see them, when you login via IMAP to the the new server? >> >> === >> >> If a), did you ran rsync, while the source host is in production still? Do >> you use "H" option to preserve hardlinks and/or "--delete" if you run rsync >> multiple times? >> >> If b), did you verified the duplication with a IMAP client, that does _no_ >> caching on its own or with a fresh and empty local cache / profile / ... . >> > I used -avu, where: > -a equals: -rlptgoD (no -H,-A,-X) > -u skip file that are newer on the receiver > > I stopped the getmail cronjob to ensure that no new messages get fetched > while sync. After that I created a new account in Icedove and at first all > went well, but after a while I got duoplicates of abou 900 old messages. > I admit I only looked that up from Icedove and not in the dir from cli. When you rsync'ed multiple times, you _must_ use "--delete" as well, because in Maildir status changes, such as "read", "seen", "replied", lables a.s.o., are reflected by changing the filename of the messages. Hence, without "--delete" you get the same message multiple times with different status or labels. I tried it myself and had 1:1 copies of the mailboxes, regardless if I use the same or a fresh profile in Thunderbird. I use: rsync -essh -vaH --delete from/ target:/.../to/ - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURIsEV3r2wJMiz2NAQLNQwgAwRiTbnrPlgmKIYUMwDVWo5tztighb8+b WoYjr4XkK5SkxJH1ZUnsaNzoKYC0j/Zf+PE/kK9C32X+itkIhH8f3L8jITCzj3RR kl6/j/OE3hnMDJNSfoYabmdpzc9knXabj04GVbZNW0cwXX33eU3CQd9yHbno3Raq DVsnCIe23L7sO7giNORUIpZJuhzfri8YpvAdZsypaK0ZfMXbBxYN6lPaoEPkYcKH T91EvjbeGKJGpAKokjtyCIrBbvYPW6ARyD/y7G7qrGFvUqqZTzhEUbEPuZ6kCkFS Zc5bmv90BcQ/O1U6U2jH72A3UeCsCRBl9BWp9Cvs5P5Pbi1kyzvZrA== =GtYM -----END PGP SIGNATURE----- From h.reindl at thelounge.net Wed Feb 6 12:27:50 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 06 Feb 2013 11:27:50 +0100 Subject: [Dovecot] move mail from server with v1.0 to server w. v2.1? In-Reply-To: References: <510BEBDC.6020802@mclien.de> <510FF702.4030208@mclien.de> Message-ID: <51123026.2000400@thelounge.net> Am 06.02.2013 11:10, schrieb Steffen Kaiser: > On Mon, 4 Feb 2013, Frank wrote: > >> Steffen Kaiser wrote: >>> On Fri, 1 Feb 2013, Frank Lienhard wrote: >>> >>>> My first attempt was tho simply rsync the Maildirs along with the homes. Both deovecot versions are set up with >>>> the maildir format, but this results in ~30% doublicated mails on the new server. >>> >>> Er, a) you got 30% more files on the new server (aka target), than you have on the old server (aka source)?? >>> >>> b) Or do you see them, when you login via IMAP to the the new server? >>> >>> === >>> >>> If a), did you ran rsync, while the source host is in production still? Do you use "H" option to preserve >>> hardlinks and/or "--delete" if you run rsync multiple times? >>> >>> If b), did you verified the duplication with a IMAP client, that does _no_ caching on its own or with a fresh >>> and empty local cache / profile / ... . >>> >> I used -avu, where: >> -a equals: -rlptgoD (no -H,-A,-X) >> -u skip file that are newer on the receiver > >> I stopped the getmail cronjob to ensure that no new messages get fetched while sync. After that I created a new >> account in Icedove and at first all went well, but after a while I got duoplicates of abou 900 old messages. >> I admit I only looked that up from Icedove and not in the dir from cli. > > When you rsync'ed multiple times, you _must_ use "--delete" as well, because in Maildir status changes, such as > "read", "seen", "replied", lables a.s.o., are reflected by changing the filename of the messages. Hence, without > "--delete" you get the same message multiple times with different status or labels. > > I tried it myself and had 1:1 copies of the mailboxes, regardless if I use the same or a fresh profile in Thunderbird. > > I use: rsync -essh -vaH --delete from/ target:/.../to/ generally you should use options to REALLY make a folder a 1:1 copy, the params below are used by me since many years for ANY sort of folder sync --delete-after -tPrlHpogEAX # -z compress # -t timestamps # -P progress # -r recursive # -l links # -H hard-links # -p permissions # -o owner # -g group # -E executability # -A acls # -X xtended attributes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From ya.mwork at yandex.ru Wed Feb 6 14:07:42 2013 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Wed, 06 Feb 2013 16:07:42 +0400 Subject: [Dovecot] dovecot-2.2: broken domain quota calculation Message-ID: <5112478E.2030003@yandex.ru> Hello. Used next version of dovecot: # dovecot --version 20130205 (03a0af22100d+) Quota configuration: plugin { quota = dict:Domain quota:%d:redis:host=127.0.0.1:prefix=domain/ quota_rule = *:storage=0 quota_rule2 = Trash:storage=+5%% quota2 = dict:User quota::redis:host=127.0.0.1:prefix=user/ quota2_rule = *:storage=0 quota2_rule2 = Trash:storage=+5%% } Problem: incorrect quota calculations. Dovecot gets size and messages count of all domain from last user in list. In 2.1 works fine. Steps to reproduce problem: 1. install dovecot 2. create over 50 users with e-mails 3. execute: doveadm quota recalc -A 4. execute: doveadm quota get -A 5. get count of all users message: sum in some application result of execution doveadm quota get -A | grep MESSAGE | grep 'User quota' | awk '{print $5}' (139443 messages) 6. get size of all users message: sum in some application result of execution doveadm quota get -A | grep STORAGE | grep 'User quota' | awk '{print $5}' (2304399 Kb) 7. look at some user user at example.org Domain quota STORAGE 82408 51200000 0 user at example.org Domain quota MESSAGE 1078 - 0 user at example.org User quota STORAGE 0 209715200 0 user at example.org User quota MESSAGE 0 - 0 and see 1078 messages and 82408 Kb in domain quota 8. look at last user in list z at example.org Domain quota STORAGE 82408 51200000 0 z at example.org Domain quota MESSAGE 1078 - 0 z at example.org User quota STORAGE 82408 209715200 0 z at example.org User quota MESSAGE 1078 - 0 and see matching values. From gsansone at eumus.edu.uy Wed Feb 6 15:19:56 2013 From: gsansone at eumus.edu.uy (Gustavo Sansone) Date: Wed, 06 Feb 2013 11:19:56 -0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> <1360030140.10326.54.camel@innu> <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> Message-ID: <565beb939e973e6fbf6057cafbc27daf@eumus.edu.uy> El 2013-02-05 12:10, Gustavo Sansone escribi?: > El 2013-02-05 0:09, Timo Sirainen escribi?: >> On Tue, 2013-01-29 at 16:29 -0200, Gustavo Sansone wrote: >>> hello everyone, i'm new to this list >>> im running dovecot as imap+pop (2.1.12) and roundcube (0.8.4) as >>> client. >>> >>> having problems on saving to imap folders, with errors on some >>> accounts >>> >>> my folder structure is as follow: >>> >>> /home/user/.maildir >>> /home/user/.maildir/.Drafts >>> /home/user/.maildir/.Junk >>> /home/user/.maildir/.Sent >>> /home/user/.maildir/.Trash >>> /home/user/.maildir/.Archives >>> /home/user/.maildir/.Archives.2010 >>> ... >>> >>> i'm very confused with the namespace settings. i don't understand >>> what >>> is the correct way to configure it. >>> don't need public or shared folders. >> >> Easiest to use the default namespace settings. There you wouldn't >> have >> any namespace prefix. > > if i just remove the namespace 'INBOX.' , on 10-mail.conf then there > is something that creates new Send and Draft folders into my maildir > folder like this INBOX.Send and INBOX.Drafts. and Roundcube shows me > these new folders under INBOX (screenshot attached). > can't figure out why it happens. > sorry, forgot to mention that when INBOX.Send and INBOX.Drafts are created, Roundcube subscribe to these folders as special folders. the screenshot was taken after i added INBOX. prefix again to 10-mail.conf another important thing (i guess), is when installed dovecot, i migrated from Courier following the wiki2 documentation. >>> on dovecot.log i can see this error with my account (and all the >>> others), but it seems to work ok using roundcube: >>> imap(gsansone): Error: autocreate: No namespace found for mailbox: >>> Junk >> >> In your current setup you have namespace prefix=INBOX. so the >> mailbox >> name would be INBOX.Junk. >> >>> with a few accounts, i have this on roundcube log: >>> IMAP Error: Could not save message in Drafts in >>> /var/www/localhost/htdocs/mail/program/steps/mail/sendmail.inc >>> with error on saving to imap folder in roundcube UI >> >> Roundcube also isn't configured to have INBOX. prefix. Sounds like >> you >> should just remove it. From aoster at novanetwork.de Wed Feb 6 15:26:39 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Wed, 06 Feb 2013 14:26:39 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <510F81F2.1020809@novanetwork.de> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> <510F81F2.1020809@novanetwork.de> Message-ID: <51125A0F.4040502@novanetwork.de> Am 04.02.2013 10:40, schrieb Andreas Oster: > Am 22.01.2013 12:54, schrieb Timo Sirainen: >> On 17.1.2013, at 16.58, Andreas Oster wrote: >> >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. >> >> It should work for all namespaces. >> >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota >> >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota >> >> That should work? Worked at least in latest v2.1 hg. >> >> > Hello Timo, > > can you please confirm, that it is actually possible to have multiple > quota roots with separate/different quota values. > > Currently I have this config: > > namespace { > disabled = no > hidden = no > ignore_on_failure = no > inbox = no > list = children > location = maildir:/var/vmail/archives/%n/Maildir > prefix = Archives/ > separator = / > subscriptions = yes > type = private > } > namespace inbox { > disabled = no > hidden = no > ignore_on_failure = no > inbox = yes > list = yes > location = > prefix = > separator = / > subscriptions = yes > type = private > } > plugin { > quota = maildir:User quota:ns= > quota2 = maildir:Archives quota:ns=Archives/ > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+200M > quota_rule3 = ns=Archives/:storage=5G > } > > > With this setup I get the following error: > > > Debug: Quota root: name=User quota backend=maildir args=ns= > Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 > Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 > Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete > configuration for rule 'storage=5G' should be changed to 'storage=+5G' > Debug: Quota rule: root=User quota mailbox=ns=Archives/ > bytes=+5368709120 messages=0 > Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ > > > Thank you for your kind help. > > best regards > Andreas > Hi all, my quota issue is solved. It was a stupid configuration error and it seems that I have been blind :-( plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota2_rule = *:storage=5G } this confiurationg works like a charm. Thank you for your kind help. best regards Andreas From aoster at novanetwork.de Wed Feb 6 15:26:39 2013 From: aoster at novanetwork.de (Andreas Oster) Date: Wed, 06 Feb 2013 14:26:39 +0100 Subject: [Dovecot] how to setup different quota for multiple namespaces In-Reply-To: <510F81F2.1020809@novanetwork.de> References: <50EFE501.5060103@novanetwork.de> <50F811A0.6010802@novanetwork.de> <7791E2E2-E95B-4647-B054-920CE3101920@iki.fi> <510F81F2.1020809@novanetwork.de> Message-ID: <51125A0F.4040502@novanetwork.de> Am 04.02.2013 10:40, schrieb Andreas Oster: > Am 22.01.2013 12:54, schrieb Timo Sirainen: >> On 17.1.2013, at 16.58, Andreas Oster wrote: >> >>>> I just saw on important difference in the doc and this configuration: >>>> >>>> see >>>> http://wiki2.dovecot.org/Quota/Configuration#Quota_for_public_namespaces >>>> >>>> the ns=name syntax is for _public_ namespaces only. I just tested it >>>> with this setup and every message is counted for both namespaces, if >>>> delivered into INBOX or a mailbox of the Archive namespace. >> >> It should work for all namespaces. >> >>>>> plugin { >>>>> quota = dict:User quota::proxy::quota >>>>> quota2 = dict:Archive quota:ns=Archive.:proxy::quota >> >> quota = dict:User quota::ns=:proxy::quota >> quota2 = dict:Archive quota::ns=Archive.:proxy::quota >> >> That should work? Worked at least in latest v2.1 hg. >> >> > Hello Timo, > > can you please confirm, that it is actually possible to have multiple > quota roots with separate/different quota values. > > Currently I have this config: > > namespace { > disabled = no > hidden = no > ignore_on_failure = no > inbox = no > list = children > location = maildir:/var/vmail/archives/%n/Maildir > prefix = Archives/ > separator = / > subscriptions = yes > type = private > } > namespace inbox { > disabled = no > hidden = no > ignore_on_failure = no > inbox = yes > list = yes > location = > prefix = > separator = / > subscriptions = yes > type = private > } > plugin { > quota = maildir:User quota:ns= > quota2 = maildir:Archives quota:ns=Archives/ > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+200M > quota_rule3 = ns=Archives/:storage=5G > } > > > With this setup I get the following error: > > > Debug: Quota root: name=User quota backend=maildir args=ns= > Debug: Quota rule: root=User quota mailbox=* bytes=3221225472 messages=0 > Debug: Quota rule: root=User quota mailbox=Trash bytes=+209715200 messages=0 > Warning: quota root User quota rule ns=Archives/:storage=5G: obsolete > configuration for rule 'storage=5G' should be changed to 'storage=+5G' > Debug: Quota rule: root=User quota mailbox=ns=Archives/ > bytes=+5368709120 messages=0 > Debug: Quota root: name=Archives quota backend=maildir args=ns=Archives/ > > > Thank you for your kind help. > > best regards > Andreas > Hi all, my quota issue is solved. It was a stupid configuration error and it seems that I have been blind :-( plugin { quota = maildir:User quota:ns= quota2 = maildir:Archives quota:ns=Archives/ quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota2_rule = *:storage=5G } this confiurationg works like a charm. Thank you for your kind help. best regards Andreas From marcio.merlone at a1.ind.br Wed Feb 6 15:36:18 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 11:36:18 -0200 Subject: [Dovecot] Sieve also filter outgoing messages Message-ID: <51125C52.9040007@a1.ind.br> Greetings, A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver on postfix, perhaps the sieve script could be executed when saving to the Sent folder? Regards, -- Marcio Merlone From h.reindl at thelounge.net Wed Feb 6 15:53:02 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 06 Feb 2013 14:53:02 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51125C52.9040007@a1.ind.br> References: <51125C52.9040007@a1.ind.br> Message-ID: <5112603E.90805@thelounge.net> Am 06.02.2013 14:36, schrieb Marcio Merlone: > A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver > on postfix, perhaps the sieve script could be executed when saving to the Sent folder? there are complete different things happening for incoming messages and save sent messages you do NOT want get sieve in action if you drag&drop messages between folders and do this for sent messages would trigger the same since the client acts the same way -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From rafaelvolpeti at gmail.com Wed Feb 6 16:09:26 2013 From: rafaelvolpeti at gmail.com (Rafael VOlpe TI) Date: Wed, 6 Feb 2013 12:09:26 -0200 Subject: [Dovecot] Trouble when set mail_nfs_index = yes Message-ID: Hi Buddies, dovecot-sql.conf driver = mysql connect = host=127.0.0.1 dbname=postfix user=postfix password=secret default_pass_scheme = plain password_query = SELECT username as user, password \ FROM mailbox WHERE username = '%u' AND active = 1 user_query = SELECT concat("/var/vmail/", maildir) as home, 125 as uid, 125 as gid FROM mailbox WHERE username = '%u' Version: 2.0.19: /etc/dovecot/dovecot.conf doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/conf.d/10-mail.conf:157: fsync_disable has been renamed to mail_fsync # OS: Linux 3.2.0-37-generic x86_64 Ubuntu 12.04.2 LTS auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 125 first_valid_uid = 125 last_valid_gid = 125 last_valid_uid = 125 log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/%Lh/:INDEX=/%Lh/ mail_nfs_storage = yes mmap_disable = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = Hello, I installed a few weeks ago the Dovecot package on a Debian stable along with postfix, everything works fine except the sieve filter. I followed the howto found on the internet (google: debian postfix dovecot sieve). I'm able to view and modify sieve rules with roundcube or thunderbird plugins, but the rules are not applied when a message arrives and all messages arrive in INBOX. Here is the content of my configuration files concerning sieve : dovecot.conf protocols = imap imaps managesieve protocol managesieve { listen = *:2000 } protocol managesieve { mail_executable = /usr/lib/dovecot/managesieve managesieve_max_line_length = 65536 managesieve_logout_format = bytes=%i/%o } protocol lda { postmaster_address = postmaster at babelouest.org mail_plugins = sieve mail_plugin_dir = /usr/lib/dovecot/modules/lda log_path = info_log_path = syslog_facility = mail } plugin { sieve_dir=~/Maildir/sieve } And the postfix/main.cf config file contains for dovecot sieve the following maibox_command = /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT" dovecot_destination_recipient_limit = 1 virtual_transport = dovecot I also tried the command mailbox_command = /usr/lib/dovecot/deliver it didn't work either... Is there somthing so obvious that I didn't see it ? Thanks in advance From rs at sys4.de Wed Feb 6 17:19:26 2013 From: rs at sys4.de (Robert Schetterer) Date: Wed, 06 Feb 2013 16:19:26 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <5112603E.90805@thelounge.net> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> Message-ID: <5112747E.7030800@sys4.de> Am 06.02.2013 14:53, schrieb Reindl Harald: > > > Am 06.02.2013 14:36, schrieb Marcio Merlone: >> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver >> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? > > there are complete different things happening > for incoming messages and save sent messages > > you do NOT want get sieve in action if you drag&drop > messages between folders and do this for sent messages > would trigger the same since the client acts the same way > > Sorry Harald, i dont agree, this would be a nice feature to have in fact we internally disussed this allready, it might be used for workaround imap bugs of new outlook, and for auto flagging and auto sorting mails, anyway at recent i see no way to get it real on the tec side , but that might change in the future with i.e relation to rfc4550 Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tlx at leuxner.net Wed Feb 6 17:25:05 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 6 Feb 2013 16:25:05 +0100 Subject: [Dovecot] Sieve not filtering In-Reply-To: References: Message-ID: <20130206152505.GA9635@nihlus.leuxner.net> * Nicolas Mora 2013.02.06 16:06: > plugin { > sieve_dir=~/Maildir/sieve > } Better to post 'dovecot -n' which is the active config. http://wiki.dovecot.org/LDA/Sieve/Dovecot Not sure what the default was for the active script in the old Dovecot versions as 'sieve_dir' is the upload directory for scripts to be referenced (see Wiki). Active config will tell. You may raise logging levels and it will tell you what happens when the LDA tries to spawn sieve: http://wiki.dovecot.org/Logging Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From skdovecot at smail.inf.fh-brs.de Wed Feb 6 17:45:02 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 Feb 2013 16:45:02 +0100 (CET) Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <5112747E.7030800@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 6 Feb 2013, Robert Schetterer wrote: > Am 06.02.2013 14:53, schrieb Reindl Harald: >> Am 06.02.2013 14:36, schrieb Marcio Merlone: >>> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver >>> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? >> >> there are complete different things happening >> for incoming messages and save sent messages >> >> you do NOT want get sieve in action if you drag&drop >> messages between folders and do this for sent messages >> would trigger the same since the client acts the same way > > Sorry Harald, i dont agree, this would be a nice feature to have > in fact we internally disussed this allready, it might be used for > workaround imap bugs of new outlook, and for auto flagging and auto > sorting mails, anyway at recent i see no way to get it real on the tec > side , but that might change in the future with i.e relation to rfc4550 A combination of BCC (instead of IMAP-Append to Sent) and a stable detection of "outgoing" will work. E.g. BCC: myaccount+outgoing at mydomain Then Sieve must be able to detect subaddressing or default mailbox to fileinto. First is not availble for all installations. Last is not available at all, I think. Maybe in combination with a "From" check and/or check of last recieved header to prevent misuse, when an insider (needs to know the subaddress) mails to myaccount+outgoing at mydomain from an external MTA. Using BCC instead of IMAP-Append-to-Sent also has the advantages that the message is transferred only once and that the Recieved header has the queueid for later trouble shooting. Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURJ6f13r2wJMiz2NAQImZAf+PWNSGGA5rlep9Onf2GXMpGYpvkDpGj1G DcWRykGEO9kGhQwbO/B7AIQ21bCI9PrYg2pRN9XMGpH4RF9ippsICjlOW9Eo2n5X Lt/0dd/Jdyj5UhbvdcxzI1VLfDItbl2N6mNBTClo9V/DDKjo5W89gzIaxOQoTIvz LTGj1EFn17UUtP67/uE4zUcJ7/+rc2BhLcBL2ZpJXbKT4WFvjT/THuJ2UagYTdhZ 9+nvtNyxq54iGQzja5dR8CEXbcuyVtIgY5MZXBP3PMIfGrkfcTP8Iz4dTV0caO4/ QqrSJfe/K50h7pnYr+PlnjMtKu3vbDwkr/Qk8o4I4lBYlAwZtKKyjg== =YsoM -----END PGP SIGNATURE----- From marcio.merlone at a1.ind.br Wed Feb 6 17:57:41 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 13:57:41 -0200 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <5112747E.7030800@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> Message-ID: <51127D75.8090100@a1.ind.br> Em 06-02-2013 13:19, Robert Schetterer escreveu: > Am 06.02.2013 14:53, schrieb Reindl Harald: >> Am 06.02.2013 14:36, schrieb Marcio Merlone: >>> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver >>> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? >> there are complete different things happening >> for incoming messages and save sent messages >> >> you do NOT want get sieve in action if you drag&drop >> messages between folders and do this for sent messages >> would trigger the same since the client acts the same way > Sorry Harald, i dont agree, this would be a nice feature to have > in fact we internally disussed this allready, it might be used for > workaround imap bugs of new outlook, and for auto flagging and auto > sorting mails, anyway at recent i see no way to get it real on the tec > side , but that might change in the future with i.e relation to rfc4550 My specific need in this case is that we must forward copy all mails with a specific subject and/or destination to a mail account for auditing and registry purposes. It already works for messages which has at least on inside destination (multiple exact same copies when multiple inside destination, but what the heck...), but if a message is sent with only outside destinations I get no copy. It should work similar to postfix always_bcc, but only when a criteria matches. Regards. -- Marcio Merlone From tss at iki.fi Wed Feb 6 18:23:47 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 18:23:47 +0200 Subject: [Dovecot] Trouble when set mail_nfs_index = yes In-Reply-To: References: Message-ID: <36C41FF4-BD63-4BE8-AFA6-25E410D1E11E@iki.fi> On 6.2.2013, at 16.09, Rafael VOlpe TI wrote: > I use mailbox in nfs share. > When i set the option "mail_nfs_index = yes" the dovecot fails to start. > > Looking at logs, only prompted this: > > "Feb 6 11:07:25 picapau kernel: [ 1436.376023] init: dovecot main process > (4177) terminated with status 89" without more info. You're not looking into the correct log file. Dovecot logs another error message about it. http://wiki2.dovecot.org/Logging Anyway, better not use mail_nfs_index anyway. Use http://wiki2.dovecot.org/Director instead. From p at sys4.de Wed Feb 6 18:24:45 2013 From: p at sys4.de (Patrick Ben Koetter) Date: Wed, 6 Feb 2013 17:24:45 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> Message-ID: <20130206162444.GB4645@sys4.de> * Steffen Kaiser : > On Wed, 6 Feb 2013, Robert Schetterer wrote: > > > Am 06.02.2013 14:53, schrieb Reindl Harald: > >> Am 06.02.2013 14:36, schrieb Marcio Merlone: > >>> A probably simple question and answer: can a sieve script be executed on outgoing messages? I already use deliver > >>> on postfix, perhaps the sieve script could be executed when saving to the Sent folder? Barry (Leiba) wrote . AFAIK it awaits further discussion. p at rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Wed Feb 6 18:25:59 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 18:25:59 +0200 Subject: [Dovecot] saving to imap folders and namespace settings In-Reply-To: <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> References: <2f865bdb07244ed27edeabc0a7bb50e0@eumus.edu.uy> <1360030140.10326.54.camel@innu> <0d50e5f641ab68e1206f2d08b78f9f5b@eumus.edu.uy> Message-ID: <0CAD42D4-FB5E-4BED-A5C7-345278F68CAB@iki.fi> On 5.2.2013, at 16.10, Gustavo Sansone wrote: >>> i'm very confused with the namespace settings. i don't understand what >>> is the correct way to configure it. >>> don't need public or shared folders. >> >> Easiest to use the default namespace settings. There you wouldn't have >> any namespace prefix. > > if i just remove the namespace 'INBOX.' , on 10-mail.conf then there is something that creates new Send and Draft folders into my maildir folder like this INBOX.Send and INBOX.Drafts. and Roundcube shows me these new folders under INBOX (screenshot attached). > can't figure out why it happens. Because different clients you're using have been configured to use different namespace settings. This is unfortunately how IMAP protocol still works. From Dovecot's side either set the prefix to INBOX. or to empty, and from all of the clients' sides make sure they use the matching configuration. From gfinch at ldmltd.ca Wed Feb 6 18:35:44 2013 From: gfinch at ldmltd.ca (Gregory Finch) Date: Wed, 06 Feb 2013 08:35:44 -0800 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51127D75.8090100@a1.ind.br> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> Message-ID: <51128660.4040800@ldmltd.ca> On 2013-02-06 7:57 AM, Marcio Merlone wrote: > Em 06-02-2013 13:19, Robert Schetterer escreveu: >> Am 06.02.2013 14:53, schrieb Reindl Harald: >>> Am 06.02.2013 14:36, schrieb Marcio Merlone: >>>> A probably simple question and answer: can a sieve script be >>>> executed on outgoing messages? I already use deliver >>>> on postfix, perhaps the sieve script could be executed when saving >>>> to the Sent folder? >>> there are complete different things happening >>> for incoming messages and save sent messages >>> >>> you do NOT want get sieve in action if you drag&drop >>> messages between folders and do this for sent messages >>> would trigger the same since the client acts the same way >> Sorry Harald, i dont agree, this would be a nice feature to have >> in fact we internally disussed this allready, it might be used for >> workaround imap bugs of new outlook, and for auto flagging and auto >> sorting mails, anyway at recent i see no way to get it real on the tec >> side , but that might change in the future with i.e relation to rfc4550 > My specific need in this case is that we must forward copy all mails > with a specific subject and/or destination to a mail account for > auditing and registry purposes. It already works for messages which > has at least on inside destination (multiple exact same copies when > multiple inside destination, but what the heck...), but if a message > is sent with only outside destinations I get no copy. It should work > similar to postfix always_bcc, but only when a criteria matches. > > Regards. > Would sender_bcc_maps or recipient_bcc_maps be able to work on the postfix side? Otherwise I would suggest BCC like Steffen did and filter out what you want with sieve. Heck, with the *_bcc_maps on the postfix side you can still filter what you want with sieve. -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Wed Feb 6 18:42:39 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 18:42:39 +0200 Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On 6.2.2013, at 6.51, Joseph Tam wrote: > On a related topic, what's the easiest way to alias various common > mailbox names to one physical mailbox? For example, mapping "Trash", > "Deleted Messages", "Junk" to the same mailbox? > > Would you use the SPECIAL-USE, or is there a better way to do this? > Namescape configuration? Virtual plugin? Ideally SPECIAL-USE will take care of it in future. For now you should configure it, but also realize that it won't work for all clients. There's also mailbox_alias plugin in v2.1.10+: http://wiki2.dovecot.org/Plugins/MailboxAlias (the link will work in a few minutes). From stephan at rename-it.nl Wed Feb 6 19:01:42 2013 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 06 Feb 2013 18:01:42 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <20130206162444.GB4645@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <20130206162444.GB4645@sys4.de> Message-ID: <51128C76.4030501@rename-it.nl> On 2/6/2013 5:24 PM, Patrick Ben Koetter wrote: > Barry (Leiba) wrote . AFAIK it > awaits further discussion. p at rick This is on my TODO list, but Dovecot will need to gain IMAP METADATA support first. Regards, Stephan. From marcio.merlone at a1.ind.br Wed Feb 6 19:12:34 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 15:12:34 -0200 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51128660.4040800@ldmltd.ca> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> <51128660.4040800@ldmltd.ca> Message-ID: <51128F02.5090102@a1.ind.br> Em 06-02-2013 14:35, Gregory Finch escreveu: > Would sender_bcc_maps or recipient_bcc_maps be able to work on the > postfix side? Otherwise I would suggest BCC like Steffen did and > filter out what you want with sieve. Heck, with the *_bcc_maps on the > postfix side you can still filter what you want with sieve. I've been poking around postfix now, a subject_bcc_maps with a pcre map would do :), and more realistic, a pcre header_checks with a milter may be the way to go, must read about milter first. Steffen cited "outgoing detection" - I only asked about outgoing because incoming is already done with sieve, outgoing is the missing part. A single solution for both is the ideal, so you can forget about outgoing detection. IMAP-append-to-send also were a blind shot from me, forget it also. In sum, when a subject matches a pcre, bcc it to another account, no matter what direction. Regarding Steffen suggestion, I am unable to figure what he meant. :P -- Marcio Merlone From tss at iki.fi Wed Feb 6 19:34:16 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 19:34:16 +0200 Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On 5.2.2013, at 21.52, Radek Novotn? wrote: > let me ask a question, please. Is it possible in dovecot to set up per user special-use folder names? > > Imagine situation with two users where first prefere another language that second. > > mailbox Sent { > special_use = \Sent > } > > for english speaking users and > > mailbox "Odeslan? po?ta" { > special_use = \Sent > } > > for czech speaking users. Like Patrick mentioned this isn't how it should be used.. But it would be possible anyway by returning the proper settings from userdb lookup. Easier probably set both of these in the namespace, then only configure which one will be autocreated (only if both mailboxes are created the client sees two \Sent flags). The autocreation can be made per-user by returning namespace/inbox/Sent/auto=create or namespace/inbox/Odeslan? po?ta/auto=create field from userdb. From rs at sys4.de Wed Feb 6 19:40:56 2013 From: rs at sys4.de (Robert Schetterer) Date: Wed, 06 Feb 2013 18:40:56 +0100 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <51127D75.8090100@a1.ind.br> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> Message-ID: <511295A8.8020302@sys4.de> Am 06.02.2013 16:57, schrieb Marcio Merlone: > It should work similar to postfix always_bcc, but only when a criteria > matches. incomming mails can be filtered through sieve, always_bcc etc and + adressing maybe be a way to solve this, i tested somthing like this for mail archiving, however the filter rules may get complex Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From marcio.merlone at a1.ind.br Wed Feb 6 19:46:20 2013 From: marcio.merlone at a1.ind.br (Marcio Merlone) Date: Wed, 06 Feb 2013 15:46:20 -0200 Subject: [Dovecot] Sieve also filter outgoing messages In-Reply-To: <511295A8.8020302@sys4.de> References: <51125C52.9040007@a1.ind.br> <5112603E.90805@thelounge.net> <5112747E.7030800@sys4.de> <51127D75.8090100@a1.ind.br> <511295A8.8020302@sys4.de> Message-ID: <511296EC.4070201@a1.ind.br> Em 06-02-2013 15:40, Robert Schetterer escreveu: > Am 06.02.2013 16:57, schrieb Marcio Merlone: >> It should work similar to postfix always_bcc, but only when a criteria >> matches. > incomming mails can be filtered through sieve, always_bcc etc and + > adressing maybe be a way to solve this, i tested somthing like this > for mail archiving, however the filter rules may get complex Incoming is done with sieve, the problem comes to outgoing without any rcpt to inside, in which case it becomes incoming too. It must happen only on a specific subject.... -- Marcio Merlone From tss at iki.fi Wed Feb 6 20:30:05 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 6 Feb 2013 20:30:05 +0200 Subject: [Dovecot] Subfolders problem In-Reply-To: References: <1360029796.10326.49.camel@innu> Message-ID: On 5.2.2013, at 10.00, Adam Maciejewski wrote: > # 2.1.10: /etc/dovecot/dovecot.conf v2.1.14 has some bugfix related to this.. Also I couldn't reproduce it anymore, so probably fixed. From jtam.home at gmail.com Thu Feb 7 03:59:09 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 6 Feb 2013 17:59:09 -0800 (PST) Subject: [Dovecot] Per user special-use folder names In-Reply-To: References: Message-ID: On Wed, 6 Feb 2013, Timo Sirainen wrote: >> On a related topic, what's the easiest way to alias various common >> mailbox names to one physical mailbox? For example, mapping "Trash", >> "Deleted Messages", "Junk" to the same mailbox? >> >> Would you use the SPECIAL-USE, or is there a better way to do this? >> Namescape configuration? Virtual plugin? > > Ideally SPECIAL-USE will take care of it in future. For now you should > configure it, but also realize that it won't work for all clients. > There's also mailbox_alias plugin in v2.1.10+: > http://wiki2.dovecot.org/Plugins/MailboxAlias (the link will work in a > few minutes). Perfect. I added a link to this page to the parent Plugins page Joseph Tam From f.biermann at eye-on.de Thu Feb 7 12:04:33 2013 From: f.biermann at eye-on.de (Frederic Biermann) Date: Thu, 7 Feb 2013 11:04:33 +0100 Subject: [Dovecot] fts_solr search in subfolders? Message-ID: Hello, i am using dovecot 2.1.x with Solr 4.x and it is all working fine. Search in current selected folder is extremly fast. But i want to ask if it is possible that i have selected inbox, setup a search and get results from subfolders. This is not working for me. I have to select the subfolder and then i get results. Is there any solution to fix this? best regards lexusburn From raabe at froglogic.com Thu Feb 7 12:22:24 2013 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 07 Feb 2013 11:22:24 +0100 Subject: [Dovecot] File permissions used for automatically created mailbox Message-ID: <51138060.5040404@froglogic.com> Hi, I'm running Dovecot 2.1.12 on FreeBSD (quite successfully so, thanks for this nice piece of software!). One thing which is slightly annoying though is that automatically created mailboxes (I have lda_mailbox_autocreate set) don't have the file permissions I'd like them to have. I'm using a vmail-based system, i.e. all mail is owned by vmail:vmail; another member of the vmail group is called 'backup', which has read access to all mail in order to create backups. All mail is stored beneath /home/vmail, e.g. /home/vmail/frerich/Maildir Right now, newly created mailbox directories (e.g. /home/vmail/bob) have 0700 permissions, but I'd like to have 0750 for all directories and 0640 for all files so that all files and directories are group-readable for backup purposes. Does anybody know whether this is configurable somehow? Right now, my workaround consists running this cron script every night: #!/bin/sh chown -R vmail:vmail /home/vmail/ find /home/vmail/ -type d -print0 | xargs -0 chmod 0750 find /home/vmail/ -type f -print0 | xargs -0 chmod 0640 -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From kavish.karkera at yahoo.com Thu Feb 7 13:19:31 2013 From: kavish.karkera at yahoo.com (Kavish Karkera) Date: Thu, 7 Feb 2013 19:19:31 +0800 (SGT) Subject: [Dovecot] fts_solr search in subfolders? In-Reply-To: References: Message-ID: <1360235971.78313.YahooMailNeo@web193505.mail.sg3.yahoo.com> Hi lexusburn, We are too trying to setting up the same scenario, would like know what configuration / settings you have used in apache solr, Help me? :) Regards, Kavish Karkera ________________________________ From: Frederic Biermann To: dovecot at dovecot.org Sent: Thursday, 7 February 2013 3:34 PM Subject: [Dovecot] fts_solr search in subfolders? Hello, i am using dovecot 2.1.x with Solr 4.x and it is all working fine. Search in current selected folder is extremly fast. But i want to ask if it is possible that i have selected inbox, setup a search and get results from subfolders. This is not working for me. I have to select the subfolder and then i get results. Is there any solution to fix this? best regards lexusburn From skdovecot at smail.inf.fh-brs.de Thu Feb 7 13:34:32 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 7 Feb 2013 12:34:32 +0100 (CET) Subject: [Dovecot] File permissions used for automatically created mailbox In-Reply-To: <51138060.5040404@froglogic.com> References: <51138060.5040404@froglogic.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 7 Feb 2013, Frerich Raabe wrote: > I'm running Dovecot 2.1.12 on FreeBSD (quite successfully so, thanks for this > nice piece of software!). One thing which is slightly annoying though is that > automatically created mailboxes (I have lda_mailbox_autocreate set) don't > have the file permissions I'd like them to have. > > Right now, newly created mailbox directories (e.g. /home/vmail/bob) have 0700 > permissions, but I'd like to have 0750 for all directories and 0640 for all > files so that all files and directories are group-readable for backup > purposes. > > Does anybody know whether this is configurable somehow? As far as I know and in my 2.1.14 it is like that: + the .XYZ directory inherits its permission to any new .XYZ.ABC mailbox (second level and down). + the Maildir/ directory inherits its permission to any new .XYZ mailbox (top level). + Files get the permission of the directory minus "x". This is true for "CREATE" command and Dovecot deliver. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURORSF3r2wJMiz2NAQKCKwf+Oz98Fwub9G+QKwEJWIY5O5EtdFJlEIlH rzILIgsM9TRwuzdIFC+ZvAwY8EjOBd+EEWAnp2Ie3PEnP0mVloWUGFh6jyo1CYM3 lvROgjqXpo3++6+6KvHVCHUP21hK9nfJrYiGf5ESj0PMTlKPS4kBgdOhGnMQayM3 3Tn8RcpngzG0IG/ZNfW780uSAfmK0e7YOek5BumsRtt5J7cpozL2X6ce91ymOi4A fzyKTit+KLpwiglUj+68ZB39eX1SmHqrv8UeEXmxKCCjxoxHnk+pq/30MJd3a7lF xOLW8yJx6qFUbs0J3TymN+rSKTDkkaaNU3QM9rsQjy2W5XWYF+5FKg== =ONer -----END PGP SIGNATURE----- From mark at msapiro.net Thu Feb 7 21:06:28 2013 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 7 Feb 2013 11:06:28 -0800 Subject: [Dovecot] Dovecot 2.2.beta1 Message-ID: Just an FYI, I have been running 2.2.beta1 for a few days now with no problems, but I'm not using any of the new features ;) -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From kharford at constantcontact.com Fri Feb 8 00:12:39 2013 From: kharford at constantcontact.com (Harford, Ken) Date: Thu, 7 Feb 2013 22:12:39 +0000 Subject: [Dovecot] user uknown Message-ID: Hi, I upgraded to version 2.1.1 from version 1.0.7 and I am having issues with postfix and dovecot reading my userdb file. Here is my dovecot config: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.1.1.el5 x86_64 CentOS release 5.7 (Final) auth_debug = yes auth_socket_path = /var/run/dovecot/auth-userdb default_internal_user = vmail default_login_user = dovecot disable_plaintext_auth = no auth_mechanisms = plain log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir mbox_write_locks = fcntl namespace inbox { inbox = yes location = prefix = } passdb { args = username_format=%u /etc/dovecot.ctctmon/%d/userdb driver = passwd-file } postmaster_address = postmaster at example.com protocols = imap lmtp service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap { process_limit = 1024 } ssl = required ssl_cert = service=lda Feb 07 17:05:27 auth: Debug: passwd-file(ctctsc9 at p2-ctctmon.constantcontact.com): lookup: user=ctctsc9 at p2-ctctmon. file=/etc/dovecot.ctctmon/p2-ctctmon./userdb Feb 07 17:05:27 auth: Info: passwd-file(ctctsc9 at p2-ctctmon.): unknown user Feb 07 17:05:27 lda: Debug: auth input: Feb 07 17:05:27 auth: Debug: master out: NOTFOUND 1 Postfix: Feb 7 17:07:51 p2-ctctmon101 postfix/pipe[17969]: C91831C8168: to=, relay=dovecot, delay=0.1, delays=0.09/0/0/0.02, dsn=5.1.1, status=bounced (user unknown) Userdb: ctctsc1:$1$e3k3Xp.u$QdSaJMh79dhivHUuaToDO0:::ctctsc1 Any ideas would be greatly appreciated!! Ken From user+dovecot at localhost.localdomain.org Fri Feb 8 01:26:19 2013 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Fri, 08 Feb 2013 00:26:19 +0100 Subject: [Dovecot] dsync: Invalid server handshake In-Reply-To: <36C41C6B-F529-4787-A12E-C55975B44629@ct.heise.de> References: <36C41C6B-F529-4787-A12E-C55975B44629@ct.heise.de> Message-ID: <5114381B.5060209@localhost.localdomain.org> On 02/05/2013 07:36 PM Dusan Zivadinovic wrote: > ? > and I get this error: > > dsync-local(dz): Error: Invalid server handshake: dsync-server 2 > dsync-remote(dz): Error: Invalid client handshake: dsync-client 1 > > # lokal machine: dovecot 2.0.19apple1 running on Mac OS X 10.8.2 Server > using standard Apple-Dovecot-Configuration > > # remote machine: 2.1.12 running on Mac OS X 10.4 using PAM authentication > > > I cant seem to find any documentation to this error, nor do I find any dsync entries > in the logs of both machines. Does anyone have a hint? You are using different minor-versions of Dovecot; v2.0 v/s v2.1. Dovecot v2.0 uses dsync 'version 1', and v2.1 uses dsync 'version 2'. If there aren't to much big mailboxes, I would stop imap/pop3 on the old server, move the mailboxes to the new server and start iamp/pop3 on the new server. Or you have to update Dovecot to version 2.1 on the older server. Then you should be able to dsync between both servers. Regards, Pascal -- The trapper recommends today: c01dcafe.1303900 at localdomain.org From tlx at leuxner.net Fri Feb 8 09:25:33 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 8 Feb 2013 08:25:33 +0100 Subject: [Dovecot] user uknown In-Reply-To: References: Message-ID: <20130208072533.GA31594@nihlus.leuxner.net> * Harford, Ken 2013.02.07 23:12: > passdb { > args = username_format=%u /etc/dovecot.ctctmon/%d/userdb > driver = passwd-file http://wiki2.dovecot.org/AuthDatabase/PasswdFile %u expects user at domain in the file > Userdb: > ctctsc1:$1$e3k3Xp.u$QdSaJMh79dhivHUuaToDO0:::ctctsc1 Add the domain component to ctctsc1 in this file or set username_format=%n. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From amateo at um.es Fri Feb 8 09:29:09 2013 From: amateo at um.es (Angel L. Mateo) Date: Fri, 08 Feb 2013 08:29:09 +0100 Subject: [Dovecot] Disabling auth caching just from one passdb Message-ID: <5114A945.4090700@um.es> Hello, I'm using various password databases (one of them is a ldap database, which is the used for users connecting with email clients, and the other with pam, using a pam module integrated with our SSO that is used from our webmail). I would like to disable password caching in my pam database (because 'passwords' provided by our SSO are single use tokens, so caching them has no benefit), but not in the ldap. As far as I know, auth_cache_size is the option to enable caching in all databases. But, is there any option I could use to disable it in a specific password database? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From jtam.home at gmail.com Fri Feb 8 09:59:43 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 7 Feb 2013 23:59:43 -0800 (PST) Subject: [Dovecot] Imap process crash: assertion failed: (full_fs_access) In-Reply-To: References: Message-ID: Any guess at what would cause this? Feb 7 21:20:53 server dovecot: imap(user): Panic: file mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion failed: (full_fs_access) (Sorry, no core dump) According to my logs, this user couldn't start an IMAP session for a 2 hour stretch for hundreds of connections. Then the problem seemed to have went away. I logged back as that user using a master password, but couldn't recreate this problem. The backtrace is just hex numbers (stripped executables) and I didn't drop privileges, so it refused to drop core, so that's no help. Joseph Tam From peter.rindfuss at wzb.eu Fri Feb 8 11:24:05 2013 From: peter.rindfuss at wzb.eu (Peter Rindfuss) Date: Fri, 08 Feb 2013 10:24:05 +0100 Subject: [Dovecot] sieve /editheader / addheader :last crash Message-ID: <5114C435.4040105@wzb.eu> Hi Everybody, I'm using dovecot 2.1.14 from http://xi.rename-it.nl/debian/ with debian squeeze stable. In general, sieve works fine for me, but "addheader :last ..." from the editheader extension causes a crash when it processes a message. I was able to narrow this down a bit: It only happens if "addheader :last" is the first editheader command executed in a script. If it is preceded by any "deleteheader" or "addheader" without ":last", everything is fine. This is what I get from sieve-test when it crashes: sieve-test(root): Panic: file istream.c: line 133 (i_stream_read): assertion failed: (_stream->skip != _stream->pos) sieve-test(root): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4881a) [0x7f5e0b37681a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f5e0b376902] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f5e0b349f7f] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x17c) [0x7f5e0b37da9c] -> /usr/lib/dovecot/libdovecot.so.0(+0x51453) [0x7f5e0b37f453] -> /usr/lib/dovecot/libdovecot.so.0(+0x5150e) [0x7f5e0b37f50e] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7f5e0b37d989] -> /usr/lib/dovecot/libdovecot.so.0(+0x53fe1) [0x7f5e0b381fe1] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7f5e0b37d989] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x52) [0x7f5e0b37dc72] -> /usr/lib/dovecot/libdovecot.so.0(io_stream_copy+0x41) [0x7f5e0b38ae21] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e1de) [0x7f5e0b38c1de] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_send_istream+0x33) [0x7f5e0b38acb3] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_continue+0x38) [0x7f5e0be0caf8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xb8) [0x7f5e0be376d8] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x56) [0x7f5e0be086c6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x4a) [0x7f5e0be3ce9a] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x3fa66) [0x7f5e0c103a66] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x366b4) [0x7f5e0c0fa6b4] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x3ff) [0x7f5e0c0fabef] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x56) [0x7f5e0c10b456] -> sieve-test(main+0x73e) [0x403a5e] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f5e0abdec8d] -> sieve-test() [0x403219] Aborted Best regards Peter From paolo.miotto at uniud.it Fri Feb 8 14:17:40 2013 From: paolo.miotto at uniud.it (Paolo Miotto) Date: Fri, 08 Feb 2013 13:17:40 +0100 Subject: [Dovecot] Moving index files on another disk: no troubles but need more assurances Message-ID: <20130208131740.39334u183vmu13ac@webmail.uniud.it> Hi, I'm using dovecot 1.2.15 on debian squeeze with maildir, and I want to move index files out of the mailboxes disk to reduce disk I/O. I need to get the assurance that the switch is completely transparent to my clients (IMAP and POP). I read the wiki documentation and have found this thread (http://dovecot.org/pipermail/dovecot/2009-September/042665.html), which makes me confident, even if it speaks of a older version and only pop3. So I set up a test server, copied a true mailbox and changed mail location from: mail_location = maildir:~/Maildir to mail_location = maildir:~/Maildir:INDEX=/srv/indexes/%1n/%n (all users under the same domain, no need to use %d). On the IMAP side all seems to work fine, new indexes are created in the new location (dovecot.index.log on 1st access, dovecot.index.cache on message manipolation). A test client (evolution) doesn't notice at all, and I have verified via a telnet imap session that UIDs don't change. But I see that when I move the indexes the IMAP HIGHESTMODSEQ response changes and is reset to 1. Can this baffle the clients? On the rfc I don't found reference to this status walking backwards. Need I to verify that all messages have the virtual size (W=nnnn) in the file name (the mailboxes are from a previous cyrus installation, then we switch to dovecot 1.2)? What other test you suggest that checking UIDL on the pop3 side? What happens if the indexes disk disappears (broken or removed from bay or all paths down for FC/iscsi)? Can dovecot 1.2 continue with INDEX=MEMORY as for a disk full? Sorry for the many questions.... Paolo ---------------------------------------------------------------------- SEMEL (SErvizio di Messaging ELettronico) - AINF, Universita' di Udine From tss at iki.fi Fri Feb 8 15:08:50 2013 From: tss at iki.fi (Timo Sirainen) Date: Fri, 08 Feb 2013 15:08:50 +0200 Subject: [Dovecot] v2.1.15 released Message-ID: <1360328930.3230.12.camel@hurina> http://dovecot.org/releases/2.1/dovecot-2.1.15.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.15.tar.gz.sig v2.1.14's dovecot.index.cache fixes caused Dovecot to use more disk I/O and memory than was necessary. From tss at iki.fi Fri Feb 8 15:30:21 2013 From: tss at iki.fi (Timo Sirainen) Date: Fri, 08 Feb 2013 15:30:21 +0200 Subject: [Dovecot] test Message-ID: <1360330221.3230.13.camel@hurina> mailman archive process seems to have crashed and hasn't been writing to archives. lets see if it works again after restart.. From AxelLuttgens at swing.be Fri Feb 8 16:39:38 2013 From: AxelLuttgens at swing.be (Axel Luttgens) Date: Fri, 8 Feb 2013 15:39:38 +0100 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever Message-ID: Hello, I'm currently experimenting with the 2.1.x series and am facing a behavior I can't explain; I must be doing something wrong and probably miss the obvious... ;-) This is a very basic build on Mac OS X 10.8.2: ./configure --prefix=/_COD/dovecot-2.1.14 --sysconfdir=/_ETC --localstatedir=/_VAR --with-sqlite --with-ldap=yes (Yes, I know, a strange layout, yet very convenient for testing various things here.) Dovecot's configuration is very basic too: # 2.1.14: /_ETC/dovecot/dovecot.conf # OS: Darwin 12.2.1 x86_64 auth_verbose = yes disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mboxes:INBOX=~/mboxes/inbox passdb { args = /_ETC/dovecot/db.conf driver = sql } ssl = no userdb { driver = prefetch } It is started from launchd with the -F flag. Two users are defined in the database, user1 (100002/100002) and user2 (100003/100003), with "virtual" uids/gids. Connecting for the first time as user1 through imap: $ telnet 127.0.0.1 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login user1 pass1 a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in b logout * BYE Logging out b OK Logout completed. Connection closed by foreign host. In mail.log: dovecot[2997]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=3006, secured, session= dovecot[2997]: imap(user1): Debug: Effective uid=100002, gid=100002, home=/_Data/Mailstores/100002 dovecot[2997]: imap(user1): Debug: Home dir not found: /_Data/Mailstores/100002 dovecot[2997]: imap(user1): Debug: fs: root=/_Data/Mailstores/100002/mboxes, index=, control=, inbox=/_Data/Mailstores/100002/mboxes/inbox, alt= dovecot[2997]: imap(user1): Debug: Namespace : /_Data/Mailstores/100002/mboxes doesn't exist yet, using default permissions dovecot[2997]: imap(user1): Debug: Namespace : Using permissions from /_Data/Mailstores/100002/mboxes: mode=0700 gid=-1 dovecot[2997]: imap(user1): Disconnected: Logged out in=8 out=362 While logged in (ie after the login and before the logout), "doveadm who" shows: username # proto (pids) (ips) user1 1 imap (3093) (127.0.0.1) So far, so good. Now, let's connect as user2 for the first time, through pop: $ telnet 127.0.0.1 110 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user user2 +OK pass pass2 and the session hangs forever. This is written in the log: dovecot[2997]: pop3-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=3113, secured, session= dovecot[2997]: pop3(user2): Debug: Effective uid=100003, gid=100003, home=/_Data/Mailstores/100003 dovecot[2997]: pop3(user2): Debug: Home dir not found: /_Data/Mailstores/100003 dovecot[2997]: pop3(user2): Debug: fs: root=/_Data/Mailstores/100003/mboxes, index=, control=, inbox=/_Data/Mailstores/100003/mboxes/inbox, alt= dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes doesn't exist yet, using default permissions dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes/inbox doesn't exist yet, using default permissions dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes/inbox doesn't exist yet, using default permissions dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 while "doveadm who" pretends nobody is connected: username # proto (pids) (ips) To get rid of the pop3 process, I have to send it a KILL signal: no other way. But this is noticed by the master process: dovecot[2997]: pop3(user2): Fatal: master: service(pop3): child 3113 killed with signal 9 Note that it is possible to connect afterwards as user2 through imap. On the other hand, trying to connect as user1 or user2 through pop always hangs. TIA, Axel From AxelLuttgens at swing.be Fri Feb 8 17:14:39 2013 From: AxelLuttgens at swing.be (Axel Luttgens) Date: Fri, 8 Feb 2013 16:14:39 +0100 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - a follow-up Message-ID: <9571D194-7413-4A32-98B6-E65607F2AADA@swing.be> Suspecting a problem with the "virtual" uids/gids, I've tried with a real user for the mail user: # 2.1.14: /_ETC/dovecot/dovecot.conf # OS: Darwin 12.2.1 x86_64 auth_verbose = yes disable_plaintext_auth = no mail_debug = yes mail_gid = dovemailer mail_location = mbox:~/mboxes:INBOX=~/mboxes/inbox mail_uid = dovemailer passdb { args = /_ETC/dovecot/db.conf driver = sql } ssl = no userdb { driver = prefetch } Of course, the db queries do not return uid/gid pairs anymore. And the pop3 processes now seem to work as expected; connecting for the first time as user1: $ telnet 127.0.0.1 110 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user user1 +OK pass pass1 +OK Logged in. list +OK 0 messages: . quit +OK Logging out. Connection closed by foreign host. Hmmm... Axel From micah at riseup.net Fri Feb 8 20:58:38 2013 From: micah at riseup.net (micah anderson) Date: Fri, 08 Feb 2013 13:58:38 -0500 Subject: [Dovecot] dsync timeout? In-Reply-To: <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> References: <87libacola.fsf@minnow.riseup.net> <3B764B67-F536-4BB3-BF2A-4777F4B6B3F0@iki.fi> <87halyck05.fsf@minnow.riseup.net> <85A0B3F3-2931-42D9-AB83-816CFB88B054@moltingpenguin.com> Message-ID: <87d2watyxt.fsf@minnow.riseup.net> Sean Kamath writes: > On Jan 30, 2013, at 3:46 PM, micah anderson wrote: >> Seems that only the above process was still around and no other dsync >> processes. I have three machines that all have this happening it seems. >> >> I wonder if there is a ssh configuration option I could set to make >> these die off. > > If the ssh process isn't sending anything, and just waiting for read()s, and keepalives are turned off, the SSH session might never know the remote side is long gone. . . This time I managed to capture a process that was stuck and look at it from the server side, and the client side: on the server: 2000 19470 0.0 0.0 7512 3816 ? Ss Feb05 0:01 /usr/bin/dsync dsync-server -E -u foo # strace -s 1024 -F -p 19470 Process 19470 attached - interrupt to quit write(2, "dsync-remote(foo): Error: mdbox /srv/maildirbackups/foo/daily.1/storage: Duplicate GUID 96860517f68aa94f8b51000097f19f0b in m.41:682501 and m.37:653225\n", 167 on the client: root 19001 0.0 0.0 41308 1600 ? S Feb05 0:00 ssh -i /root/.ssh/backmaildir_id_rsa backmaildir at hoopoe-pn /usr/bin/dsync -u foo server # strace -s 1024 -F -p 19001 Process 19001 attached - interrupt to quit select(8, [4], [], NULL, NULL interestingly, now that I've been watching this more, the same users keep getting wedged. When I attempt to do a dsync of that user by hand, I get this: dsync-local(foo): Error: Unexpected reply from server: 13 d2a100118c45d24f760f000097f19f0b 3561 128 \Recent 1353980259 I tried one of the other users that was stuck, and it gave me: dsync-remote(bar): Error: Corrupted dbox file /srv/maildirbackups/bar/daily.1/storage/m.130 (around offset=22532): msg header has bad magic value This looks like there is something corrupted with the dbox for the user on the client side, is there something I can do to repair those? > If any data were transmitted, it would discover the remote side is turned off. One thing I am doing is using a ssh controlmaster socket, and if I kill the process on the client's side, the server side process also dies. micah From stephan at rename-it.nl Sat Feb 9 10:27:39 2013 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 09 Feb 2013 09:27:39 +0100 Subject: [Dovecot] sieve /editheader / addheader :last crash In-Reply-To: <5114C435.4040105@wzb.eu> References: <5114C435.4040105@wzb.eu> Message-ID: <5116087B.2070203@rename-it.nl> Op 2/8/2013 10:24 AM, Peter Rindfuss schreef: > Hi Everybody, > > I'm using dovecot 2.1.14 from http://xi.rename-it.nl/debian/ with > debian squeeze stable. > > In general, sieve works fine for me, but "addheader :last ..." from > the editheader extension causes a crash when it processes a message. > > I was able to narrow this down a bit: > It only happens if "addheader :last" is the first editheader command > executed in a script. If it is preceded by any "deleteheader" or > "addheader" without ":last", everything is fine. > > This is what I get from sieve-test when it crashes: > > sieve-test(root): Panic: file istream.c: line 133 (i_stream_read): > assertion failed: (_stream->skip != _stream->pos) I'm unable to reproduce this so far. Could you show me more details of your setup? * Output from: dovecot -n * Your exact test message * Your exact test Sieve script * The sieve-test command line Regards, Stephan. From julian.rath at gmail.com Sat Feb 9 06:01:38 2013 From: julian.rath at gmail.com (Julian David Rath) Date: Sat, 9 Feb 2013 05:01:38 +0100 Subject: [Dovecot] GSS-SPNEGO with dovecot and Outlook without Samba Message-ID: I am trying to configure a dovecot2 IMAP server to inter-operate with a active directory to authenticate users. The users should be able to login without a password on a domain-joined client(Outlook). Is it possible to do this only with kerberos? I don't want to put a crappy winbind on my mailserver... I already configured my server to authenticate via kerberos(GSSAPI), but Outlook does not even try to authenticate using GSSAPI, it rather uses PLAIN. I would appreciate any help, many thanks in advance. From peter.rindfuss at wzb.eu Sat Feb 9 21:20:42 2013 From: peter.rindfuss at wzb.eu (Peter Rindfuss) Date: Sat, 09 Feb 2013 20:20:42 +0100 Subject: [Dovecot] sieve /editheader / addheader :last crash In-Reply-To: <5116087B.2070203@rename-it.nl> References: <5114C435.4040105@wzb.eu> <5116087B.2070203@rename-it.nl> Message-ID: <5116A18A.2040103@wzb.eu> Am 2013-02-09 09:27, schrieb Stephan Bosch: > > I'm unable to reproduce this so far. Could you show me more details of > your setup? > > * Output from: dovecot -n > * Your exact test message > * Your exact test Sieve script > * The sieve-test command line > Hi Stephan, attached are "dovecot -n", a test message, and three very similar scripts one of which causes the crash and two not. I'm inclined to say that the message itself does not matter. This ist the sieve-test cmd line: sieve-test -t - -a 'brot at wzb.eu' -l /wzb/mail/brot/Maildir -e -x +editheader /wzb/mail/brot/sieve/{bad|good1|good2}.sieve mail Initially I encountered the problem in real deliveries, not in sieve-test. It is always reproducible. Best, Peter -------------- next part -------------- # # 2013-02-08 PR # # causes crash # require ["editheader"]; addheader :last "X-Someheader" "Crash"; deleteheader "X-Otherheader"; -------------- next part -------------- # 2.1.14 (443ff272317f): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_mechanisms = plain login auth_worker_max_count = 60 default_client_limit = 2000 default_process_limit = 200 disable_plaintext_auth = no first_valid_gid = 8 first_valid_uid = 8 hostname = imap.wzb.eu last_valid_gid = 8 last_valid_uid = 8 listen = * log_path = /var/log/dovecot/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_gid = mail mail_location = maildir:~/Maildir mail_privileged_group = mail mail_uid = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = squat fts_squat = partial=4 full=10 recipient_delimiter = + sieve = ~/.dovecot.sieve sieve_after = /var/lib/dovecot/sieve/after.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +editheader } postmaster_address = root at calypso.wzb.eu protocols = " imap lmtp sieve pop3" service auth { unix_listener auth-userdb { group = mail mode = 0600 user = mail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 1024 process_min_avail = 10 } service imap { process_limit = 8192 } service lmtp { inet_listener lmtp { address = 193.174.6.5 127.0.0.1 port = 24 } process_min_avail = 10 user = mail } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 2048 } service pop3-login { inet_listener pop3 { port = 10110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { process_limit = 128 } ssl_cert = Return-Path: Received: from athene.wzb.eu ([193.174.6.3]) by imap.wzb.eu (Dovecot) with LMTP id RnDFD4CYElHhQgAAHSXelQ for ; Wed, 06 Feb 2013 18:53:04 +0100 Received: from mout.web.de ([212.227.15.3]) by athene.wzb.eu with esmtp (Exim 4.75) (envelope-from ) id 1U39B4-0001f9-TT for brot at wzb.eu; Wed, 06 Feb 2013 18:53:04 +0100 Received: from 3capp-webde-bs15.server.lan ([172.19.170.15]) by mriweb.server.lan (mriweb002) with ESMTPA (Nemesis) id 0MLkbN-1U2sKP38b5-000Y67 for ; Wed, 06 Feb 2013 18:52:58 +0100 Received: from [193.174.6.50] by 3capp-webde-bs15.server.lan with HTTP; Wed Feb 06 18:52:58 CET 2013 MIME-Version: 1.0 Message-ID: From: =?UTF-8?Q?=22Peter_Rindfu=C3=9F=22?= To: brot at wzb.eu Subject: +++ sieve addheader :last +++ Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Date: Wed, 6 Feb 2013 18:52:58 +0100 (CET) Sensitivity: Normal X-Provags-ID: V03:K0:6lCMlXLjuBgCnXkcBwJTgQbIiiRb5Bd0a1AQQ7a/uQK sCDK/KfOHBK7WQnRs8SlTj42min3T6DI7FxgagEA0O/CWMdINx pVZopA2fc2ARvdVtcP31abI/v2eKP+iKOREELDoblEkhnjIqJ0 iuQJFBB256fg3hhrAe+mNhbbk5wTxlm2cX9DNqrNdwqSG62Xr5 XZiQr8dWzAqdxEqbuLhf0wekHpkA4nnmZpKUIipkXhyt6GwYqa CBgZ+dIGF4JTVxaSoexgptMyl3fmmsmBD32J6Oj6+Nt57j548p cB5KEaCWmJlgWQHKcNuQUMGFVe3 X-WZB-Sender: peter.rindfuss at web.de X-WZB-Virus-Scanned: by Clam AntiVirus and F-Secure AntiVirus at athene.wzb.eu X-WZB-Spam-Score: 7 X-WZB-Spam-Report: Content analysis details: pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [212.227.15.3 listed in list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (peter.rindfuss[at]web.de) -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 0.0 HTML_MESSAGE BODY: HTML included in message 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts X-Priority: 3 (Normal) Importance: Normal Priority: normal

test lmtp +++++





From lists at luigirosa.com Sun Feb 10 11:13:54 2013 From: lists at luigirosa.com (Luigi Rosa) Date: Sun, 10 Feb 2013 10:13:54 +0100 Subject: [Dovecot] Dovecot 2.2.beta1 In-Reply-To: References: Message-ID: <511764D2.6050306@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark Sapiro said the following on 07/02/2013 20:06: > Just an FYI, I have been running 2.2.beta1 for a few days now with no > problems, but I'm not using any of the new features ;) Would like to, but on the server I use for testing I have pigeonhole and I cannot compile it against 2.2B1 Is there a version for 2.2? Making install in src make[1]: Entering directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src' Making install in lib-sieve make[2]: Entering directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src/lib-sieve' Making install in plugins make[3]: Entering directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src/lib-sieve/plugins' Making install in vacation make[4]: Entering directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src/lib-sieve/plugins/vacation' /bin/sh ../../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. - -I../../../.. -I./../../ -I/usr/local/include/dovecot -std=gnu99 -g -O2 - -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith - -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime - -Wstrict-aliasing=2 -MT cmd-vacation.lo -MD -MP -MF .deps/cmd-vacation.Tpo -c - -o cmd-vacation.lo cmd-vacation.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../../.. -I./../../ - -I/usr/local/include/dovecot -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes - -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 - -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -MT cmd-vacation.lo -MD -MP -MF .deps/cmd-vacation.Tpo -c cmd-vacation.c -fPIC - -DPIC -o .libs/cmd-vacation.o In file included from cmd-vacation.c:24: ./../../sieve-generator.h:78: error: expected specifier-qualifier-list before 'ARRAY_DEFINE' cmd-vacation.c: In function 'ext_vacation_operation_execute': cmd-vacation.c:696: warning: implicit declaration of function 'ARRAY_DEFINE' cmd-vacation.c:696: error: 'norm_addresses' undeclared (first use in this function) cmd-vacation.c:696: error: (Each undeclared identifier is reported only once cmd-vacation.c:696: error: for each function it appears in.) cmd-vacation.c:696: error: expected expression before 'const' cmd-vacation.c:712: warning: type defaults to 'int' in declaration of 'type name' cmd-vacation.c:712: error: size of array 'type name' is negative cmd-vacation.c:726: warning: type defaults to 'int' in declaration of 'type name' cmd-vacation.c:727: warning: type defaults to 'int' in declaration of 'type name' make[4]: *** [cmd-vacation.lo] Error 1 make[4]: Leaving directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src/lib-sieve/plugins/vacation' make[3]: *** [install-recursive] Error 1 make[3]: Leaving directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src/lib-sieve/plugins' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src/lib-sieve' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/usr/src/dovecot-2.1-pigeonhole-0.3.3/src' make: *** [install-recursive] Error 1 Ciao, luigi - -- / +--[Luigi Rosa]-- \ Your mouse has moved. Please wait while Windows restarts for the change to take effect. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlEXZM0ACgkQ3kWu7Tfl6ZS4lACbB1sO/DRWJ2JDZpYlw3ZmzB4G MgAAoL7RcDj5Ev3I7GriBzDMMhhVztnO =+nYj -----END PGP SIGNATURE----- From julian.rath at gmail.com Sun Feb 10 12:36:49 2013 From: julian.rath at gmail.com (Julian David Rath) Date: Sun, 10 Feb 2013 11:36:49 +0100 Subject: [Dovecot] GSS-SPNEGO with dovecot and Outlook without Samba Message-ID: I am trying to configure a dovecot2 IMAP server to inter-operate with a active directory to authenticate users. The users should be able to login without a password on a domain-joined client(Outlook). Is it possible to do this only with kerberos? I don't want to put a crappy winbind on my mailserver... I already configured my server to authenticate via kerberos(GSSAPI), but Outlook does not even try to authenticate using GSSAPI, it rather uses PLAIN. I would appreciate any help, many thanks in advance. From lists at kokelnet.de Sun Feb 10 14:02:45 2013 From: lists at kokelnet.de (Tobias Hachmer) Date: Sun, 10 Feb 2013 13:02:45 +0100 Subject: [Dovecot] Dovecot 2.2.beta1 In-Reply-To: <511764D2.6050306@luigirosa.com> References: <511764D2.6050306@luigirosa.com> Message-ID: <2300424.fbNJHExuLf@tobias-pc> On Sunday 10 February 2013 10:13:54 Luigi Rosa wrote: > Would like to, but on the server I use for testing I have pigeonhole and I > cannot compile it against 2.2B1 > > Is there a version for 2.2? Yes, see http://pigeonhole.dovecot.org/download.html Pigeonhole package for Dovecot v2.2 (development) # v2.2 code tree: hg clone http://hg.rename-it.nl/dovecot-2.2-pigeonhole/ Regards, Tobias Hachmer From lists at luigirosa.com Sun Feb 10 14:22:03 2013 From: lists at luigirosa.com (Luigi Rosa) Date: Sun, 10 Feb 2013 13:22:03 +0100 Subject: [Dovecot] Dovecot 2.2.beta1 In-Reply-To: <2300424.fbNJHExuLf@tobias-pc> References: <511764D2.6050306@luigirosa.com> <2300424.fbNJHExuLf@tobias-pc> Message-ID: <511790EB.4020807@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tobias Hachmer said the following on 10/02/2013 13:02: >> Is there a version for 2.2? > > Yes, > > see http://pigeonhole.dovecot.org/download.html > > Pigeonhole package for Dovecot v2.2 (development) # v2.2 code tree: hg > clone http://hg.rename-it.nl/dovecot-2.2-pigeonhole/ Sorry, I didn't notice the second half of the download page Both Dovecot 2.2b and Pigeonhole are working as expected right now on my home server with a few mail accounts. Ciao, luigi - -- / +--[Luigi Rosa]-- \ The number of unlikely things that can happen is so large, you can be assured that unlikely things are likely. --Neil deGrasse Tyson @neiltyson via Twitter -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlEXkOsACgkQ3kWu7Tfl6ZQS1QCeKEg6jhHHXNIUFCY9bJFMFvks ucwAniTKa2n/nUzHG7KY5XgD9x/nKzyF =zlRu -----END PGP SIGNATURE----- From jerry at seibercom.net Sun Feb 10 15:49:11 2013 From: jerry at seibercom.net (Jerry) Date: Sun, 10 Feb 2013 08:49:11 -0500 Subject: [Dovecot] GSS-SPNEGO with dovecot and Outlook without Samba In-Reply-To: References: Message-ID: <20130210084911.221212df@scorpio> On Sun, 10 Feb 2013 11:36:49 +0100 Julian David Rath articulated: > I am trying to configure a dovecot2 IMAP server to inter-operate with > a active directory to authenticate users. The users should be able to > login without a password on a domain-joined client(Outlook). Is it > possible to do this only with kerberos? I don't want to put a crappy > winbind on my mailserver... I already configured my server to > authenticate via kerberos(GSSAPI), but Outlook does not even try to > authenticate using GSSAPI, it rather uses PLAIN. > > I would appreciate any help, many thanks in advance. There is no need to re-post your question. If no one answered it the first time, they probably won't the second time either. To make your question more "answerable" you might try posting the following: The Output of "dovecot -n" The version of the domains that you are trying to authenticate with. Your OS and version and any other pertinent network specific data. The version of Outlook that you are experiencing problems with and its patch level if available. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From julf.helsingius at gmail.com Sun Feb 10 17:16:12 2013 From: julf.helsingius at gmail.com (Johan Helsingius) Date: Sun, 10 Feb 2013 16:16:12 +0100 Subject: [Dovecot] dovecot, postfix and sieve - broken by Ubuntu upgrade 10.10 -> 12.04 Message-ID: <5117B9BC.3030606@gmail.com> Hi! In upgrading from Ubuntu 10.10 to 12.04, my mail setup (dovecot using sieve and postfix) broke pretty badly. Have gotten teh imap server working, but local delivery doesn't work. - sieve gets called, but all "fileto" actions result in "failed to store into mailbox 'zzz': Mailbox doesn't exist: zzz Dovecot version is 2.0.19, configuration is: # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-37-generic i686 Ubuntu 12.04.2 LTS ext4 auth_mechanisms = plain login base_dir = /var/run/dovecot/ disable_plaintext_auth = no info_log_path = /var/log/dovecot.info listen = * log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Mail server ready. mail_location = maildir:/srv/mail/%u mail_max_userip_connections = 256 mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_dotlock_change_timeout = 10 mins mbox_lock_timeout = 20 mins mbox_very_dirty_syncs = yes passdb { driver = pam } plugin { sieve = /home/julf/sieve } protocols = imap service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 } user = root } service imap-login { chroot = login user = dovecot } service managesieve-login { chroot = login user = dovecot } service pop3-login { chroot = login user = dovecot } ssl = no userdb { driver = passwd } From h.reindl at thelounge.net Sun Feb 10 17:27:19 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 10 Feb 2013 16:27:19 +0100 Subject: [Dovecot] dovecot, postfix and sieve - broken by Ubuntu upgrade 10.10 -> 12.04 In-Reply-To: <5117B9BC.3030606@gmail.com> References: <5117B9BC.3030606@gmail.com> Message-ID: <5117BC57.5070906@thelounge.net> Am 10.02.2013 16:16, schrieb Johan Helsingius: > In upgrading from Ubuntu 10.10 to 12.04, my mail setup (dovecot using sieve and > postfix) broke pretty badly. Have gotten teh imap server working, but local > delivery doesn't work. - sieve gets called, but all "fileto" actions result in > "failed to store into mailbox 'zzz': Mailbox doesn't exist: zzz the much more interesting infomration would be what dovecot version Ubuntu 10.10 had especially if and how you planned migration or did you blindly upgrade without consider what this means for services and testing at all? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From julf.helsingius at gmail.com Sun Feb 10 17:42:50 2013 From: julf.helsingius at gmail.com (Johan Helsingius) Date: Sun, 10 Feb 2013 16:42:50 +0100 Subject: [Dovecot] dovecot, postfix and sieve - broken by Ubuntu upgrade 10.10 -> 12.04 In-Reply-To: <5117BC57.5070906@thelounge.net> References: <5117B9BC.3030606@gmail.com> <5117BC57.5070906@thelounge.net> Message-ID: <5117BFFA.8090508@gmail.com> > the much more interesting infomration would be what dovecot version > Ubuntu 10.10 had I of course realized that as I was writing my message, but I am afraid I don't have that information available any more. > especially if and how you planned migration or did you blindly upgrade > without consider what this means for services and testing at all? Definitely a mea culpa here. Not much planning, just hoping for the best - counting on config files not having changed too much. I am the first one to admit it definitely isn't how it *should* be done. Julf From h.reindl at thelounge.net Sun Feb 10 17:49:06 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 10 Feb 2013 16:49:06 +0100 Subject: [Dovecot] dovecot, postfix and sieve - broken by Ubuntu upgrade 10.10 -> 12.04 In-Reply-To: <5117BFFA.8090508@gmail.com> References: <5117B9BC.3030606@gmail.com> <5117BC57.5070906@thelounge.net> <5117BFFA.8090508@gmail.com> Message-ID: <5117C172.4060308@thelounge.net> Am 10.02.2013 16:42, schrieb Johan Helsingius: >> the much more interesting infomration would be what dovecot version >> Ubuntu 10.10 had > > I of course realized that as I was writing my message, but I am afraid I don't > have that information available any more. > >> especially if and how you planned migration or did you blindly upgrade >> without consider what this means for services and testing at all? > > Definitely a mea culpa here. Not much planning, just hoping for the > best - counting on config files not having changed too much. I am the > first one to admit it definitely isn't how it *should* be done http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.10-maverick Jan 21 18:55:39 li181-194 dovecot: Dovecot v1.2.12 starting up (core dumps disabled) so you upgraded from doveot 1.2 to 2.0 a major mistake without planning always setup a virtual machine, configure it like your running server and play around with test and snapshots until you have the needed changes for your setup figured out - for any service http://wiki2.dovecot.org/Upgrading/2.0 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From julf.helsingius at gmail.com Sun Feb 10 18:06:10 2013 From: julf.helsingius at gmail.com (Johan Helsingius) Date: Sun, 10 Feb 2013 17:06:10 +0100 Subject: [Dovecot] dovecot, postfix and sieve - broken by Ubuntu upgrade 10.10 -> 12.04 In-Reply-To: <5117C172.4060308@thelounge.net> References: <5117B9BC.3030606@gmail.com> <5117BC57.5070906@thelounge.net> <5117BFFA.8090508@gmail.com> <5117C172.4060308@thelounge.net> Message-ID: <5117C572.5030100@gmail.com> > so you upgraded from doveot 1.2 to 2.0 Evidently, yes. > a major mistake without planning Agree. > always setup a virtual machine, configure it like your running server and play around > with test and snapshots until you have the needed changes for your setup figured out Indeed, a lesson learned. > http://wiki2.dovecot.org/Upgrading/2.0 Thanks! Julf From marc at r4l.com Sun Feb 10 18:48:50 2013 From: marc at r4l.com (Marc Jauvin) Date: Sun, 10 Feb 2013 11:48:50 -0500 Subject: [Dovecot] cPanel && Dovecot 2.x Message-ID: I would like to know if dovecot 2.0.9+ can work with cPanel. I know there is a template for it in cPanel 11.34.1, but when I tried to switch to 2.0.9 (from 1.2.17), it didn't work. I didn't put too much time to test it since I only have one live system to test it with, and I have many customers hosted on this server and I cannot disrupt the service. Has anyone been able to make this work, and if so, is there anything special to do other than rebuilding the config with "/scripts/builddovecotconf" ? Thanks. From CMarcus at Media-Brokers.com Sun Feb 10 19:38:56 2013 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 10 Feb 2013 12:38:56 -0500 Subject: [Dovecot] cPanel && Dovecot 2.x In-Reply-To: References: Message-ID: <5117DB30.5060409@Media-Brokers.com> I imagine this is a question best asked on the cPanel support list, or from whoever you are using for your hosting... On 2013-02-10 11:48 AM, Marc Jauvin wrote: > I would like to know if dovecot 2.0.9+ can work with cPanel. I know there > is a template for it in cPanel 11.34.1, but when I tried to switch to 2.0.9 > (from 1.2.17), it didn't work. I didn't put too much time to test it since > I only have one live system to test it with, and I have many customers > hosted on this server and I cannot disrupt the service. > > Has anyone been able to make this work, and if so, is there anything > special to do other than rebuilding the config with > "/scripts/builddovecotconf" ? > > Thanks. > > > -- Best regards, Charles From rs at sys4.de Sun Feb 10 20:50:13 2013 From: rs at sys4.de (Robert Schetterer) Date: Sun, 10 Feb 2013 19:50:13 +0100 Subject: [Dovecot] dovecot, postfix and sieve - broken by Ubuntu upgrade 10.10 -> 12.04 In-Reply-To: <5117BFFA.8090508@gmail.com> References: <5117B9BC.3030606@gmail.com> <5117BC57.5070906@thelounge.net> <5117BFFA.8090508@gmail.com> Message-ID: <5117EBE5.6090106@sys4.de> Am 10.02.2013 16:42, schrieb Johan Helsingius: >> the much more interesting infomration would be what dovecot version >> Ubuntu 10.10 had > > I of course realized that as I was writing my message, but I am afraid I don't > have that information available any more. > >> especially if and how you planned migration or did you blindly upgrade >> without consider what this means for services and testing at all? > > Definitely a mea culpa here. Not much planning, just hoping for the > best - counting on config files not having changed too much. I am the > first one to admit it definitely isn't how it *should* be done. > > Julf > precise has look http://packages.ubuntu.com/precise/dovecot-common 1:2.0.19 lucid has 1:1.2.9 in orig release oneric 1:2.0.13 Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From rs at sys4.de Sun Feb 10 21:23:52 2013 From: rs at sys4.de (Robert Schetterer) Date: Sun, 10 Feb 2013 20:23:52 +0100 Subject: [Dovecot] dovecot, postfix and sieve - broken by Ubuntu upgrade 10.10 -> 12.04 In-Reply-To: <5117EBE5.6090106@sys4.de> References: <5117B9BC.3030606@gmail.com> <5117BC57.5070906@thelounge.net> <5117BFFA.8090508@gmail.com> <5117EBE5.6090106@sys4.de> Message-ID: <5117F3C8.4010803@sys4.de> Am 10.02.2013 19:50, schrieb Robert Schetterer: > Am 10.02.2013 16:42, schrieb Johan Helsingius: >>> the much more interesting infomration would be what dovecot version >>> Ubuntu 10.10 had >> >> I of course realized that as I was writing my message, but I am afraid I don't >> have that information available any more. >> >>> especially if and how you planned migration or did you blindly upgrade >>> without consider what this means for services and testing at all? >> >> Definitely a mea culpa here. Not much planning, just hoping for the >> best - counting on config files not having changed too much. I am the >> first one to admit it definitely isn't how it *should* be done. >> >> Julf >> > > precise has > > look > > http://packages.ubuntu.com/precise/dovecot-common > > 1:2.0.19 > > lucid has > > 1:1.2.9 > > in orig release > > oneric > > 1:2.0.13 > > Best Regards > MfG Robert Schetterer > looks like 2.1.7 is in quantal so if you wanna stay at ubuntu orig source consider go quantal , or recompile the sources at precise however best way perhaps is use http://xi.rename-it.nl/debian/pool/testing-auto/dovecot-2.1/ 2.1.15 packs also look on package matrix at http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages more info what are the diffs is at http://wiki2.dovecot.org/Upgrading/ Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Mon Feb 11 01:15:49 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 01:15:49 +0200 Subject: [Dovecot] dovecot-2.2: dsync to imapc not working In-Reply-To: <5111F388.9030305@yandex.ru> References: <5111F388.9030305@yandex.ru> Message-ID: <1360538149.10326.70.camel@innu> On Wed, 2013-02-06 at 10:09 +0400, Evgeny Basov wrote: > # dovecot --version > 20130205 (03a0af22100d+) > > built with imapc backend. > > I'm tried to sync mailboxes from another server after clean mail > directory localy: > > # dsync -v -o imapc_user="user at example.org" -o imapc_password="pass" > -o imapc_host=imap.example.org -o imapc_features=rfc822.size -o > mailbox_list_index=no backup -R -f -u "user at example.org" imapc: > > and get this message > > dsync(user at example.org): Error: Exporting mailbox INBOX failed: Backend > doesn't support GUIDs, sync with header hashes instead Should work better now in latest hg. From tss at iki.fi Mon Feb 11 03:16:45 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 03:16:45 +0200 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <1360545405.10326.82.camel@innu> On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. How about this, added to hg: plugin { # LDA/LMTP allows saving the last mail to bring user from under quota to # over quota, if the quota doesn't grow too high. Default is to allow as # long as quota will stay under 10% above the limit. Also allowed e.g. 10M. #quota_last_extra = 10%% } Each quota root has its own limit, so if using multiple quota roots (pretty rare) you'd have to set also quota2_last_extra, etc. I wonder if there's a better name for this than "last_extra".. From tss at iki.fi Mon Feb 11 03:26:05 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 03:26:05 +0200 Subject: [Dovecot] Moving index files on another disk: no troubles but need more assurances In-Reply-To: <20130208131740.39334u183vmu13ac@webmail.uniud.it> References: <20130208131740.39334u183vmu13ac@webmail.uniud.it> Message-ID: <1360545965.10326.89.camel@innu> On Fri, 2013-02-08 at 13:17 +0100, Paolo Miotto wrote: > So I set up a test server, copied a true mailbox and changed mail > location from: > mail_location = maildir:~/Maildir > to > mail_location = maildir:~/Maildir:INDEX=/srv/indexes/%1n/%n > (all users under the same domain, no need to use %d). > > On the IMAP side all seems to work fine, new indexes are created in > the new location (dovecot.index.log on 1st access, dovecot.index.cache > on message manipolation). > > A test client (evolution) doesn't notice at all, and I have verified > via a telnet imap session that UIDs don't change. Your load probably temporarily increases so cache file gets filled. > But I see that when I move the indexes the IMAP HIGHESTMODSEQ response > changes and is reset to 1. Can this baffle the clients? On the rfc I > don't found reference to this status walking backwards. Hmm. Good question. I remember wondering about using some kind of a timestamp-based root value. Something like time()*1000 would probably work quite well, but this of course increases the size of all the modseq values, which somewhat wastes bandwidth and also makes debugging a bit more annoying :) Thunderbird is probably the only widely used client that uses QRESYNC. You could see if it breaks. > Need I to verify that all messages have the virtual size (W=nnnn) in > the file name (the mailboxes are from a previous cyrus installation, > then we switch to dovecot 1.2)? W=size or S=size aren't required in the filenames. Also if you change them, make sure to change dovecot-uidlist also or the UIDs change. > What other test you suggest that checking UIDL on the pop3 side? POP3 session with UIDL command? :) Anyway POP3 UIDLs are never stored in index files. > What happens if the indexes disk disappears (broken or removed from > bay or all paths down for FC/iscsi)? Can dovecot 1.2 continue with > INDEX=MEMORY as for a disk full? There is code to do something like that, but it might not work perfectly. Especially with v1.2 that is rather old. From tss at iki.fi Mon Feb 11 03:28:20 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 03:28:20 +0200 Subject: [Dovecot] Imap process crash: assertion failed: (full_fs_access) In-Reply-To: References: Message-ID: <1360546100.10326.90.camel@innu> On Thu, 2013-02-07 at 23:59 -0800, Joseph Tam wrote: > Any guess at what would cause this? > > Feb 7 21:20:53 server dovecot: imap(user): Panic: file > mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion > failed: (full_fs_access) LIST command that had invalid parameters that somehow got through the first validity check. What namespace config do you have? From tss at iki.fi Mon Feb 11 03:29:34 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 03:29:34 +0200 Subject: [Dovecot] fts_solr search in subfolders? In-Reply-To: References: Message-ID: <1360546174.10326.91.camel@innu> On Thu, 2013-02-07 at 11:04 +0100, Frederic Biermann wrote: > Hello, > > i am using dovecot 2.1.x with Solr 4.x and it is all working fine. > Search in current selected folder is extremly fast. > > But i want to ask if it is possible that i have selected inbox, setup a search and get results from subfolders. > This is not working for me. I have to select the subfolder and then i get results. > > Is there any solution to fix this? Create a virtual folder that contains all the folders you want to search: http://wiki2.dovecot.org/Plugins/Virtual From tss at iki.fi Mon Feb 11 03:31:33 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 03:31:33 +0200 Subject: [Dovecot] Disabling auth caching just from one passdb In-Reply-To: <5114A945.4090700@um.es> References: <5114A945.4090700@um.es> Message-ID: <1360546293.10326.93.camel@innu> On Fri, 2013-02-08 at 08:29 +0100, Angel L. Mateo wrote: > Hello, > > I'm using various password databases (one of them is a ldap database, > which is the used for users connecting with email clients, and the other > with pam, using a pam module integrated with our SSO that is used from > our webmail). I would like to disable password caching in my pam > database (because 'passwords' provided by our SSO are single use tokens, > so caching them has no benefit), but not in the ldap. > > As far as I know, auth_cache_size is the option to enable caching in > all databases. But, is there any option I could use to disable it in a > specific password database? Not without patching the code. Maybe some day in future some auth settings could be overridden within passdb{} and userdb{}, but this isn't that easy to implement. But that is needed in any case in future to provide namespace { imapc_* } settings. From sven at svenhartge.de Mon Feb 11 03:43:47 2013 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 11 Feb 2013 02:43:47 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> Message-ID: <19gc77qsdov8@mids.svenhartge.de> Timo Sirainen wrote: > How about this, added to hg: > plugin { > # LDA/LMTP allows saving the last mail to bring user from under quota to > # over quota, if the quota doesn't grow too high. Default is to allow as > # long as quota will stay under 10% above the limit. Also allowed e.g. 10M. > #quota_last_extra = 10%% > } > Each quota root has its own limit, so if using multiple quota roots > (pretty rare) you'd have to set also quota2_last_extra, etc. > I wonder if there's a better name for this than "last_extra".. quota_grace_extra quota_grace_limit quota_over_quota quota_overflow Gr??e, Sven. -- Sigmentation fault. Core dumped. From daniel.parthey at informatik.tu-chemnitz.de Mon Feb 11 04:38:40 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 11 Feb 2013 03:38:40 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <19gc77qsdov8@mids.svenhartge.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> Message-ID: <20130211023840.GA14418@daniel.localdomain> Sven Hartge wrote: > Timo Sirainen wrote: > > I wonder if there's a better name for this than "last_extra".. quota_surplus Regards Daniel -- https://plus.google.com/103021802792276734820 From dave at dawoodfall.net Mon Feb 11 06:14:29 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 11 Feb 2013 04:14:29 +0000 Subject: [Dovecot] Keyword Flags Message-ID: <20130211041429.GA19089@Blackswan> I mostly use Mutt to read mail but today I fired up Thunderbird. I have a shell script that looks for new mail, which looks for files in cur/ and new/ ending with the server hostname or a comma. When I fired up TB it seems that all new mail has been given a flag of ,a which I gather means it matches a keyword. My questions are: How did this happen? Did TB do something strange to my mail when it fetched headers? If I change my search parameters to ',[a-z]' will this only catch new mail? Cheers -- When all you have is a Swiss Army Knife, every problem looks like email. -- Peter da Silva From tss at iki.fi Mon Feb 11 06:24:49 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 06:24:49 +0200 Subject: [Dovecot] Keyword Flags In-Reply-To: <20130211041429.GA19089@Blackswan> References: <20130211041429.GA19089@Blackswan> Message-ID: <1360556689.10326.94.camel@innu> On Mon, 2013-02-11 at 04:14 +0000, David Woodfall wrote: > I mostly use Mutt to read mail but today I fired up Thunderbird. > > I have a shell script that looks for new mail, which looks for files > in cur/ and new/ ending with the server hostname or a comma. > > When I fired up TB it seems that all new mail has been given a flag of > ,a which I gather means it matches a keyword. Yes, see the 0 number in dovecot-keywords > My questions are: > > How did this happen? > Did TB do something strange to my mail when it fetched headers? > If I change my search parameters to ',[a-z]' will this only catch new > mail? I guess it marks it as $nonspam. From dave at dawoodfall.net Mon Feb 11 06:39:31 2013 From: dave at dawoodfall.net (David Woodfall) Date: Mon, 11 Feb 2013 04:39:31 +0000 Subject: [Dovecot] Keyword Flags In-Reply-To: <1360556689.10326.94.camel@innu> References: <20130211041429.GA19089@Blackswan> <1360556689.10326.94.camel@innu> Message-ID: <20130211043931.GB19089@Blackswan> On (11/02/13 06:24), Timo Sirainen put forth the proposition: >On Mon, 2013-02-11 at 04:14 +0000, David Woodfall wrote: >> I mostly use Mutt to read mail but today I fired up Thunderbird. >> >> I have a shell script that looks for new mail, which looks for files >> in cur/ and new/ ending with the server hostname or a comma. >> >> When I fired up TB it seems that all new mail has been given a flag of >> ,a which I gather means it matches a keyword. > >Yes, see the 0 number in dovecot-keywords > >> My questions are: >> >> How did this happen? >> Did TB do something strange to my mail when it fetched headers? >> If I change my search parameters to ',[a-z]' will this only catch new >> mail? > >I guess it marks it as $nonspam. I just had a look in the dovecot.keywords file and it had: 0 junk 1 nonjunk So it looks like TB labelled all my mailing list mail as junk. From dbenfell at gmail.com Mon Feb 11 07:59:33 2013 From: dbenfell at gmail.com (David Benfell) Date: Sun, 10 Feb 2013 21:59:33 -0800 Subject: [Dovecot] Keyword Flags In-Reply-To: <20130211043931.GB19089@Blackswan> References: <20130211041429.GA19089@Blackswan> <1360556689.10326.94.camel@innu> <20130211043931.GB19089@Blackswan> Message-ID: <511888C5.2010407@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2/10/2013 8:39 PM, David Woodfall wrote: > > So it looks like TB labelled all my mailing list mail as junk. Thunderbird is initially pretty stupid about what it labels as junk. As you unmark non-junk and mark actual junk, it gets better, but never great. It's certainly no substitute for spamassassin. But this wanders off topic for this list, so I'll leave it there. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRGIjEAAoJEAB6S8caAYoZ6hkP/0cMy5I6a5VJ+ZO7khQiFZgZ 8J9IU6D4FOFrAKkkNgjNV0BLV3zVgkY00OVKJWsFdP3ojS5BIrvLSPor/T1IHRSf ElbycA2kcqbTc4idASDsC7kEk6eUxtG+V+J75e8+HHBzY/jEgvdeLq15+BcN+/PK hHpoZ63WeWKhz6WGJDB9CyX84e1kVNiazrwIqHHqHbZdLV5Y3Qjuk3AsJz9DKJ+l ev/qWhjkitQ5CxJ6bh+cHtzkKiMMiGPUDSEWlN7Bbcg3Zs0iNc61sUagB+PzuaMC Ft6DvBKwGkaT7h1LWZL4i7l3qWNwoLa1HDq406IQYvi2+BnO/vt58y7UITYk2CiG 0mZcZZFkSuCivLQNX++B7vQQmXpY0UtmfD0hdBZjKIugyanhEnPHRr8FDHgsB+Xc lN5KNcMKstLoFWe22U75PjskVDfotiIeM7l+GNhXMUih2wisB73cMhS/e5gLnv03 OU2kQLL/eze1XNAB9ZGTtBQ05A13z8FlsgiHcDr8o4Ry6il6Nz7yChpcayTT6YOw M13qt3ahN55wCKoyuctb9sTLMA6NHpYDi/9P+wG/AJ8uv3a1wSP3sdkl4nh0v5y1 s4Qbv9qOE9hdzRnl+qVjKAbPmTh0JJq/fAxTFPPsUN5S19UD0eBlRUEM6iYeVwAy 9hFU2RCkYQjLP3vOzbuU =6QjG -----END PGP SIGNATURE----- From ya.mwork at yandex.ru Mon Feb 11 10:07:51 2013 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Mon, 11 Feb 2013 12:07:51 +0400 Subject: [Dovecot] dovecot-2.2: dsync to imapc not working In-Reply-To: <1360538149.10326.70.camel@innu> References: <5111F388.9030305@yandex.ru> <1360538149.10326.70.camel@innu> Message-ID: <5118A6D7.9090206@yandex.ru> 11.02.2013 03:15, Timo Sirainen ?????: > Should work better now in latest hg. # dovecot --version 20130211 (ed5fa76f0dc7+) Thanks. Work better but not fine. Not all emails was syncrornized: dsync(user at example.com): Debug: brain M: in state=sync_mails dsync(user at example.com): Debug: brain M: in box 'Postmaster' recv_state=mails send_state=done dsync(user at example.com): Debug: brain M: import mail uid 43414 guid dsync(user at example.com): Debug: brain M: out box 'Postmaster' recv_state=mails send_state=done changed=1 dsync(user at example.com): Debug: brain M: out state=sync_mails changed=1 dsync(user at example.com): Debug: brain S: in state=sync_mails dsync(user at example.com): Debug: brain S: in box 'Postmaster' recv_state=recv_last_common send_state=mails dsync(user at example.com): Debug: brain S: out box 'Postmaster' recv_state=recv_last_common send_state=done changed=1 dsync(user at example.com): Debug: brain S: out state=sync_mails changed=1 dsync(user at example.com): Debug: brain M: in state=sync_mails dsync(user at example.com): Debug: brain M: in box 'Postmaster' recv_state=mails send_state=done dsync(user at example.com): Error: Mailbox Postmaster: Remote didn't send mail UID=43483 dsync(user at example.com): Error: Mailbox Postmaster: Remote didn't send mail UID=43484 dsync(user at example.com): Error: Mailbox Postmaster: Remote didn't send mail UID=43418 dsync(user at example.com): Error: Mailbox Postmaster: Remote didn't send mail UID=43485 ? dsync(user at example.com): Error: Mailbox Postmaster: Remote didn't send mail UID=43578 dsync(user at example.com): Error: Mailbox Postmaster: Remote didn't send mail UID=43579 dsync(user at example.com): Debug: brain M: out box 'Postmaster' recv_state=recv_last_common send_state=done changed=1 dsync(user at example.com): Debug: brain M: out state=sync_mails changed=1 dsync(user at example.com): Debug: brain S: in state=sync_mails dsync(user at example.com): Debug: brain S: in box 'Postmaster' recv_state=recv_last_common send_state=done dsync(user at example.com): Debug: brain S: out box '' recv_state=recv_last_common send_state=done changed=1 dsync(user at example.com): Debug: brain S: out state=slave_recv_mailbox changed=1 ? From noel.butler at ausics.net Mon Feb 11 10:45:34 2013 From: noel.butler at ausics.net (Noel Butler) Date: Mon, 11 Feb 2013 18:45:34 +1000 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <19gc77qsdov8@mids.svenhartge.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> Message-ID: <1360572334.4986.1.camel@tardis> On Mon, 2013-02-11 at 02:43 +0100, Sven Hartge wrote: > > I wonder if there's a better name for this than "last_extra".. > > quota_grace_extra Sounds good -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From f.biermann at eye-on.de Mon Feb 11 11:46:31 2013 From: f.biermann at eye-on.de (Frederic Biermann) Date: Mon, 11 Feb 2013 10:46:31 +0100 Subject: [Dovecot] fts_solr search in subfolders? In-Reply-To: <1360546174.10326.91.camel@innu> References: <1360546174.10326.91.camel@innu> Message-ID: hello, created virtual folder "virtuall.all" now usin this folder is working fine. But when i want to search in this virtual folder i get these errors with fts-solr: Feb 11 10:37:24 myserver dovecot: indexer-worker(username at mailadress.de): Panic: file solr-connection.c: line 419 (solr_connection_select): assertion failed: (!conn->posting) Feb 11 10:37:24 myserver dovecot: indexer-worker(username at mailadress.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0xcff170] -> /usr/lib/dovecot/libdovecot.so.0 [0xcff1cd] -> /usr/lib/dovecot/libdovecot.so.0 [0xcfe7b4] -> /usr/lib/dovecot/lib21_fts_solr_plugin.so [0x3cfec5] -> /usr/lib/dovecot/lib21_fts_solr_plugin.so [0x3cc63f] -> /usr/lib/dovecot/lib20_fts_plugin.so(fts_backend_get_last_uid+0x6b) [0x3c09cb] -> /usr/lib/dovecot/lib20_fts_plugin.so [0x3c53c1] -> /usr/lib/dovecot/lib20_fts_plugin.so [0x3c6019] -> /usr/lib/dovecot/lib20_virtual_plugin.so [0x8d3115] -> /usr/lib/dovecot/lib20_fts_plugin.so [0x3c5f18] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0xf) [0x345f9f] -> dovecot/indexer-worker [0x8049d3f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x42) [0xd0bcb2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xef) [0xd0d23f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x48) [0xd0bc58] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2d) [0xcf5c6d] -> Feb 11 10:37:24 myserver dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for username at mailadress.de Feb 11 10:37:24 myserver dovecot: indexer-worker(username at mailadress.de): Fatal: master: service(indexer-worker): child 30511 killed with signal 6 (core dumps disabled) Feb 11 10:37:24 myserver dovecot: imap(username at mailadress.de): Error: indexer failed to index mailbox virtual.all After some (realy long time) i get correct answer with the right UIDs of mails, but it's not from solr. :-( Search in normal inbox is not concerned. Search there is very fast and i log i get these: Feb 11 10:44:44 myserver dovecot: indexer-worker(username at mailadress.de): Indexed 9 messages in INBOX Anyone some ideas? Am 11.02.2013 um 02:29 schrieb Timo Sirainen: > On Thu, 2013-02-07 at 11:04 +0100, Frederic Biermann wrote: >> Hello, >> >> i am using dovecot 2.1.x with Solr 4.x and it is all working fine. >> Search in current selected folder is extremly fast. >> >> But i want to ask if it is possible that i have selected inbox, setup a search and get results from subfolders. >> This is not working for me. I have to select the subfolder and then i get results. >> >> Is there any solution to fix this? > > Create a virtual folder that contains all the folders you want to > search: http://wiki2.dovecot.org/Plugins/Virtual > > From jtam.home at gmail.com Mon Feb 11 13:20:10 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 11 Feb 2013 03:20:10 -0800 (PST) Subject: [Dovecot] Imap process crash: assertion failed: (full_fs_access) In-Reply-To: References: Message-ID: Timo Sirainen writes: > > Any guess at what would cause this? > > > > Feb 7 21:20:53 server dovecot: imap(user): Panic: file > > mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion > > failed: (full_fs_access) > > LIST command that had invalid parameters that somehow got through the > first validity check. What namespace config do you have? I got more info from the user that had this problem. He reported that he was was setting up a new Email client (Nexus4 Android) and he sent me a screenshot of his setup. Everything look ordinary except perhaps the IMAP prefix set to "mail". My setup documents inform users to leave it blank, but sometimes they use "mail/", a holdover from the uw-imapd days. I provide some aliases for that, but I don't know what would happen if you leave off the trailing "/". My namespace settings as reported by dovecot -n: namespace { inbox = yes location = prefix = separator = / } (The following are aliases to keep backward compatibility) namespace { alias_for = hidden = yes list = no location = prefix = / separator = / } namespace { alias_for = hidden = yes list = no location = prefix = mail/ separator = / } namespace { alias_for = hidden = yes list = no location = prefix = ~/mail/ separator = / } namespace { alias_for = hidden = yes list = no location = prefix = ~%u/mail/ separator = / } I've asked the user to change his prefix, and he'll probably report back. Joseph Tam From CMarcus at Media-Brokers.com Mon Feb 11 13:56:43 2013 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 11 Feb 2013 06:56:43 -0500 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <19gc77qsdov8@mids.svenhartge.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> Message-ID: <5118DC7B.3040509@Media-Brokers.com> On 2013-02-10 8:43 PM, Sven Hartge wrote: > quota_grace_extra Smaller is better... why not just quota_grace -- Best regards, Charles From dovecot at lists.wgwh.ch Mon Feb 11 16:26:19 2013 From: dovecot at lists.wgwh.ch (Oli Schacher) Date: Mon, 11 Feb 2013 15:26:19 +0100 Subject: [Dovecot] dovecot 2.2: doveadm-server crash in latest hg Message-ID: <20130211152619.76944273@boscos.leetdreams.ch> Hi Timo With the current hg (ed5fa76f0dc7), I get a crash as soon as I append a single message to a dsync replicated mailbox. Setup is the same as in the thread "dsync replication errors" ( http://www.dovecot.org/list/dovecot/2013-January/070574.html ) Feb 11 15:01:19 doco1 dovecot: imap-login: Login: user=, method=PLAIN, rip=192.168.23.130, lip=192.168.23.61, mpid=1709, session= Feb 11 15:01:32 doco1 dovecot: doveadm: Fatal: master: service(doveadm): child 1713 killed with signal 11 (core dumped) Feb 11 15:01:56 doco1 dovecot: doveadm: Fatal: master: service(doveadm): child 1715 killed with signal 11 (core dumped) bt full is attached. Thanks, Oli -- message transmitted on 100% recycled electrons -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: bt-full.txt URL: From tss at iki.fi Mon Feb 11 17:38:32 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Feb 2013 17:38:32 +0200 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <20130211023840.GA14418@daniel.localdomain> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> <20130211023840.GA14418@daniel.localdomain> Message-ID: On 11.2.2013, at 4.38, Daniel Parthey wrote: > Sven Hartge wrote: >> Timo Sirainen wrote: >>> I wonder if there's a better name for this than "last_extra".. > > quota_surplus I like this. But my English isn't good enough to know if that word really fits it? The Finnish translation looks pretty close I think.. Of course, if I don't know that word so well, how many other non-English admins will know what it means?.. I guess it's mostly related to some money things. From CMarcus at Media-Brokers.com Mon Feb 11 17:43:55 2013 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 11 Feb 2013 10:43:55 -0500 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> <20130211023840.GA14418@daniel.localdomain> Message-ID: <511911BB.7000905@Media-Brokers.com> On 2013-02-11 10:38 AM, Timo Sirainen wrote: > On 11.2.2013, at 4.38, Daniel Parthey wrote: > >> Sven Hartge wrote: >>> Timo Sirainen wrote: >>>> I wonder if there's a better name for this than "last_extra".. >> quota_surplus > I like this. But my English isn't good enough to know if that word really fits it? The Finnish translation looks pretty close I think.. > > Of course, if I don't know that word so well, how many other non-English admins will know what it means?.. I guess it's mostly related to some money things. It does fit it, but I thought of on I like better... quote_trigger -- Best regards, Charles From trashcan at odo.in-berlin.de Mon Feb 11 18:19:06 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Mon, 11 Feb 2013 17:19:06 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> <20130211023840.GA14418@daniel.localdomain> Message-ID: <61499E9B-2998-4E14-9AED-26BE0C51C19C@odo.in-berlin.de> On 11.02.2013, at 16:38, Timo Sirainen wrote: > On 11.2.2013, at 4.38, Daniel Parthey wrote: >> Sven Hartge wrote: >>> Timo Sirainen wrote: >>>> I wonder if there's a better name for this than "last_extra".. >> >> quota_surplus > > I like this. But my English isn't good enough to know if that word really fits it? It does, but what about: quota_final_add_on Regards, Michael From lists at kokelnet.de Mon Feb 11 18:35:30 2013 From: lists at kokelnet.de (Tobias Hachmer) Date: Mon, 11 Feb 2013 17:35:30 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <61499E9B-2998-4E14-9AED-26BE0C51C19C@odo.in-berlin.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <61499E9B-2998-4E14-9AED-26BE0C51C19C@odo.in-berlin.de> Message-ID: <2304471.ejn3KTWc91@tobias-pc> On Monday 11 February 2013 17:19:06 Michael Grimm wrote: > >>> Timo Sirainen wrote: > >>>> I wonder if there's a better name for this than "last_extra".. > >> > >> quota_surplus > > > > I like this. But my English isn't good enough to know if that word really > > fits it? > It does, but what about: > > quota_final_add_on What means surplus? Why not simplify things and calll it - quota_mailplus - quota_plusmail - quota_oneplus - quota_plusone Tobias Hachmer From dg at dguhl.org Mon Feb 11 18:44:15 2013 From: dg at dguhl.org (Dennis Guhl) Date: Mon, 11 Feb 2013 17:44:15 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <5118DC7B.3040509@Media-Brokers.com> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> <5118DC7B.3040509@Media-Brokers.com> Message-ID: <20130211164415.GA1488@PC211.ikt.de> On Mon, Feb 11, 2013 at 06:56:43AM -0500, Charles Marcus wrote: > On 2013-02-10 8:43 PM, Sven Hartge wrote: > >quota_grace_extra > > Smaller is better... why not just > > quota_grace Sounds good for me. In particular the similarity to the disk quotas grace period is blatant. Dennis From f.bonnet at esiee.fr Mon Feb 11 19:00:35 2013 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 11 Feb 2013 18:00:35 +0100 Subject: [Dovecot] Migration to Gmail ... Message-ID: <511923B3.6010607@esiee.fr> Hello Anyone has migrated Dovecot IMAP only mailboxes server to GMAIL ? We use MBOX format. Any feedback welcome ... Thanks you From h.reindl at thelounge.net Mon Feb 11 19:13:24 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 11 Feb 2013 18:13:24 +0100 Subject: [Dovecot] Migration to Gmail ... In-Reply-To: <511923B3.6010607@esiee.fr> References: <511923B3.6010607@esiee.fr> Message-ID: <511926B4.3050202@thelounge.net> Am 11.02.2013 18:00, schrieb Frank Bonnet: > Anyone has migrated Dovecot IMAP only mailboxes server to GMAIL ? use imapsync to copy messages to different servers > We use MBOX format does not matter in context of IMAP -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From michael.abbott at apple.com Mon Feb 11 21:32:21 2013 From: michael.abbott at apple.com (Mike Abbott) Date: Mon, 11 Feb 2013 13:32:21 -0600 Subject: [Dovecot] error running indexer-worker as non-root user Message-ID: <3BE01AE6-E524-4C35-9FAD-874C4FAA876C@apple.com> Using dovecot-2.1.15 if I run indexer-worker as a non-root user it fails with an error: Feb 11 13:06:47 indexer-worker: Error: user foobar: Error reading configuration: net_connect_unix(/var/run/dovecot/config) failed: Permission denied This is what I added to 10-master.conf: service indexer-worker { user = _dovecot } The wiki says it should work: indexer-worker [...] ? user=root, but the privileges are (temporarily) dropped to the mail user's privileges after userdb lookup. If only a single UID is used, user can be set to the mail UID for higher security, because the process can't gain root privileges anymore. And the code says it should work: static void drop_privileges(void) { [...] if (set.uid != 0) { /* open config connection before dropping privileges */ Of course the config socket is locked down as it should be: srw------- 1 root daemon 0 Feb 11 13:06 /var/run/dovecot/config Here's where the error comes from: 2 libdovecot.0.dylib 0x00000001019612a0 master_service_settings_read + 2168 3 libdovecot.0.dylib 0x000000010196193e master_service_settings_cache_read + 294 4 libdovecot-storage.0.dylib 0x000000010185d9ad mail_storage_service_read_settings + 510 5 libdovecot-storage.0.dylib 0x000000010185da5d mail_storage_service_lookup + 96 6 libdovecot-storage.0.dylib 0x000000010185f06a mail_storage_service_lookup_next + 42 7 indexer-worker 0x00000001018101a1 master_connection_input + 335 8 libdovecot.0.dylib 0x0000000101972455 io_loop_call_io + 46 9 libdovecot.0.dylib 0x000000010197343e io_loop_handler_run + 214 10 libdovecot.0.dylib 0x0000000101972600 io_loop_run + 77 11 libdovecot.0.dylib 0x00000001019601ef master_service_run + 24 12 indexer-worker 0x000000010180ff17 main + 490 13 libdyld.dylib 0x00007fff89d2f7b5 start + 1 Is this a bug or am I configuring it incorrectly? Thanks. From rlillard at sonic.net Mon Feb 11 21:38:28 2013 From: rlillard at sonic.net (Raymond Lillard) Date: Mon, 11 Feb 2013 11:38:28 -0800 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <1360545405.10326.82.camel@innu> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> Message-ID: <511948B4.50009@sonic.net> On 02/10/2013 05:16 PM, Timo Sirainen wrote: > On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: >> Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. >> >> So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. > > How about this, added to hg: > > plugin { > # LDA/LMTP allows saving the last mail to bring user from under quota to > # over quota, if the quota doesn't grow too high. Default is to allow as > # long as quota will stay under 10% above the limit. Also allowed e.g. 10M. > #quota_last_extra = 10%% > } > > Each quota root has its own limit, so if using multiple quota roots > (pretty rare) you'd have to set also quota2_last_extra, etc. > > I wonder if there's a better name for this than "last_extra".. quota_size_elasticity From dave.mehler at gmail.com Mon Feb 11 22:34:54 2013 From: dave.mehler at gmail.com (David Mehler) Date: Mon, 11 Feb 2013 15:34:54 -0500 Subject: [Dovecot] backing up maildir dovecot files Message-ID: Hello, I'm having to migrate servers. Both are dovecot2 systems. I'm wanting to copy over my mail store from one system to the other. I'd like to preserve dates/times of emails. These are maildir setups on both boxes, I'd like to be as transparent to the end user as possible. I currently have the first dovecot system offline and the second dovecot system is offline. Recommendations? Thanks. Dave. From h.reindl at thelounge.net Mon Feb 11 22:47:57 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 11 Feb 2013 21:47:57 +0100 Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: References: Message-ID: <511958FD.8040205@thelounge.net> Am 11.02.2013 21:34, schrieb David Mehler: > I'm having to migrate servers. Both are dovecot2 systems. I'm wanting > to copy over my mail store from one system to the other. I'd like to > preserve dates/times of emails. These are maildir setups on both > boxes, I'd like to be as transparent to the end user as possible. I > currently have the first dovecot system offline and the second dovecot > system is offline. assuming that BOTH servers are down as fro any other transfers like mysql-datadirs and any critical things which should go save and fast by preserve attributes: rsync --force --delete-after -tPrlHpogEAXz /folder/ root at newserver:/folder/ # -z compress # -t timestamps # -P progress # -r recursive # -l links # -H hard-links # -p permissions # -o owner # -g group # -E executability # -A acls # -X xtended attributes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From slitt at troubleshooters.com Mon Feb 11 23:37:41 2013 From: slitt at troubleshooters.com (Steve Litt) Date: Mon, 11 Feb 2013 16:37:41 -0500 Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: <511958FD.8040205@thelounge.net> References: <511958FD.8040205@thelounge.net> Message-ID: <20130211163741.3bd8e49c@mydesk> On Mon, 11 Feb 2013 21:47:57 +0100 Reindl Harald wrote: > > > Am 11.02.2013 21:34, schrieb David Mehler: > > I'm having to migrate servers. Both are dovecot2 systems. I'm > > wanting to copy over my mail store from one system to the other. > > I'd like to preserve dates/times of emails. These are maildir > > setups on both boxes, I'd like to be as transparent to the end user > > as possible. I currently have the first dovecot system offline and > > the second dovecot system is offline. > > assuming that BOTH servers are down as fro any other transfers > like mysql-datadirs and any critical things which should go > save and fast by preserve attributes: > > rsync --force --delete-after -tPrlHpogEAXz /folder/ > root at newserver:/folder/ > > # -z compress > # -t timestamps > # -P progress > # -r recursive > # -l links > # -H hard-links > # -p permissions > # -o owner > # -g group > # -E executability > # -A acls > # -X xtended attributes That's how I'd do it also. I think if you didn't mind including -D, which from my understanding is --special and --devices, neither of which I'd expect in a maildir, and if you wanted to do your deletion manually after the fact, would this be equivalent? rsync -PaHAX /folder/ The man page says -a (--archive) is equivalent to -rlptgoD, and you have all but D listed up there. My thinking on later manually deleting the old maildir, instead of letting rsync do it, is that if somehow, some weird thing goes wrong, I have the old one for backup. Who knows, maybe I copied the thing on top of the wrong other maildir and have to back it out -- I'd have the old maildir as a reference of which files. Of course there are arguments for instant deletion too, like the fact that a person runs the risk of accidentally deleting the wrong one, or the fact that a huge maildir folder could take hours to delete. Personally, I never delete til I'm sure it got to the right place and is backed up there, and sometimes I keep it on the old box for a month "just in case" if the old box won't be used anymore. SteveT From h.reindl at thelounge.net Mon Feb 11 23:44:28 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 11 Feb 2013 22:44:28 +0100 Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: <20130211163741.3bd8e49c@mydesk> References: <511958FD.8040205@thelounge.net> <20130211163741.3bd8e49c@mydesk> Message-ID: <5119663C.8030508@thelounge.net> Am 11.02.2013 22:37, schrieb Steve Litt: > On Mon, 11 Feb 2013 21:47:57 +0100 > Reindl Harald wrote: > >> >> >> Am 11.02.2013 21:34, schrieb David Mehler: >>> I'm having to migrate servers. Both are dovecot2 systems. I'm >>> wanting to copy over my mail store from one system to the other. >>> I'd like to preserve dates/times of emails. These are maildir >>> setups on both boxes, I'd like to be as transparent to the end user >>> as possible. I currently have the first dovecot system offline and >>> the second dovecot system is offline. >> >> assuming that BOTH servers are down as fro any other transfers >> like mysql-datadirs and any critical things which should go >> save and fast by preserve attributes: >> >> rsync --force --delete-after -tPrlHpogEAXz /folder/ >> root at newserver:/folder/ >> >> # -z compress >> # -t timestamps >> # -P progress >> # -r recursive >> # -l links >> # -H hard-links >> # -p permissions >> # -o owner >> # -g group >> # -E executability >> # -A acls >> # -X xtended attributes > > That's how I'd do it also. I think if you didn't mind including -D, > which from my understanding is --special and --devices, neither of > which I'd expect in a maildir, and if you wanted to do your deletion > manually after the fact, would this be equivalent? well, i have them ususally in my "rsync.sh" but did not expect such files in a maildir, but yes it does not hurt > rsync -PaHAX /folder/ > > The man page says -a (--archive) is equivalent to -rlptgoD, and you > have all but D listed up there. > > My thinking on later manually deleting the old maildir, instead of > letting rsync do it, is that if somehow, some weird thing goes wrong, I > have the old one for backup. Who knows, maybe I copied the thing on top > of the wrong other maildir and have to back it out -- I'd have the old > maildir as a reference of which files. you missunderstand "--delete-after" this is for delete files in the TARGET folder which is not or no longer in the source and IMHO very very important if you want sync folders 1:1 because old artefacts can have very bad effects without you merge folders and if i know "hey my source contains exactly what i need, not more and lot less" this is not what i would like and never did in 10 years IT -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From pch14 at myzel.net Tue Feb 12 00:04:56 2013 From: pch14 at myzel.net (Hungerburg) Date: Mon, 11 Feb 2013 23:04:56 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <20130211164415.GA1488@PC211.ikt.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> <5118DC7B.3040509@Media-Brokers.com> <20130211164415.GA1488@PC211.ikt.de> Message-ID: <51196B08.4000002@myzel.net> Am 2013-02-11 17:44, schrieb Dennis Guhl: > On Mon, Feb 11, 2013 at 06:56:43AM -0500, Charles Marcus wrote: >> On 2013-02-10 8:43 PM, Sven Hartge wrote: >> >> quota_grace > > Sounds good for me. In particular the similarity to the disk quotas > grace period is blatant. Not a native english speaker, but quota_grace is very concise and resonates in a way, that makes it easily rememberable: the quota is not enforced in strictness, but handled gracefully. The server for once turns a blind eye to the quota. Documentation: quota_grace - specify an amount in that a single incoming message may exceed the quota without being rejected. Useful in situations, where ... *Alternatively* it could also be a binary option: Only start enforcing the quota after the fact, when it is already exceeded. -- peter From CMarcus at Media-Brokers.com Tue Feb 12 00:17:11 2013 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 11 Feb 2013 17:17:11 -0500 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <51196B08.4000002@myzel.net> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> <19gc77qsdov8@mids.svenhartge.de> <5118DC7B.3040509@Media-Brokers.com> <20130211164415.GA1488@PC211.ikt.de> <51196B08.4000002@myzel.net> Message-ID: <51196DE7.9060907@Media-Brokers.com> On 2013-02-11 5:04 PM, Hungerburg wrote: > Am 2013-02-11 17:44, schrieb Dennis Guhl: >> On Mon, Feb 11, 2013 at 06:56:43AM -0500, Charles Marcus wrote: >>> On 2013-02-10 8:43 PM, Sven Hartge wrote: >>> >>> quota_grace >> >> Sounds good for me. In particular the similarity to the disk quotas >> grace period is blatant. > > Not a native english speaker, but quota_grace is very concise and > resonates in a way, that makes it easily rememberable: the quota is > not enforced in strictness, but handled gracefully. The server for > once turns a blind eye to the quota. > > Documentation: quota_grace - specify an amount in that a > single incoming message may exceed the quota without being rejected. > Useful in situations, where ... > > *Alternatively* it could also be a binary option: Only start enforcing > the quota after the fact, when it is already exceeded. True, most systems have limits on the size of an email they will accept, which in and of itself would serve as the 'quota_grace' amount... From daniel.parthey at informatik.tu-chemnitz.de Tue Feb 12 01:35:33 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Tue, 12 Feb 2013 00:35:33 +0100 Subject: [Dovecot] Migration to Gmail ... In-Reply-To: <511923B3.6010607@esiee.fr> References: <511923B3.6010607@esiee.fr> Message-ID: <20130211233533.GA10124@daniel.localdomain> Frank Bonnet wrote: > Anyone has migrated Dovecot IMAP only mailboxes server to GMAIL ? Try https://github.com/imapsync/imapsync Regards Daniel From rs at sys4.de Tue Feb 12 09:30:39 2013 From: rs at sys4.de (Robert Schetterer) Date: Tue, 12 Feb 2013 08:30:39 +0100 Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: References: Message-ID: <5119EF9F.9060000@sys4.de> Am 11.02.2013 21:34, schrieb David Mehler: > Hello, > > I'm having to migrate servers. Both are dovecot2 systems. I'm wanting > to copy over my mail store from one system to the other. I'd like to > preserve dates/times of emails. These are maildir setups on both > boxes, I'd like to be as transparent to the end user as possible. I > currently have the first dovecot system offline and the second dovecot > system is offline. > > Recommendations? > > Thanks. > Dave. > try rsync, also dsync or imapsync may help Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From skdovecot at smail.inf.fh-brs.de Tue Feb 12 10:35:13 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 12 Feb 2013 09:35:13 +0100 (CET) Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: <5119663C.8030508@thelounge.net> References: <511958FD.8040205@thelounge.net> <20130211163741.3bd8e49c@mydesk> <5119663C.8030508@thelounge.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 11 Feb 2013, Reindl Harald wrote: > Am 11.02.2013 22:37, schrieb Steve Litt: >> On Mon, 11 Feb 2013 21:47:57 +0100 >> Reindl Harald wrote: >>> Am 11.02.2013 21:34, schrieb David Mehler: >>>> I'm having to migrate servers. Both are dovecot2 systems. I'm >>>> wanting to copy over my mail store from one system to the other. >>>> I'd like to preserve dates/times of emails. These are maildir >>>> setups on both boxes, I'd like to be as transparent to the end user >>>> as possible. I currently have the first dovecot system offline and >>>> the second dovecot system is offline. >>> >>> assuming that BOTH servers are down as fro any other transfers >>> like mysql-datadirs and any critical things which should go >>> save and fast by preserve attributes: >>> >>> rsync --force --delete-after -tPrlHpogEAXz /folder/ >>> root at newserver:/folder/ >>> >>> # -z compress >>> # -t timestamps >>> # -P progress >>> # -r recursive >>> # -l links >>> # -H hard-links >>> # -p permissions >>> # -o owner >>> # -g group >>> # -E executability >>> # -A acls >>> # -X xtended attributes >> >> That's how I'd do it also. I think if you didn't mind including -D, >> which from my understanding is --special and --devices, neither of >> which I'd expect in a maildir, and if you wanted to do your deletion >> manually after the fact, would this be equivalent? > > well, i have them ususally in my "rsync.sh" but did not expect > such files in a maildir, but yes it does not hurt > >> rsync -PaHAX /folder/ >> >> The man page says -a (--archive) is equivalent to -rlptgoD, and you >> have all but D listed up there. >> >> My thinking on later manually deleting the old maildir, instead of >> letting rsync do it, is that if somehow, some weird thing goes wrong, I >> have the old one for backup. Who knows, maybe I copied the thing on top >> of the wrong other maildir and have to back it out -- I'd have the old >> maildir as a reference of which files. > > you missunderstand "--delete-after" > > this is for delete files in the TARGET folder which is not > or no longer in the source and IMHO very very important > if you want sync folders 1:1 because old artefacts can have > very bad effects > > without you merge folders and if i know "hey my source contains > exactly what i need, not more and lot less" this is not what i > would like and never did in 10 years IT If you use "--delete-after" you could add "--fuzzy", which theoretically could save transfers because of filename renames. Anyway: Any form of "--delete" is required for Maildir, IMHO, because Labels, Tags, Keywords (or whatever the MUA calls it) and status information (seen, read, deleted) are reflected by the filename. Hence, if one does not "--delete", the _same_ message might pop up in the Maildir multiple times but with different status and/or tags. Same applies to new messages, because they are storred in /new and later moved to /cur. So one ends with the same message in "new" and in "cur". - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURn+wV3r2wJMiz2NAQJYnwgAwGfCHBFglm800nbaHFhsdWeuVkhGJjnW l0QtbmQLaSdQOkWwdVkmNEQkJLmwvfhddG0a9GblofI/zn8EBN+3EDR/CM0d5Y8f jy1yqemT26sFTVC6NlTqhDF8zq/Oi5WK7ftUfYfrt90MtmMD1rzmBQ5Q/N+Tteae TeE0jeHNL0rHl9DoAV9AkBzhJPnYYodSK2lA1oewazlzRpzJSiHYmgVh4RiSmsyk 4DAtch5ZrqLnsx1A/mUHAfDvdk66j2Os0bLXqbHu9ZAzz5Xb9bbNfpu52u1Ukg/o TfXfiGA4aRix6I1OGePdXmi/qXTzPVLBzhhIZdMiJ2CKCWNFbIYC7w== =ugv1 -----END PGP SIGNATURE----- From karol.jurak at gmail.com Tue Feb 12 12:19:56 2013 From: karol.jurak at gmail.com (Karol Jurak) Date: Tue, 12 Feb 2013 11:19:56 +0100 Subject: [Dovecot] dsync 2.1.10 does not correctly handle renames of mailbox subtrees Message-ID: Hi, It seems that dsync 2.1.10 does not correctly handle renames of mailbox subtrees. The scenario is as follows. There are two servers, A and B, both running dovecot 2.1.10 and configured to use mdbox. Automatic replication between them is disabled, ie. replication plugin is not active and replicator service has process_min_avail = 0. I perform the following actions: 1. On A: doveadm mailbox create -u test_user a a/b a/b/c d 2. On A: doveadm sync -u test_user -d The result is as expected: on both A and B there are a/b/c and d mailboxes. Now I connect to A via IMAP and rename a/b to d/b. The result is as expected: on A there are a and d/b/c mailboxes. Next I execute on A 'doveadm sync -u mailstoragetest -d' and now the result is not as expected: there are a/b/c and d/b subtrees on both A and B and a/b is just a directory, ie. it does not contain dbox-Mails (the rest are proper mailboxes). dovecot.mailbox.logs have the following records appended to them: A: #2112: rename 33259783092361d18c60ba1c7cfe973c (2013-02-12 10:42:55) - this probably is the rename of a/b to d/b (recursive, ie. including children - in this case a/b/c => d/b/c) by the IMAP client #2136: rename ee2be6da91ae879476b152d7ca041f1b (1970-01-01 01:00:00) - rename of d/b/c to a/b/c by dsync (unexpected/incorrect) B: #1104: rename 33259783092361d18c60ba1c7cfe973c (1970-01-01 01:00:00) - rename of a/b to d/b (non recursive) by dsync What seems to be missing is the record in A's dovecot.mailbox.log describing the rename of a/b/c to d/b/c (or in general case the set of records describing the renames of all the children of a/b). Would appending such records allow dsync to make correct decisions as to which mailboxes and on which servers to rename? Thanks for any help. # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.1 auth_master_user_separator = * auth_username_format = %Ln default_vsz_limit = 2 G dotlock_use_excl = no dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l5 log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_access_groups = mail mail_location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n mail_plugins = acl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile } protocols = imap lmtp pop3 service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service auth { executable = /usr/lib/dovecot/auth unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } unix_listener auth-userdb { mode = 0666 } user = root } service dict { unix_listener dict { mode = 0666 } } service doveadm { process_min_avail = 1 service_count = 1024 } service imap-login { chroot = login executable = /usr/lib/dovecot/imap-login inet_listener imap { address = * port = 143 } service_count = 0 user = dovecot } service imap { executable = /usr/lib/dovecot/imap } service lmtp { inet_listener lmtp { port = 24 } process_limit = 48 } service pop3-login { chroot = login executable = /usr/lib/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } service_count = 0 user = dovecot } service pop3 { executable = /usr/lib/dovecot/pop3 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = acl } protocol lmtp { mail_plugins = acl } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = acl } -- Karol Jurak From alessio at skye.it Tue Feb 12 12:34:48 2013 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 12 Feb 2013 11:34:48 +0100 Subject: [Dovecot] Dict quota timeout error: Has anything changed in the latest versions? Message-ID: <511A1AC8.3070506@skye.it> Hi, I'm using dict quota with mysql for track them via SQL query instead of read maildirsize file. Primary quota and enforcing is provided via userdb and is "maildir" type. Since dovecot 2.1.10-13 (but perhaps even before) when a new user log in via POP/IMAP or receive an email for the first time dovecot created the entry for the users in the table "quota" of dovecot database with quota usage and number of messages (before of this login/delivery the user has no entry in the quota table of dovecot DB). After the update to 2.1.14 when a new user login or receive an email for the first time dovecot was unable to create (immediately) the entry in quota table and in the log print these errors: Feb 12 11:03:49 master: Info: Dovecot v2.1.15 starting up (core dumps disabled) Feb 12 11:04:02 auth-worker(31421): Info: mysql(localhost): Connected to database vpopmail Feb 12 11:04:02 pop3-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=31422, secured, session= Feb 12 11:04:08 pop3(alessio at domain.com): Info: Disconnected: Logged out top=0/0, retr=0/0, del=0/9, size=18618 Feb 12 11:04:41 pop3-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=31424, secured, session= Feb 12 11:05:11 pop3(alessio at domain.com): Info: Disconnected: Logged out top=0/0, retr=0/0, del=1/9, size=18618 Feb 12 11:05:11 dict: Info: mysql(10.10.3.19): Connected to database dovecot [ here is when I "quit" from telnet session] Feb 12 11:05:41 pop3(alessio at domain.com): Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds Feb 12 11:06:11 pop3(alessio at domain.com): Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds Feb 12 11:06:41 pop3(alessio at domain.com): Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds Feb 12 11:07:11 pop3(alessio at domain.com): Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds Feb 12 11:07:41 pop3(alessio at domain.com): Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds Feb 12 11:08:11 pop3(alessio at domain.com): Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds Feb 12 11:08:41 pop3(alessio at domain.com): Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds [ only now telnet close the session ] This is the telnet session: # telnet 0 110 Trying 0.0.0.0... Connected to 0. Escape character is '^]'. +OK Dovecot ready. user alessio at domain.com +OK pass ciao +OK Logged in. list +OK 8 messages: 1 1594 2 1594 3 1594 4 1594 5 1594 6 1594 7 3755 8 3714 . dele 8 +OK Marked to be deleted. quit [ here dovecot wait about 3 minuts before close the connecctions ] +OK Logging out, messages deleted. Connection closed by foreign host. After 3 minutes of wait in the mysql database dovecot insert the entry for the new users: mysql> use dovecot Database changed mysql> select * from quota where username='alessio at domain.com'; +--------------------------+---------+----------+ | username | bytes | messages | +--------------------------+---------+----------+ | alessio at domain.com | 2144030 | 15 | +--------------------------+---------+----------+ Errors like Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after 30 seconds were present also before the upgrade but they were occasional, now I had to disable "dict quota". Can this be fixed? My dovecot configuration: # cat /usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext connect = host=10.10.3.19 dbname=dovecot user=dovecot password=XXX map { pattern = priv/quota/storage table = quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = username value_field = messages } map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire_stamp fields { username = $user mailbox = $mailbox } } # dovecot -n # 2.1.15: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf sqlquota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl zlib maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /usr/local/dovecot-2.1/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User usage::proxy::sqlquota quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } service quota-warning { executable = script /usr/local/dovecot-2.1/etc/dovecot/scripts/quota-warning.sh unix_listener quota-warning { user = vpopmail } user = vpopmail } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From aalberola at gtt.es Tue Feb 12 12:36:02 2013 From: aalberola at gtt.es (Antonio Alberola) Date: Tue, 12 Feb 2013 11:36:02 +0100 Subject: [Dovecot] Authentication with many Windows AD Message-ID: <004a01ce090c$c16bad40$444307c0$@gtt.es> Dear All, I'm a new user of Dovecot and I need support with the authentication process. I use Dovecot 2.0.9 and Postfix 2.6.6 on host x86_64-redhat-linux-gnu 2.6.32-279.14.1.el6.x86_64 CentOS release 6.3 (Final) I need to select the validation method depending on the user. I have users that have to be authenticate with UNIX account, and I have others thats have to be authenticate with Windows AD. For example, some sysadmins and special accounts needs validate with Unix accounts. Other normal users have to authenticate with Windows AD, I have many Windows controllers. Some users authenticate to a specific Windows AD and other users are validated in another one. How I can do this? Thank you very much. Antonio Alberola From amateo at um.es Tue Feb 12 14:56:47 2013 From: amateo at um.es (Angel L. Mateo) Date: Tue, 12 Feb 2013 13:56:47 +0100 Subject: [Dovecot] I can't sync with dsync and imapc Message-ID: <511A3C0F.8080701@um.es> Hello, I am deploying a new dovecot server to replace and old one, both of the with dovecot 2.1.9. Now I want to synchronize imap mailbox from the old to the new one. So, in my new server I run (whith a $ sudo dsync -v -D -o imapc_host="" -o imapc_password="" -o imapc_features=rfc822.size backup -R -f -u myuser at um.es imapc: doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_backend_rescan (this is usually intentional, so just ignore this message) doveadm(myuser at um.es): Debug: Added userdb setting: plugin/quota_rule=*:storage=10G doveadm(myuser at um.es): Debug: Effective uid=2012, gid=204, home=/home/buzones/b1/an/myuser doveadm(myuser at um.es): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled doveadm(myuser at um.es): Debug: Quota root: name=User quota backend=dict args=:file:/home/buzones/b1/an/myuser/quota doveadm(myuser at um.es): Debug: Quota rule: root=User quota mailbox=* bytes=10737418240 messages=0 doveadm(myuser at um.es): Debug: Quota rule: root=User quota mailbox=Trash bytes=+1073741824 messages=0 doveadm(myuser at um.es): Debug: dict quota: user=myuser at um.es, uri=file:/home/buzones/b1/an/myuser/quota, noenforcing=0 doveadm(myuser at um.es): Debug: Namespace : type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/home/buzones/b1/an/myuser/mdbox:INDEX=/home/indexes/i1/an/myuser:ALT=/home/alt/a1/an/myuser doveadm(myuser at um.es): Debug: fs: root=/home/buzones/b1/an/myuser/mdbox, index=/home/indexes/i1/an/myuser, control=, inbox=, alt=/home/alt/a1/an/myuser doveadm(myuser at um.es): Debug: Namespace : Using permissions from /home/buzones/b1/an/myuser/mdbox: mode=0700 gid=-1 doveadm(myuser at um.es): Debug: acl: initializing backend with data: vfile doveadm(myuser at um.es): Debug: acl: acl username = myuser at um.es doveadm(myuser at um.es): Debug: acl: owner = 1 doveadm(myuser at um.es): Debug: acl vfile: Global ACL directory: (none) doveadm(myuser at um.es): Debug: Namespace : type=private, prefix=BORRADOS., sep=., inbox=no, hidden=yes, list=no, subscriptions=yes location=mdbox:/home/buzones/b1/an/myuser/expunged doveadm(myuser at um.es): Debug: fs: root=/home/buzones/b1/an/myuser/expunged, index=, control=, inbox=, alt= doveadm(myuser at um.es): Debug: Namespace BORRADOS.: Using permissions from /home/buzones/b1/an/myuser/expunged: mode=0700 gid=-1 doveadm(myuser at um.es): Debug: acl: initializing backend with data: vfile doveadm(myuser at um.es): Debug: acl: acl username = myuser at um.es doveadm(myuser at um.es): Debug: acl: owner = 1 doveadm(myuser at um.es): Debug: acl vfile: Global ACL directory: (none) dsync(myuser at um.es): Debug: Effective uid=2012, gid=204, home=/home/buzones/b1/an/myuser dsync(myuser at um.es): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled dsync(myuser at um.es): Debug: Quota root: name=User quota backend=dict args=:file:/home/buzones/b1/an/myuser/quota dsync(myuser at um.es): Debug: Quota rule: root=User quota mailbox=* bytes=10737418240 messages=0 dsync(myuser at um.es): Debug: Quota rule: root=User quota mailbox=Trash bytes=+1073741824 messages=0 dsync(myuser at um.es): Debug: dict quota: user=myuser at um.es, uri=file:/home/buzones/b1/an/myuser/quota, noenforcing=0 dsync(myuser at um.es): Debug: Namespace : type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=imapc: dsync(myuser at um.es): Debug: imapc: root=, index=, control=, inbox=, alt= dsync(myuser at um.es): Debug: imapc(myotis30.um.es:143): Looking up IP address dsync(myuser at um.es): Debug: imapc(myotis30.um.es:143): Connecting to 155.54.211.169:143 dsync(myuser at um.es): Debug: imapc(myotis30.um.es:143): Server capabilities: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN dsync(myuser at um.es): Debug: imapc(myotis30.um.es:143): Authenticating as master for user myuser at um.es dsync(myuser at um.es): Debug: imapc(myotis30.um.es:143): Server capabilities: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE QUOTA ACL RIGHTS=texk dsync(myuser at um.es): Debug: imapc(myotis30.um.es:143): Authenticated successfully dsync(myuser at um.es): Debug: acl: initializing backend with data: vfile dsync(myuser at um.es): Debug: acl: acl username = myuser at um.es dsync(myuser at um.es): Debug: acl: owner = 1 dsync(myuser at um.es): Debug: acl vfile: Global ACL directory: (none) dsync(myuser at um.es): Debug: Namespace : type=private, prefix=BORRADOS., sep=., inbox=no, hidden=yes, list=no, subscriptions=yes location=mdbox:/home/buzones/b1/an/myuser/expunged dsync(myuser at um.es): Debug: fs: root=/home/buzones/b1/an/myuser/expunged, index=, control=, inbox=, alt= dsync(myuser at um.es): Debug: Namespace BORRADOS.: Using permissions from /home/buzones/b1/an/myuser/expunged: mode=0700 gid=-1 dsync(myuser at um.es): Debug: acl: initializing backend with data: vfile dsync(myuser at um.es): Debug: acl: acl username = myuser at um.es dsync(myuser at um.es): Debug: acl: owner = 1 dsync(myuser at um.es): Debug: acl vfile: Global ACL directory: (none) dsync(myuser at um.es): Debug: acl vfile: file /home/buzones/b1/an/myuser/mdbox/mailboxes/dovecot-acl not found dsync(myuser at um.es): Error: Failed to sync mailbox INBOX: Internal error occurred. Refer to server log for more information. [2013-02-12 13:52:29] dsync(myuser at um.es): Debug: acl vfile: file /home/buzones/b1/an/myuser/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-acl not found dsync(myuser at um.es): Error: Can't delete mailbox INBOX: INBOX can't be deleted. dsync(myuser at um.es): Info: INBOX: only in dest (guid=edd76012063b1a5138270000a9061514) dsync(myuser at um.es): Error: Trying to open a non-listed mailbox with guid=edd76012063b1a5138270000a9061514 dsync(myuser at um.es): Error: msg iteration failed: Couldn't open mailbox edd76012063b1a5138270000a9061514 dsync(myuser at um.es): Error: Trying to open a non-listed mailbox with guid=edd76012063b1a5138270000a9061514 dsync(myuser at um.es): Debug: imapc(myotis30.um.es:143): Disconnected In the old server I get the error: Feb 12 13:52:29 myotis30 dovecot: imap(myuser at um.es): Error: Failed to autocreate mailbox INBOX: Permission denied Feb 12 13:52:29 myotis30 dovecot: imap(myuser at um.es): Debug: acl: Mailbox not in dovecot-acl-list: Archives Feb 12 13:52:29 myotis30 dovecot: imap(myuser at um.es): Debug: acl: Mailbox not in dovecot-acl-list: Archives.2012 Feb 12 13:52:29 myotis30 dovecot: imap(myuser at um.es): Debug: acl: Mailbox not in dovecot-acl-list: Sent Items So it seems dsync is trying to create a mailbox in the original server. What am I doing wrong? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From dave.mehler at gmail.com Tue Feb 12 18:06:49 2013 From: dave.mehler at gmail.com (David Mehler) Date: Tue, 12 Feb 2013 11:06:49 -0500 Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: References: <511958FD.8040205@thelounge.net> <20130211163741.3bd8e49c@mydesk> <5119663C.8030508@thelounge.net> Message-ID: Hello, Thanks, if I did: rsync --force --delete-after -tPrlHpogEAXz /folder/ /backups/newmailstore/ could I then tar up the newmailstore folder with something like tar zcf without messing anything up? Thanks. Dave. On 2/12/13, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mon, 11 Feb 2013, Reindl Harald wrote: > >> Am 11.02.2013 22:37, schrieb Steve Litt: >>> On Mon, 11 Feb 2013 21:47:57 +0100 >>> Reindl Harald wrote: >>>> Am 11.02.2013 21:34, schrieb David Mehler: >>>>> I'm having to migrate servers. Both are dovecot2 systems. I'm >>>>> wanting to copy over my mail store from one system to the other. >>>>> I'd like to preserve dates/times of emails. These are maildir >>>>> setups on both boxes, I'd like to be as transparent to the end user >>>>> as possible. I currently have the first dovecot system offline and >>>>> the second dovecot system is offline. >>>> >>>> assuming that BOTH servers are down as fro any other transfers >>>> like mysql-datadirs and any critical things which should go >>>> save and fast by preserve attributes: >>>> >>>> rsync --force --delete-after -tPrlHpogEAXz /folder/ >>>> root at newserver:/folder/ >>>> >>>> # -z compress >>>> # -t timestamps >>>> # -P progress >>>> # -r recursive >>>> # -l links >>>> # -H hard-links >>>> # -p permissions >>>> # -o owner >>>> # -g group >>>> # -E executability >>>> # -A acls >>>> # -X xtended attributes >>> >>> That's how I'd do it also. I think if you didn't mind including -D, >>> which from my understanding is --special and --devices, neither of >>> which I'd expect in a maildir, and if you wanted to do your deletion >>> manually after the fact, would this be equivalent? >> >> well, i have them ususally in my "rsync.sh" but did not expect >> such files in a maildir, but yes it does not hurt >> >>> rsync -PaHAX /folder/ >>> >>> The man page says -a (--archive) is equivalent to -rlptgoD, and you >>> have all but D listed up there. >>> >>> My thinking on later manually deleting the old maildir, instead of >>> letting rsync do it, is that if somehow, some weird thing goes wrong, I >>> have the old one for backup. Who knows, maybe I copied the thing on top >>> of the wrong other maildir and have to back it out -- I'd have the old >>> maildir as a reference of which files. >> >> you missunderstand "--delete-after" >> >> this is for delete files in the TARGET folder which is not >> or no longer in the source and IMHO very very important >> if you want sync folders 1:1 because old artefacts can have >> very bad effects >> >> without you merge folders and if i know "hey my source contains >> exactly what i need, not more and lot less" this is not what i >> would like and never did in 10 years IT > > If you use "--delete-after" you could add "--fuzzy", which theoretically > could save transfers because of filename renames. > > Anyway: Any form of "--delete" is required for Maildir, IMHO, because > Labels, Tags, Keywords (or whatever the MUA calls it) and status > information (seen, read, deleted) are reflected by the filename. Hence, if > one does not "--delete", the _same_ message might pop up in the Maildir > multiple times but with different status and/or tags. Same applies to new > messages, because they are storred in /new and later moved to > /cur. So one ends with the same message in "new" and in "cur". > > - -- > Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iQEVAwUBURn+wV3r2wJMiz2NAQJYnwgAwGfCHBFglm800nbaHFhsdWeuVkhGJjnW > l0QtbmQLaSdQOkWwdVkmNEQkJLmwvfhddG0a9GblofI/zn8EBN+3EDR/CM0d5Y8f > jy1yqemT26sFTVC6NlTqhDF8zq/Oi5WK7ftUfYfrt90MtmMD1rzmBQ5Q/N+Tteae > TeE0jeHNL0rHl9DoAV9AkBzhJPnYYodSK2lA1oewazlzRpzJSiHYmgVh4RiSmsyk > 4DAtch5ZrqLnsx1A/mUHAfDvdk66j2Os0bLXqbHu9ZAzz5Xb9bbNfpu52u1Ukg/o > TfXfiGA4aRix6I1OGePdXmi/qXTzPVLBzhhIZdMiJ2CKCWNFbIYC7w== > =ugv1 > -----END PGP SIGNATURE----- > From AxelLuttgens at swing.be Tue Feb 12 19:39:38 2013 From: AxelLuttgens at swing.be (Axel Luttgens) Date: Tue, 12 Feb 2013 18:39:38 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <1360545405.10326.82.camel@innu> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1360545405.10326.82.camel@innu> Message-ID: <29403D1D-EDCC-40F0-B225-91C54998C228@swing.be> Le 11 f?vr. 2013 ? 02:16, Timo Sirainen a ?crit : > [...] > > I wonder if there's a better name for this than "last_extra".. Hello Timo, If I've correctly understood, the idea is to accept an additional message provided there is some free space AND the size of that message additional doesn't exceed above last_extra. An additional message, even if over-quota strictly speaking, might thus be tolerated. Or the idea of an absolute quota may thus come with some tolerance. quota_tolerance? Axel From bam at miranda.org Tue Feb 12 22:28:56 2013 From: bam at miranda.org (Bret Martin) Date: Tue, 12 Feb 2013 15:28:56 -0500 Subject: [Dovecot] Marking messages read / retaining date with Sieve In-Reply-To: <20130131013939.GB25874@anubis.morrow.me.uk> References: <20130130235625.GA25874@anubis.morrow.me.uk> <5E50AF20-896A-40C1-ADB6-F39003E56726@miranda.org> <20130131013939.GB25874@anubis.morrow.me.uk> Message-ID: <6EB4EC5E-8692-4FE9-A20F-FF87CA28BA20@miranda.org> On Jan 30, 2013, at 8:39 PM, Ben Morrow wrote: [...] > Mail::IMAPClient (which I usually prefer) will work with a > preauthenticated socket, but you need to create a socketpair explicitly, > fork and exec dovecot/imap with one end of the pair on STDIN/STDOUT, > then pass the other end to Mail::IMAPClient->new as the Socket > parameter. Something like this (I've left out error checking) [...] Thanks again for your help! To close the loop on this, I ended up doing the following, although it seems to fail on mailboxes with large numbers of messages (on the order of 10,000 or so; I didn't test carefully enough to find the exact number) use Mail::IMAPClient; use Socket; use strict; socketpair( my $dovecot, my $client, AF_UNIX, SOCK_STREAM, PF_UNSPEC ); unless ( fork() ) { open( STDIN, '<&', $client ); open( STDOUT, '>&', $client ); exec( '/usr/lib/dovecot/imap' ); } close( $client ); my $imap = Mail::IMAPClient->new( Socket => $dovecot ); foreach my $folder( sort $imap->folders() ) { print( "$folder\n" ); $imap->select( $folder ); $imap->set_flag( 'Seen', $imap->search( 'ALL' ) ); } For the moment the failure on large folders was easier to handle by just doing those folders manually, although I'm curious if anyone knows the reason that might fail. --Bret From JLock at csolve.net Tue Feb 12 23:25:15 2013 From: JLock at csolve.net (Jason Lock) Date: Tue, 12 Feb 2013 21:25:15 +0000 Subject: [Dovecot] FreeBSD, Dovecot and ZFS Message-ID: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> Mail Issues - FreeBSD Hello, my apologies if this may be the wrong forum but hoping that maybe someone might be able to provide some insight. Have a very sporadic and strange issue with our mail servers running Dovecot on FreeBSD. There are three servers hosting Dovecot with FreeBSD as the underlying operating system. All three connect to a NAS server, again running FreeBSD and ZFS. When the specific issue occurs, clients connecting to check mail via POP3 or IMAP experience long delays and timeouts. To the point where POP3 Logins fail due to the timeouts. The issue is further compounded by clients increasing the number of attempts to check mail. Part of the frustration in attempting to diagnose the issue is not knowing the root cause or symptom that initates the issue. Wondering if anyone has experienced anything similar, or suggestions on ways to help identify the root cause From jtam.home at gmail.com Tue Feb 12 23:49:50 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 12 Feb 2013 13:49:50 -0800 (PST) Subject: [Dovecot] Imap process crash: assertion failed: (full_fs_access) In-Reply-To: References: Message-ID: On Mon, 11 Feb 2013, Joseph Tam wrote: > Timo Sirainen writes: > >> > Any guess at what would cause this? >> > > Feb 7 21:20:53 server dovecot: imap(user): Panic: file >> > mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion >> > failed: (full_fs_access) >> >> LIST command that had invalid parameters that somehow got through the >> first validity check. What namespace config do you have? > > I got more info from the user that had this problem. He reported that he > was was setting up a new Email client (Nexus4 Android) and he sent me a > screenshot of his setup. Everything look ordinary except perhaps the > IMAP prefix set to "mail". > ... > I've asked the user to change his prefix, and he'll probably report back. He reported that removing the mail prefix solved it for him. I then proceeded to update dovecot from 2.1.14 -> 2.1.15, and asked him to change it back to the previoux prefix ("mail"). That also worked. So I don't know what unwedged him, but it's working now. Joseph Tam From CMarcus at Media-Brokers.com Wed Feb 13 00:08:13 2013 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 12 Feb 2013 17:08:13 -0500 Subject: [Dovecot] FreeBSD, Dovecot and ZFS In-Reply-To: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> References: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> Message-ID: <511ABD4D.2010707@Media-Brokers.com> On 2013-02-12 4:25 PM, Jason Lock wrote: > When the specific issue occurs, clients connecting to check mail via POP3 or IMAP experience long delays and timeouts. To the point where POP3 Logins fail due to the timeouts. The issue is further compounded by clients increasing the number of attempts to check mail. WAG would be similar issues you can face when using NFS with multiple servers accessing it (file locking issues). The solution would be to use Director to make sure users are always directed to the same server. http://wiki2.dovecot.org/Director If that isn't the problem, then much more info would be needed (ie, doveconf -n output, logs exhibiting the problem, etc)... -- Best regards, */Charles /* From dovecot.org at practiceofcode.com Wed Feb 13 01:12:51 2013 From: dovecot.org at practiceofcode.com (Jason Discount) Date: Wed, 13 Feb 2013 09:12:51 +1000 Subject: [Dovecot] Is there any way to mask or hide the INBOX label for Dovecot 1.2.x Shared Mailboxes? Message-ID: <802F6258-DC15-43A1-8ED0-CA5FA09C4AA7@practiceofcode.com> Hello, I'm setting up a new Dovecot sever to replace an old Cyrus server, and need to keep things as consistent as possible. I have a couple local accounts, like junkmail and notjunkmail that users get shared access to, and can drop things in to train sa-update. However, when users share over junkmail's inbox, it's displayed in the hierarchy: ? Other Users ? junkmail ? INBOX Is there any way to get junkmail's INBOX to just show up at Other Users/junkmail and not require that extra depth? ? Other Users ? junkmail I'm running Debian Squeeze and the standard dovecot package available therein: > # uname -a > Linux internet.digitalquay.com.au 2.6.32-5-xen-amd64 #1 SMP Sun Sep 23 13:49:30 UTC 2012 x86_64 GNU/Linux > #dovecot --version > 1.2.15 I'm running in a configuration where all mail is owned by the vmail user, due to the impression I'm under, that it's a better way if there is to be shared mailboxes. Relevant bits of dovecot -n: > mail_uid: vmail > mail_gid: vmail > mail_location: maildir:/var/spool/dovecot/user/%u > mail_plugins: acl imap_acl > namespace: > type: private > separator: / > inbox: yes > list: yes > subscriptions: yes > namespace: > type: shared > separator: / > prefix: Other Users/%%u/ > location: maildir:/var/spool/dovecot/user/%%u:INDEX="/var/spool/dovecot/user/%u/Other\ Users/%%u" > list: children > auth default: > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: passwd > args: uid=vmail gid=vmail blocking=yes mail=maildir:/var/spool/dovecot/user/%u Am I barking up the wrong tree, or is this possible? Thanks, Jay From stephan at rename-it.nl Wed Feb 13 01:27:54 2013 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 13 Feb 2013 00:27:54 +0100 Subject: [Dovecot] sieve /editheader / addheader :last crash In-Reply-To: <5116A18A.2040103@wzb.eu> References: <5114C435.4040105@wzb.eu> <5116087B.2070203@rename-it.nl> <5116A18A.2040103@wzb.eu> Message-ID: <511ACFFA.8000408@rename-it.nl> On 2/9/2013 8:20 PM, Peter Rindfuss wrote: > Am 2013-02-09 09:27, schrieb Stephan Bosch: >> >> I'm unable to reproduce this so far. Could you show me more details of >> your setup? >> >> * Output from: dovecot -n >> * Your exact test message >> * Your exact test Sieve script >> * The sieve-test command line >> > > Hi Stephan, > > attached are "dovecot -n", a test message, and three very similar > scripts one of which causes the crash and two not. > > I'm inclined to say that the message itself does not matter. > > This ist the sieve-test cmd line: > sieve-test -t - -a 'brot at wzb.eu' -l /wzb/mail/brot/Maildir -e -x > +editheader /wzb/mail/brot/sieve/{bad|good1|good2}.sieve mail > > Initially I encountered the problem in real deliveries, not in > sieve-test. It is always reproducible. I will take another look at this in a few days. Regards, Stephan. From kwebb at teradactyl.com Wed Feb 13 01:29:38 2013 From: kwebb at teradactyl.com (Kristen J. Webb) Date: Tue, 12 Feb 2013 16:29:38 -0700 Subject: [Dovecot] Converting a POP3 client to IMAP Message-ID: <511AD062.3020600@teradactyl.com> I have a dovecot 2.0 dovecot server set up for https/imap with spam, av, and some preliminary sieve filters in place. Our current email is outsourced and so we have used POP3 to keep all email on our systems for many years. I have run tests to move mail to the new imap server and that seems to work pretty well. However, we have lots of mail going back over a decade and used a lot of aliases that forwarded email to many users. I find the moving mail multiple times using Thunderbird results in multiple copies of the same email. Is there a best practice for moving mail such that we can take advantage of the hard link feature for all those older copies that are currently in POP3 files? It would also be nice to be able to auto file things. I don't see the sieve filters being activated when I copy mail in Thunderbird. Many thanks in advance for any guidance. Kris -- This message is NOT encrypted -------------------------------- Mr. Kristen J. Webb Chief Technology Officer Teradactyl LLC. 2450 Baylor Dr. S.E. Albuquerque, New Mexico 87106 Phone: 1-505-338-6000 Email: kwebb at teradactyl.com Web: http://www.teradactyl.com Providers of Scalable Backup Solutions for Unique Data Environments -------------------------------- NOTICE TO RECIPIENTS: Any information contained in or attached to this message is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this transmittal, you are hereby notified that you received this transmittal in error, and we request that you please delete and destroy all copies and attachments in your possession, notify the sender that you have received this communication in error, and note that any review or dissemination of, or the taking of any action in reliance on, this communication is expressly prohibited. Regular internet e-mail transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate, and it should not be relied upon as such. If you prefer to communicate with Teradactyl LLC. using secure (i.e., encrypted and/or digitally signed) e-mail transmission, please notify the sender. Otherwise, you will be deemed to have consented to communicate with Teradactyl via regular internet e-mail transmission. Please note that Teradactyl reserves the right to intercept, monitor, and retain all e-mail messages (including secure e-mail messages) sent to or from its systems as permitted by applicable law. From ben at morrow.me.uk Wed Feb 13 02:24:02 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Wed, 13 Feb 2013 00:24:02 +0000 Subject: [Dovecot] FreeBSD, Dovecot and ZFS In-Reply-To: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> References: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> Message-ID: <20130213002402.GB900@anubis.morrow.me.uk> At 9PM +0000 on 12/02/13 you (Jason Lock) wrote: > Mail Issues - FreeBSD > > Hello, my apologies if this may be the wrong forum but hoping that > maybe someone might be able to provide some insight. This may turn out to be something better addressed on freebsd-stable, but this is a perfectly good place to start. > Have a very sporadic and strange issue with our mail servers running > Dovecot on FreeBSD. There are three servers hosting Dovecot with > FreeBSD as the underlying operating system. All three connect to a > NAS server, again running FreeBSD and ZFS. Over NFS, I assume? What version, what mount options, and what type of authentication? What locking strategies is Dovecot using? Are there any suspicious messages in syslog on either machine? > When the specific issue occurs, clients connecting to check mail via > POP3 or IMAP experience long delays and timeouts. To the point where > POP3 Logins fail due to the timeouts. The issue is further compounded > by clients increasing the number of attempts to check mail. Are the delays happening before or after login? If you can provoke this and get a 'procstat -k' for the relevant dovecot process this might be helpful. If 'long' delays means several minutes, running something along the lines of 'procstat -k $(pgrep -U dovecot -U doveauth)' every minute or so for a while might be one way to catch this, though this will collect a lot of data rather fast so you will need some way to locate the relevant entry. Ben From ben at morrow.me.uk Wed Feb 13 02:40:19 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Wed, 13 Feb 2013 00:40:19 +0000 Subject: [Dovecot] Marking messages read / retaining date with Sieve In-Reply-To: <6EB4EC5E-8692-4FE9-A20F-FF87CA28BA20@miranda.org> References: <20130130235625.GA25874@anubis.morrow.me.uk> <5E50AF20-896A-40C1-ADB6-F39003E56726@miranda.org> <20130131013939.GB25874@anubis.morrow.me.uk> <6EB4EC5E-8692-4FE9-A20F-FF87CA28BA20@miranda.org> Message-ID: <20130213004018.GC900@anubis.morrow.me.uk> At 3PM -0500 on 12/02/13 you (Bret Martin) wrote: > > To close the loop on this, I ended up doing the following, although it > seems to fail on mailboxes with large numbers of messages (on the > order of 10,000 or so; I didn't test carefully enough to find the > exact number) > > use Mail::IMAPClient; > use Socket; > use strict; > > socketpair( my $dovecot, my $client, AF_UNIX, SOCK_STREAM, PF_UNSPEC ); > > unless ( fork() ) { > open( STDIN, '<&', $client ); > open( STDOUT, '>&', $client ); > exec( '/usr/lib/dovecot/imap' ); > } > close( $client ); > > my $imap = Mail::IMAPClient->new( Socket => $dovecot ); > > foreach my $folder( sort $imap->folders() ) { > print( "$folder\n" ); > $imap->select( $folder ); > $imap->set_flag( 'Seen', $imap->search( 'ALL' ) ); > } You're not specifying Ranges in the ->new call, and you're calling ->search in list context; either of these will force ->search to return a complete list of message numbers, which you are then trying to pass back in a single IMAP command. You want to either loop through the messages or set Ranges and call ->search in scalar context to get an (object which stringifies to an) IMAP range specification, which will be much shorter than the whole list. Ben From delphij at delphij.net Wed Feb 13 02:43:31 2013 From: delphij at delphij.net (Xin Li) Date: Tue, 12 Feb 2013 16:43:31 -0800 Subject: [Dovecot] FreeBSD, Dovecot and ZFS In-Reply-To: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> References: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> Message-ID: <511AE1B3.50801@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2/12/13 1:25 PM, Jason Lock wrote: > Mail Issues - FreeBSD > > Hello, my apologies if this may be the wrong forum but hoping that > maybe someone might be able to provide some insight. > > Have a very sporadic and strange issue with our mail servers > running Dovecot on FreeBSD. There are three servers hosting > Dovecot with FreeBSD as the underlying operating system. All three > connect to a NAS server, again running FreeBSD and ZFS. Speaking for NFS on FreeBSD, please note that the FreeBSD NFS client and server generally respects application's sync() operations which could cause delays, if the backing pool is not configured with any fast ZIL device, ideally a SSD. Normally a 'gstat -I 1s' on the NFS server will give you busy disks with small throughput, in this case, a ZIL would be helpful. Before buying a SSD, you may want to try disabling 'sync' on dataset (be careful though, that doing so would increase the chance of data loss in the event of power loss, make sure to revert the change with 'zfs inherit sync' on the datasets once you are done with experiments) and see if that helps, if so, then a SSD ZIL would be helpful; higher end systems can use a Fusion IO or ZeusRAM for that purpose. Another possible cause is compression, at this time, FreeBSD's ZFS compression thread runs at kernel priority and that would cause the system to do ZFS compression before doing everything else. If you use compression=gzip, you will see stalls more often, for short term, please consider using compression=lzjb or have dovecot to do the compression; for mid term, please use my recent LZ4 port that would consume way less CPU and have better compression ratio than lzjb; for long term, we are going to implement a scheduling policy which is similar to Solaris's System Duty Cycle and have the compatibility shims to emulate that. Lastly, try increasing your NFS iod numbers. The system default is very conservative and may become a bottleneck when you have a lot of users on one system. Hope this helps... If you have further questions about FreeBSD please consider posting to freebsd-stable at freebsd.org . Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJRGuGyAAoJEG80Jeu8UPuzWdYH/iybDczQnLzvlzKfrGn/Kdm7 197P2wK4phktNJVrNjYKFYc8CqelcLoSfZiRGDc3CxpmKnrwRkUWnZMXfpYUVJVL 3SSpmNY73h4atrEpyyoKmz4tIxJp7c24IB8201j/vc09yEaooy2dc9pJrJ5cAcjz d7eSidcQQk8VdqCBk8haDpB2igZZXWz8zgZ8mKPcqSkGuxZkzZwZCkZc7XfPsCGK GKswLtXutHA19mKs+wBLJ1r+ZQxabI8ZwQi1V2+VBFz4a5edkQJVbW6VvFSmzjO9 HfbW0uO+wJztSHiceKEUiJ7bIm5ygolhA2BvZ5WlY2VkejXbN6nfQkOlvmDeS0s= =sMYy -----END PGP SIGNATURE----- From rs at sys4.de Wed Feb 13 09:10:18 2013 From: rs at sys4.de (Robert Schetterer) Date: Wed, 13 Feb 2013 08:10:18 +0100 Subject: [Dovecot] FreeBSD, Dovecot and ZFS In-Reply-To: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> References: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> Message-ID: <511B3C5A.9030506@sys4.de> Am 12.02.2013 22:25, schrieb Jason Lock: > Mail Issues - FreeBSD > > Hello, my apologies if this may be the wrong forum but hoping that maybe someone might be able to provide some insight. > > Have a very sporadic and strange issue with our mail servers running Dovecot on FreeBSD. There are three servers hosting Dovecot with FreeBSD as the underlying operating system. All three connect to a NAS server, again running FreeBSD and ZFS. > > When the specific issue occurs, clients connecting to check mail via POP3 or IMAP experience long delays and timeouts. To the point where POP3 Logins fail due to the timeouts. The issue is further compounded by clients increasing the number of attempts to check mail. > > Part of the frustration in attempting to diagnose the issue is not knowing the root cause or symptom that initates the issue. > > Wondering if anyone has experienced anything similar, or suggestions on ways to help identify the root cause > show your debug logs, dovecot confs, nfs setups are special, what did you setup to honor nfs Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From skdovecot at smail.inf.fh-brs.de Wed Feb 13 09:20:21 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 13 Feb 2013 08:20:21 +0100 (CET) Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: References: <511958FD.8040205@thelounge.net> <20130211163741.3bd8e49c@mydesk> <5119663C.8030508@thelounge.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 12 Feb 2013, David Mehler wrote: > Thanks, if I did: > > rsync --force --delete-after -tPrlHpogEAXz /folder/ /backups/newmailstore/ > > could I then tar up the newmailstore folder with something like tar > zcf without messing anything up? keep in mind, that the backup of the Maildir with rsync is no 100% consistent snapshot, because of the filename renames; otherwise: yes. > On 2/12/13, Steffen Kaiser wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Mon, 11 Feb 2013, Reindl Harald wrote: >> >>> Am 11.02.2013 22:37, schrieb Steve Litt: >>>> On Mon, 11 Feb 2013 21:47:57 +0100 >>>> Reindl Harald wrote: >>>>> Am 11.02.2013 21:34, schrieb David Mehler: >>>>>> I'm having to migrate servers. Both are dovecot2 systems. I'm >>>>>> wanting to copy over my mail store from one system to the other. >>>>>> I'd like to preserve dates/times of emails. These are maildir >>>>>> setups on both boxes, I'd like to be as transparent to the end user >>>>>> as possible. I currently have the first dovecot system offline and >>>>>> the second dovecot system is offline. >>>>> >>>>> assuming that BOTH servers are down as fro any other transfers >>>>> like mysql-datadirs and any critical things which should go >>>>> save and fast by preserve attributes: >>>>> >>>>> rsync --force --delete-after -tPrlHpogEAXz /folder/ >>>>> root at newserver:/folder/ >>>>> >>>>> # -z compress >>>>> # -t timestamps >>>>> # -P progress >>>>> # -r recursive >>>>> # -l links >>>>> # -H hard-links >>>>> # -p permissions >>>>> # -o owner >>>>> # -g group >>>>> # -E executability >>>>> # -A acls >>>>> # -X xtended attributes >>>> >>>> That's how I'd do it also. I think if you didn't mind including -D, >>>> which from my understanding is --special and --devices, neither of >>>> which I'd expect in a maildir, and if you wanted to do your deletion >>>> manually after the fact, would this be equivalent? >>> >>> well, i have them ususally in my "rsync.sh" but did not expect >>> such files in a maildir, but yes it does not hurt >>> >>>> rsync -PaHAX /folder/ >>>> >>>> The man page says -a (--archive) is equivalent to -rlptgoD, and you >>>> have all but D listed up there. >>>> >>>> My thinking on later manually deleting the old maildir, instead of >>>> letting rsync do it, is that if somehow, some weird thing goes wrong, I >>>> have the old one for backup. Who knows, maybe I copied the thing on top >>>> of the wrong other maildir and have to back it out -- I'd have the old >>>> maildir as a reference of which files. >>> >>> you missunderstand "--delete-after" >>> >>> this is for delete files in the TARGET folder which is not >>> or no longer in the source and IMHO very very important >>> if you want sync folders 1:1 because old artefacts can have >>> very bad effects >>> >>> without you merge folders and if i know "hey my source contains >>> exactly what i need, not more and lot less" this is not what i >>> would like and never did in 10 years IT >> >> If you use "--delete-after" you could add "--fuzzy", which theoretically >> could save transfers because of filename renames. >> >> Anyway: Any form of "--delete" is required for Maildir, IMHO, because >> Labels, Tags, Keywords (or whatever the MUA calls it) and status >> information (seen, read, deleted) are reflected by the filename. Hence, if >> one does not "--delete", the _same_ message might pop up in the Maildir >> multiple times but with different status and/or tags. Same applies to new >> messages, because they are storred in /new and later moved to >> /cur. So one ends with the same message in "new" and in "cur". >> >> - -- >> Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.11 (GNU/Linux) >> >> iQEVAwUBURn+wV3r2wJMiz2NAQJYnwgAwGfCHBFglm800nbaHFhsdWeuVkhGJjnW >> l0QtbmQLaSdQOkWwdVkmNEQkJLmwvfhddG0a9GblofI/zn8EBN+3EDR/CM0d5Y8f >> jy1yqemT26sFTVC6NlTqhDF8zq/Oi5WK7ftUfYfrt90MtmMD1rzmBQ5Q/N+Tteae >> TeE0jeHNL0rHl9DoAV9AkBzhJPnYYodSK2lA1oewazlzRpzJSiHYmgVh4RiSmsyk >> 4DAtch5ZrqLnsx1A/mUHAfDvdk66j2Os0bLXqbHu9ZAzz5Xb9bbNfpu52u1Ukg/o >> TfXfiGA4aRix6I1OGePdXmi/qXTzPVLBzhhIZdMiJ2CKCWNFbIYC7w== >> =ugv1 >> -----END PGP SIGNATURE----- >> > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURs+tV3r2wJMiz2NAQIvngf/c37Hbg30lIq0wFPCif9IAG+BO8yDRAa3 lYY2t7DlszWopeEBvV22HjUC8SlzywECYEhBFRWhJVfqNyu55oI867of+RZ6lO0A lNcThmNixX5IsFiUnN9S9NYrTENC2qhccPdrTCAcm6A6CtaR1ydyeZDxlmvmTWe4 Za60LVqlsIVoZQ146yGE5nSIctix8JOE5kfGO5NurYXTfHt9CrPj5JvlPmRUdasp Mtn+QpxaxzFa8NGECSThKDVilMPFvqfT+JrhjWnkM1v31hNjJj4F3DYhk1L0YlAL 4wUjskXq1ytoM8+k9c8rqCy5DvM7g8N2ip0c0Buvjt+RctaHE4F9jg== =moaG -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Wed Feb 13 09:56:01 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 13 Feb 2013 08:56:01 +0100 (CET) Subject: [Dovecot] Converting a POP3 client to IMAP In-Reply-To: <511AD062.3020600@teradactyl.com> References: <511AD062.3020600@teradactyl.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 12 Feb 2013, Kristen J. Webb wrote: > Our current email is outsourced and so we > have used POP3 to keep all email on our > systems for many years. > > Is there a best practice for moving mail > such that we can take advantage of the > hard link feature for all those older copies > that are currently in POP3 files? > > It would also be nice to be able to auto file > things. I don't see the sieve filters being > activated when I copy mail in Thunderbird. you could use fetchmail to get the messages from the POP3 box and pass them to Dovecot deliver. So you can use Sieve filters (or not, if you turn Sieve off for this). Then let the messages spool to Maildir. Then use fdupes to find duplicates and hardlink them. Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURtHEV3r2wJMiz2NAQLv7wf/afBQrT5EvVzXux4zoPo/ZJTt8upHaueV fQiuIdqmzGSiOgrXpZyxOcL9fsBFOQYuYVzPDyISbOdUj8Pf+9pjTdkDlRDKiOYC IXBWj/7I5x64V1Mpy+4TkSnqA8NSB3Tzu0Rp0xOwTa73QwSqjuVw7oO0oqD4nLWj 5CEvk6laEmEsEO/XrlEcuA9OzK32oztx+CxCnTWPrH30ARL4GfckObFkiPOR57XD JXKVJ+PigvPdsUK7aT1/V8ow4an9cjfsCwr4uxeMprfdHfoyu6ghphVl1iB8wCDi Gt7kDU70hR4eK7wArl9J3RLnsr0Ap5o8EHXl+OqLFkM5NCsKYdqjDQ== =9qoB -----END PGP SIGNATURE----- From dovecot at anes.su Wed Feb 13 10:28:53 2013 From: dovecot at anes.su (Anes Mukhametov) Date: Wed, 13 Feb 2013 12:28:53 +0400 Subject: [Dovecot] Dsync converting dbox->Maildir Message-ID: Hi! I've changed my mailbox format from dbox to Maildir. The problem is maildir message filename extension has no S=/W= field after migration. Using dovecot 2.1.15. From h.reindl at thelounge.net Wed Feb 13 11:18:38 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 13 Feb 2013 10:18:38 +0100 Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: References: <511958FD.8040205@thelounge.net> <20130211163741.3bd8e49c@mydesk> <5119663C.8030508@thelounge.net> Message-ID: <511B5A6E.1050208@thelounge.net> Am 13.02.2013 08:20, schrieb Steffen Kaiser: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 12 Feb 2013, David Mehler wrote: > >> Thanks, if I did: >> >> rsync --force --delete-after -tPrlHpogEAXz /folder/ /backups/newmailstore/ >> >> could I then tar up the newmailstore folder with something like tar >> zcf without messing anything up? > > keep in mind, that the backup of the Maildir with rsync is no 100% consistent snapshot, because of the filename > renames; otherwise: yes who would rename them and why? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Wed Feb 13 12:32:18 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 Feb 2013 11:32:18 +0100 Subject: [Dovecot] Is there any way to mask or hide the INBOX label for Dovecot 1.2.x Shared Mailboxes? In-Reply-To: <802F6258-DC15-43A1-8ED0-CA5FA09C4AA7@practiceofcode.com> References: <802F6258-DC15-43A1-8ED0-CA5FA09C4AA7@practiceofcode.com> Message-ID: On 13.2.2013, at 0.12, Jason Discount wrote: > I'm setting up a new Dovecot sever to replace an old Cyrus server, and need to keep things as consistent as possible. I have a couple local accounts, like junkmail and notjunkmail that users get shared access to, and can drop things in to train sa-update. However, when users share over junkmail's inbox, it's displayed in the hierarchy: > > ? Other Users > ? junkmail > ? INBOX > > Is there any way to get junkmail's INBOX to just show up at Other Users/junkmail and not require that extra depth? > > ? Other Users > ? junkmail mail_shared_explicit_inbox=no > I'm running Debian Squeeze and the standard dovecot package available therein: > >> # uname -a >> Linux internet.digitalquay.com.au 2.6.32-5-xen-amd64 #1 SMP Sun Sep 23 13:49:30 UTC 2012 x86_64 GNU/Linux >> #dovecot --version >> 1.2.15 You need v2.1.6 or newer. Debian backports has 2.1.7. From tss at iki.fi Wed Feb 13 12:44:20 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 Feb 2013 11:44:20 +0100 Subject: [Dovecot] Imap process crash: assertion failed: (full_fs_access) In-Reply-To: References: Message-ID: On 11.2.2013, at 12.20, Joseph Tam wrote: > > Timo Sirainen writes: > >> > Any guess at what would cause this? >> > > Feb 7 21:20:53 server dovecot: imap(user): Panic: file >> > mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion >> > failed: (full_fs_access) >> LIST command that had invalid parameters that somehow got through the >> first validity check. What namespace config do you have? > > namespace { > alias_for = > hidden = yes > list = no > location = > prefix = / > separator = / > } This one causes these crashes. Fixed in v2.2 now: http://hg.dovecot.org/dovecot-2.2/rev/fb367a977077 From tss at iki.fi Wed Feb 13 12:46:44 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 Feb 2013 11:46:44 +0100 Subject: [Dovecot] dsync 2.1.10 does not correctly handle renames of mailbox subtrees In-Reply-To: References: Message-ID: <4593645D-1183-4D54-B39E-A1A6CD5E694A@iki.fi> On 12.2.2013, at 11.19, Karol Jurak wrote: > It seems that dsync 2.1.10 does not correctly handle renames of mailbox > subtrees. The scenario is as follows. I'm hoping I won't have to fix this in v2.1. v2.2 has a redesigned dsync where rename syncing should work much better. From karol.jurak at gmail.com Wed Feb 13 13:22:12 2013 From: karol.jurak at gmail.com (Karol Jurak) Date: Wed, 13 Feb 2013 12:22:12 +0100 Subject: [Dovecot] dsync 2.1.10 does not correctly handle renames of mailbox subtrees In-Reply-To: <4593645D-1183-4D54-B39E-A1A6CD5E694A@iki.fi> References: <4593645D-1183-4D54-B39E-A1A6CD5E694A@iki.fi> Message-ID: On 13 February 2013 11:46, Timo Sirainen wrote: > On 12.2.2013, at 11.19, Karol Jurak wrote: > > > It seems that dsync 2.1.10 does not correctly handle renames of mailbox > > subtrees. The scenario is as follows. > > I'm hoping I won't have to fix this in v2.1. v2.2 has a redesigned dsync > where rename syncing should work much better. > > Would you mind taking a look at the following patch? It appends to dovecot.mailbox.log records describing renames of all mailboxes in a subtree, not just the top level one. My basic tests showed that this indeed allows dsync to correctly replicate the renames, but I'm not familiar with dovecot internals enough to be confident that this doesn't break something else. diff --git a/src/lib-storage/index/index-storage.c b/src/lib-storage/index/index-storage.c index 6d0771c..292c0fa 100644 --- a/src/lib-storage/index/index-storage.c +++ b/src/lib-storage/index/index-storage.c @@ -584,6 +584,9 @@ int index_storage_mailbox_rename(struct mailbox *src, struct mailbox *dest, bool rename_children) { guid_128_t guid; + struct mailbox_list_iterate_context *iter; + const char *pattern; + const struct mailbox_info *info; if (src->list->v.rename_mailbox(src->list, src->name, dest->list, dest->name, @@ -596,6 +599,23 @@ int index_storage_mailbox_rename(struct mailbox *src, struct mailbox *dest, non-selectable mailbox (directory), which doesn't even have a GUID */ mailbox_name_get_sha128(dest->name, guid); mailbox_list_add_change(src->list, MAILBOX_LOG_RECORD_RENAME, guid); + + if (rename_children) { + pattern = t_strdup_printf("%s%c*", dest->name, + mail_namespace_get_sep(dest->list->ns)); + + iter = mailbox_list_iter_init(dest->list, pattern, + MAILBOX_LIST_ITER_RETURN_NO_FLAGS); + + while ((info = mailbox_list_iter_next(iter)) != NULL) { + mailbox_name_get_sha128(info->name, guid); + mailbox_list_add_change(src->list, MAILBOX_LOG_RECORD_RENAME, + guid); + } + + mailbox_list_iter_deinit(&iter); + } + return 0; } From rob.redpath at heartinternet.co.uk Wed Feb 13 13:49:03 2013 From: rob.redpath at heartinternet.co.uk (Rob Redpath) Date: Wed, 13 Feb 2013 11:49:03 +0000 Subject: [Dovecot] Broken physical size caching in Dovecot 2.1.10 In-Reply-To: <1360030002.10326.52.camel@innu> References: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> <1360030002.10326.52.camel@innu> Message-ID: <511B7DAF.30109@heartinternet.co.uk> On 05/02/13 02:06, Timo Sirainen wrote: > On Tue, 2013-01-29 at 19:58 +0000, Rob Redpath wrote: > >> I'm encountering two, probably related, errors in my Dovecot 2.1.10 >> install. The issue relates to compressed email stored in Maildir format >> directories. >> >> In some cases, a mailbox will become inaccessible, and the following >> will be logged from a doveadm fetch:- >> >> doveadm(info at example.com): Error: Corrupted index cache file >> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken >> physical size for mail UID 2777 >> doveadm(info at example.com): Error: Cached message size smaller than >> expected (18996 < 64624) >> doveadm(info at example.com): Error: Maildir filename has wrong S value, >> renamed the file from >> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z >> to >> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z >> >> Interestingly, the file mentioned in the error ( >> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't >> exist. >> >> The uncompressed size of the file is 64624 bytes, the compressed size is >> 18996. > Dovecot's zlib plugin requires that the maildir filenames > have ,S=uncompressed size. Otherwise you get those errors. > http://dovecot.org/tools/maildir-size-check.sh can be used to fix the > filenames. > Thanks for this. The problem I'm encountering is that Dovecot renames files, and does so to their size on disk rather than their uncompressed size, undoing any corrective action that's been taken. For example:- [root at mailserver ~]# ls /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ [root at mailserver ~]# zcat /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ | wc 345 1530 24587 [root at mailserver ~]# mv /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ [root at mailserver ~]# doveadm fetch -u "info at example.com" text all > /dev/null doveadm(info at example.com): Error: Maildir filename has wrong S value, renamed the file from /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ to /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ From piotr.rotter at active24.pl Wed Feb 13 14:57:25 2013 From: piotr.rotter at active24.pl (Piotr Rotter) Date: Wed, 13 Feb 2013 13:57:25 +0100 Subject: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients Message-ID: <511B8DB5.10105@active24.pl> Hello, I have issue with dovecot (v. 2.1.13)configuration after implementation dovecot-lda (I used maildrop earlier). Now i have 9 servers with storage by NFS v3. My problem is frequently dovecot-uidlist crashing. 2013-02-13T13:34:02.447969+01:00 k1 dovecot: imap(user at domain.com): Error: Broken file /path_to_mailbox/dovecot-uidlist line 3170: Invalid data: 2013-02-13T13:34:34.527715+01:00 k1 dovecot: pop3(user at domain.com): Error: Broken file /path_to_mailbox/dovecot-uidlist line 367: Invalid data: It happens few time in minute for diferent users and has effect to re retriving messages by pop3 clients. This my NFS mount options: (rw,tcp,vers=3,actimeo=0) 10-mail.conf options: mmap_disable = yes dotlock_use_excl = yes mail_fsync=always mail_nfs_storage = yes mail_nfs_index = yes lock_method = dotlock I do not know what I did wrong and how solve this issue. Maybe NFS v4 or fcntl locks or migration to pop3_uidl_format = %g will by solution. Please some hint. Please any help for solve this issue and sorry my english. From skdovecot at smail.inf.fh-brs.de Wed Feb 13 15:07:43 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 13 Feb 2013 14:07:43 +0100 (CET) Subject: [Dovecot] backing up maildir dovecot files In-Reply-To: <511B5A6E.1050208@thelounge.net> References: <511958FD.8040205@thelounge.net> <20130211163741.3bd8e49c@mydesk> <5119663C.8030508@thelounge.net> <511B5A6E.1050208@thelounge.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 13 Feb 2013, Reindl Harald wrote: >>> Thanks, if I did: >>> >>> rsync --force --delete-after -tPrlHpogEAXz /folder/ /backups/newmailstore/ >>> >>> could I then tar up the newmailstore folder with something like tar >>> zcf without messing anything up? >> >> keep in mind, that the backup of the Maildir with rsync is no 100% consistent snapshot, because of the filename >> renames; otherwise: yes > > who would rename them and why? "/folder/" would be a Maildir on a production mail server. If rsync runs (without help from other funtionality, such as LVM or ZFS snapshots) and at the same time someone accesses the Maildir and: a) "see"s a message, which lets Dovecot rename the message file from ***/new/ to ***/cur b) tags a message with a keyword, which lets Dovecot rename the file to have some lower-case letter c) untags a keyword -> remove that letter from the filename d) changes Deleted, Read, Answered status --> add/remove an upper-case letter from filename rsync might have cached a message with an old filename no longer physically present on disk. You will see a "XYZ vanished" message and that particular message is not backuped, but removed from backup. Some scripts use rsync in a loop in assumption, that no Maildir is accessed that often, that you get a clean run of rsync eventually. Maybe, if you tag a message with a new keyword, rsync already copied dovecot-keywords without the new keyword, but copy the message, when its filename has the keyword-letter. In fact, if I want to make 100% sure I get an exact copy, I do this: rsync /from /to rsync /from /to mv /from /from_locked sleep 1 rsync /from_locked /to mv /from_locked /from "/from" is the base directory of the Maildir. In my environment that causes tempfails on delivery and internal server errors on IMAP/POP3 access. But the time between the two mv's is very small, because the main differences are handled with the first two rsync's. As you seen, I copy one user after another, which breaks hardlinks between users, but keeps memory footprint of rsync low and as well as the downtime. I do make my usual daily backups without the loop and the mv's, because it is very seldom I get the "vanished" messages. I suppose, because the script runs two hours after midnight. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURuQIF3r2wJMiz2NAQKgjgf8CtyeO6pONFU038x8O2wggcntdvGyeg8Z Uq8KR0QFVg1n0HNDaa33OYN4IxSHX8zNvElf+wc0ejQ3NUOPVDl2mdm4iEihyOYv Veb/p2iK671Nrs8nB7USwx7OE9vY8IYoB/ZSXrXGWowqOqRQIcJWHAfZ9Ewj3Rg/ iRGMUNCn7UfEDfWl+F5yWpdp/+3xJGxoWeWaegW/yfTzlJ5nKffS/SAfJlUm7zuV u31JL4fjk25uGG7alzrCxOq0z4A3PvcpGag2nkfIRbrLLmo4Wzr+09Bd2zqmSn74 /PaHlxFS6a2uy7ugqdd5kxfiZHnOS2/d6JgV428I8qS0CWugEPuFbg== =mw5e -----END PGP SIGNATURE----- From joseba.torre at ehu.es Wed Feb 13 16:19:46 2013 From: joseba.torre at ehu.es (Joseba Torre) Date: Wed, 13 Feb 2013 15:19:46 +0100 Subject: [Dovecot] Authentication with many Windows AD In-Reply-To: <004a01ce090c$c16bad40$444307c0$@gtt.es> References: <004a01ce090c$c16bad40$444307c0$@gtt.es> Message-ID: <511BA102.6020303@ehu.es> El 12/02/13 11:36, Antonio Alberola escribi?: > Dear All, > > I'm a new user of Dovecot and I need support with the authentication > process. > I use Dovecot 2.0.9 and Postfix 2.6.6 on host x86_64-redhat-linux-gnu > 2.6.32-279.14.1.el6.x86_64 CentOS release 6.3 (Final) > > I need to select the validation method depending on the user. I have users > that have to be authenticate with UNIX account, and I have others thats have > to be authenticate with Windows AD. For example, some sysadmins and special > accounts needs validate with Unix accounts. Other normal users have to > authenticate with Windows AD, I have many Windows controllers. Some users > authenticate to a specific Windows AD and other users are validated in > another one. > > How I can do this? You can set as many databases as you want. Check this: http://wiki2.dovecot.org/PasswordDatabase http://wiki2.dovecot.org/Authentication/MultipleDatabases HTH From tss at iki.fi Wed Feb 13 16:25:48 2013 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 Feb 2013 15:25:48 +0100 Subject: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients In-Reply-To: <511B8DB5.10105@active24.pl> References: <511B8DB5.10105@active24.pl> Message-ID: On 13.2.2013, at 13.57, Piotr Rotter wrote: > I have issue with dovecot (v. 2.1.13)configuration after implementation dovecot-lda (I used maildrop earlier). Now i have 9 servers with storage by NFS v3. My problem is frequently dovecot-uidlist crashing. > > 2013-02-13T13:34:02.447969+01:00 k1 dovecot: imap(user at domain.com): Error: Broken file /path_to_mailbox/dovecot-uidlist line 3170: Invalid data: > 2013-02-13T13:34:34.527715+01:00 k1 dovecot: pop3(user at domain.com): Error: Broken file /path_to_mailbox/dovecot-uidlist line 367: Invalid data: > > It happens few time in minute for diferent users and has effect to re retriving messages by pop3 clients. Use director: http://wiki2.dovecot.org/Director From JLock at csolve.net Wed Feb 13 17:30:23 2013 From: JLock at csolve.net (Jason Lock) Date: Wed, 13 Feb 2013 15:30:23 +0000 Subject: [Dovecot] FreeBSD, Dovecot and ZFS In-Reply-To: <511ABD4D.2010707@Media-Brokers.com> References: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> <511ABD4D.2010707@Media-Brokers.com> Message-ID: <8064532408EC2C48872677AD4390209433A3F885@CTIEXCH10.csolve.local> >> WAG would be similar issues you can face when using NFS with multiple servers accessing it (file locking issues). >> The solution would be to use Director to make sure users are always directed to the same server. >> http://wiki2.dovecot.org/Director >> If that isn't the problem, then much more info would be needed (ie, doveconf -n output, logs exhibiting the problem, etc)... >> -- >> >> Best regards, >> >> */Charles /* Thank you for your reply. To further expand the problem does not happen with any regularity, we went over 30 days with no issue after two weeks of sporadic occurrences. It usually only appears, if at all, any time after 2:00 PM (i.e. 14:30, 15:20, 16:10). And not every day (has not happened on a weekend). The number of POP3 and IMAP processes increase dramatically when the issue occurs. Here is a copy of the dovecot -n output # 1.2.17: /usr/local/etc/dovecot.conf # OS: FreeBSD 8.1-RELEASE-p5 i386 nfs protocols: imap imaps pop3 pop3s ssl_cert_file: /mail/shared/etc/ssl/certs/dovecot.pem ssl_key_file: /mail/shared/etc/ssl/private/dovecot.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting: Hello there, who might you be? max_mail_processes: 4096 verbose_proctitle: yes first_valid_uid: 26 first_valid_gid: 0 mail_privileged_group: mail mail_location: maildir:/mail/store/%d/%n mmap_disable: yes dotlock_use_excl: no mail_nfs_storage: yes mail_nfs_index: yes lock_method: dotlock mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: postmaster_address: {REMOVED} sendmail_path: /usr/local/sbin/exim auth default: default_realm: {REMOVED} username_format: %Lu passdb: driver: sql args: /mail/shared/etc/dovecot-sql.conf userdb: driver: passwd userdb: driver: static args: uid=26 gid=6 home=/mail/store/%d/%n From JLock at csolve.net Wed Feb 13 17:54:14 2013 From: JLock at csolve.net (Jason Lock) Date: Wed, 13 Feb 2013 15:54:14 +0000 Subject: [Dovecot] FreeBSD, Dovecot and ZFS In-Reply-To: <20130213002402.GB900@anubis.morrow.me.uk> References: <8064532408EC2C48872677AD4390209433A3D5F3@CTIEXCH10.csolve.local> <20130213002402.GB900@anubis.morrow.me.uk> Message-ID: <8064532408EC2C48872677AD4390209433A3F8BD@CTIEXCH10.csolve.local> At 9PM +0000 on 12/02/13 you (Jason Lock) wrote: > Mail Issues - FreeBSD > > Hello, my apologies if this may be the wrong forum but hoping that > maybe someone might be able to provide some insight. >> This may turn out to be something better addressed on freebsd-stable, but this is a perfectly good place to start. Thank you for your reply. > Have a very sporadic and strange issue with our mail servers running > Dovecot on FreeBSD. There are three servers hosting Dovecot with > FreeBSD as the underlying operating system. All three connect to a > NAS server, again running FreeBSD and ZFS. >> Over NFS, I assume? What version, what mount options, and what type of authentication? Yes, using NFSv3 to attach the share from the NAS to each of the POP3/IMAP servers. Only mount options set are RW. Not authentication in place, NAS and POP3/IMAP Servers share VLAN just for the NAS connections. >> What locking strategies is Dovecot using? In dovecot using the following: dotlock_use_excl: no mail_nfs_storage: yes mail_nfs_index: yes lock_method: dotlock >> Are there any suspicious messages in syslog on either machine? Nothing specific. > When the specific issue occurs, clients connecting to check mail via > POP3 or IMAP experience long delays and timeouts. To the point where > POP3 Logins fail due to the timeouts. The issue is further compounded > by clients increasing the number of attempts to check mail. >> Are the delays happening before or after login? Delays appear during login, username gets passed but then timeouts after the password is sent. >> If you can provoke this and get a 'procstat -k' for the relevant dovecot process this might be helpful. If 'long' delays means >> several minutes, running something along the lines of 'procstat -k $(pgrep -U dovecot -U doveauth)' every minute or so for a >> while might be one way to catch this, though this will collect a lot of data rather fast so you will need some way to locate the >> relevant entry. Will look to capture that information if possible, have not been able to re-create the situation in which the issue occurs. >> Ben From rob.redpath at heartinternet.co.uk Wed Feb 13 19:55:11 2013 From: rob.redpath at heartinternet.co.uk (Rob Redpath) Date: Wed, 13 Feb 2013 17:55:11 +0000 Subject: [Dovecot] Broken physical size caching in Dovecot 2.1.10 In-Reply-To: <1360030002.10326.52.camel@innu> References: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> <1360030002.10326.52.camel@innu> Message-ID: <511BD37F.7010308@heartinternet.co.uk> On 05/02/13 02:06, Timo Sirainen wrote: > On Tue, 2013-01-29 at 19:58 +0000, Rob Redpath wrote: > >> I'm encountering two, probably related, errors in my Dovecot 2.1.10 >> install. The issue relates to compressed email stored in Maildir format >> directories. >> >> In some cases, a mailbox will become inaccessible, and the following >> will be logged from a doveadm fetch:- >> >> doveadm(info at example.com): Error: Corrupted index cache file >> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken >> physical size for mail UID 2777 >> doveadm(info at example.com): Error: Cached message size smaller than >> expected (18996 < 64624) >> doveadm(info at example.com): Error: Maildir filename has wrong S value, >> renamed the file from >> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z >> to >> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z >> >> Interestingly, the file mentioned in the error ( >> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't >> exist. >> >> The uncompressed size of the file is 64624 bytes, the compressed size is >> 18996. > Dovecot's zlib plugin requires that the maildir filenames > have ,S=uncompressed size. Otherwise you get those errors. > http://dovecot.org/tools/maildir-size-check.sh can be used to fix the > filenames. > > Thanks for this. The problem I'm encountering is that Dovecot renames files, and does so to their size on disk rather than their uncompressed size, undoing any corrective action that's been taken. For example:- [root at mailserver ~]# ls /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ [root at mailserver ~]# zcat /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ | wc 345 1530 24587 [root at mailserver ~]# mv /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ [root at mailserver ~]# doveadm fetch -u "info at example.com" text all > /dev/null doveadm(info at example.com): Error: Maildir filename has wrong S value, renamed the file from /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ to /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ From rplatel at tucows.com Wed Feb 13 20:04:11 2013 From: rplatel at tucows.com (Richard Platel) Date: Wed, 13 Feb 2013 13:04:11 -0500 Subject: [Dovecot] "Renaming not supported across conflicting directory permissions" Message-ID: <04C4B084-A082-443E-A54A-1E7A147EC3BE@tucows.com> Is it possible to disable this check in 2.1.7? (without a patch and re-compile?) From philq at qsystemsengineering.com Wed Feb 13 23:30:27 2013 From: philq at qsystemsengineering.com (PhilQ) Date: Wed, 13 Feb 2013 13:30:27 -0800 (PST) Subject: [Dovecot] Requested xxxx scheme, but we have a NULL password after upgrade Message-ID: <1360791027564-40123.post@n4.nabble.com> I'm having an issue I can't seem to work around after upgrading from Dovecot 1.0.7 to 1.2.17. After getting Dovecot 1.07 working on CentOS 5.9, I decided that it might be wise to upgrade to a later version, so I stuck with 1.x and went with 1.2.17, which I had to compile from source. CentOS was originally using /etc as the starting path for Dovecot files but the source distribution puts most of the stuff under /usr/local/etc. After the usual config>make>make install dance I made the necessary changes to point to the new libraries, modules, etc. and the "imap-login: Fatal: Dovecot version mismatch: Master is v1.2.17, login is v1.0.7...." messages went away. After doing this though I cannot login, I get the following error messages: Feb 13 15:50:40 auth(default): Info: client in: AUTH 7 NTLM service=imap lip=192.168.2.102 rip=192.168.2.100 lport=143 rport=1470 Feb 13 15:50:40 auth(default): Info: client out: CONT 7 Feb 13 15:50:40 auth(default): Info: client in: CONT 7 TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFASgKAAAADw== Feb 13 15:50:40 auth(default): Info: client out: CONT 7 TlRMTVNTUAACAAAAMAAwADAAAAAFAooATj7XW6ve2hwAAAAAAAAAADgAOABgAAAAUwBlAHIAdgBlAHIAMQAuAGgAZQByAHMAYwBoAGwAYQB1AHIAZQBuAC4AYwBvAG0AAwAwAFMAZQByAHYAZQByADEALgBoAGUAcgBzAGMAaABsAGEAdQByAGUAbgAuAGMAbwBtAAAAAAA= Feb 13 15:50:40 auth(default): Info: client in: CONT 7 TlRMTVNTUAADAAAAGAAYAGoAAABoAGgAggAAAAAAAABIAAAAEAAQAEgAAAASABIAWAAAAAAAAADqAAAABQKIAgUBKAoAAAAP*CENSORED*bgBiAFEAUwBFAC0AVwBJAE4AWABQAEXO6p/WuopqQ02x1kzJGW3NoQELKw32N88JqkbMOYOVErhiS492elwBAQAAAAAAA*CENSORED*ysN9jcAAAAAAwAwAFMAZQByAHYAZQByADEALgBoAGUAcgBzAGMAaABsAGEAdQByAGUAbgAuAGMAbwBtAAAAAAAAAAAA Feb 13 15:50:40 auth(default): Info: cache(pquesinb,192.168.2.100): miss Feb 13 15:50:40 auth(default): Info: passwd-file(pquesinb,192.168.2.100): lookup: user=pquesinb file=/etc/dovecot.users Feb 13 15:50:40 auth(default): Info: password(pquesinb,192.168.2.100): Requested NTLM scheme, but we have a NULL password Feb 13 15:50:40 auth(default): Info: cache(pquesinb,192.168.2.100): miss Feb 13 15:50:40 auth(default): Info: password(pquesinb,192.168.2.100): passdb doesn't support credential lookups Feb 13 15:50:40 auth(default): Info: cache(pquesinb,192.168.2.100): miss Feb 13 15:50:40 auth(default): Info: password(pquesinb,192.168.2.100): passdb doesn't support credential lookups Feb 13 15:50:40 auth(default): Info: cache(pquesinb,192.168.2.100): miss Feb 13 15:50:40 auth(default): Info: password(pquesinb,192.168.2.100): passdb doesn't support credential lookups Feb 13 15:50:40 auth(default): Info: cache(pquesinb,192.168.2.100): miss Feb 13 15:50:40 auth(default): Info: password(pquesinb,192.168.2.100): passdb doesn't support credential lookups Feb 13 15:50:42 auth(default): Info: client out: FAIL 7 user=pquesinb Looking at the log from the old version while it was working, I was getting messages like the following: dovecot: Feb 04 14:14:21 Info: imap-login: Login: user=, method=NTLM, rip=192.168.2.100, lip=192.168.2.102 dovecot: Feb 04 14:14:21 Info: imap-login: Login: user=, method=NTLM, rip=192.168.2.100, lip=192.168.2.102 dovecot: Feb 04 14:15:03 Info: IMAP(pquesinb): Disconnected: Logged out dovecot: Feb 04 14:15:03 Info: IMAP(pquesinb): Disconnected: Logged out dovecot: Feb 04 14:15:23 Info: imap-login: Login: user=, method=NTLM, rip=192.168.2.100, lip=192.168.2.102 dovecot: Feb 04 14:15:23 Info: imap-login: Login: user=, method=NTLM, rip=192.168.2.100, lip=192.168.2.102 dovecot: Feb 04 14:16:05 Info: IMAP(pquesinb): Disconnected: Logged out dovecot: Feb 04 14:16:05 Info: IMAP(pquesinb): Disconnected: Logged out /etc/dovecot.users contains a list of usernames. Is this error the result of additional security which has been incorporated into the later version of Dovecot, or is it because my installation of the later version from source is broken, somehow incompatible, etc? Dovecot was configured to use PAM and it appeared to know the password of my account, failing when it was entered incorrectly so I'm assuming that it was successfully using PAM. I kept the same syntax in the later config file. >From dovecot.conf: passdb pam { # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=] # [cache_key=] [] # # session=yes makes Dovecot open and immediately close PAM session. Some # PAM plugins need this to work, such as pam_mkhomedir. # # setcred=yes makes Dovecot establish PAM credentials if some PAM plugins # need that. They aren't ever deleted though, so this isn't enabled by # default. # # max_requests specifies how many PAM lookups to do in one process before # recreating the process. The default is 100, because many PAM plugins # leak memory. # # cache_key can be used to enable authentication caching for PAM # (auth_cache_size also needs to be set). It isn't enabled by default # because PAM modules can do all kinds of checks besides checking password, # such as checking IP address. Dovecot can't know about these checks # without some help. cache_key is simply a list of variables (see # doc/wiki/Variables.txt) which must match for the cached data to be used. # Here are some examples: # %u - Username must match. Probably sufficient for most uses. # %u%r - Username and remote IP address must match. # %u%s - Username and service (ie. IMAP, POP3) must match. # # The service name can contain variables, for example %Ls expands to # pop3 or imap. # # Some examples: # args = session=yes %Ls args = cache_key=%u dovecot #args = dovecot } If anyone could give me some ideas on where to go from here, I'd really appreciate it. If there's little chance of getting the newer version to work with CentOS 5 then I'm ready to just drop back to the older version. Thanks a bunch. - Phil Config info follows: [root at Server1 lda]# dovecot -n # 1.2.17: /usr/local/etc/dovecot.conf # OS: Linux 2.6.18-348.el5.centos.plusxen x86_64 CentOS release 5.9 (Final) log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.log protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/pki/dovecot/certs/dovecot.pem ssl_key_file: /etc/pki/dovecot/private/dovecot.pem disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login max_mail_processes: 64 mail_location: maildir:~/Maildir maildir_very_dirty_syncs: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 mail_log_max_lines_per_sec: 100 imap_client_workarounds(default): outlook-idle imap_client_workarounds(imap): outlook-idle imap_client_workarounds(pop3): lda: mail_plugin_dir: /usr/local/lib/dovecot/lda auth default: mechanisms: ntlm plain login digest-md5 cache_size: 16 cache_ttl: 90 verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot.users passdb: driver: pam args: cache_key=%u dovecot passdb: driver: passwd passdb: driver: shadow userdb: driver: passwd -- View this message in context: http://dovecot.2317879.n4.nabble.com/Requested-xxxx-scheme-but-we-have-a-NULL-password-after-upgrade-tp40123.html Sent from the Dovecot mailing list archive at Nabble.com. From pch14 at myzel.net Thu Feb 14 00:09:57 2013 From: pch14 at myzel.net (Hungerburg) Date: Wed, 13 Feb 2013 23:09:57 +0100 Subject: [Dovecot] Authentication with many Windows AD In-Reply-To: <511BA102.6020303@ehu.es> References: <004a01ce090c$c16bad40$444307c0$@gtt.es> <511BA102.6020303@ehu.es> Message-ID: <511C0F35.5050503@myzel.net> Am 2013-02-13 15:19, schrieb Joseba Torre: > > You can set as many databases as you want. Beware that databases are processed sequentially and that failed lookups may incur a timeout. -- peter From piotr.rotter at active24.pl Thu Feb 14 00:43:13 2013 From: piotr.rotter at active24.pl (Piotr Rotter) Date: Wed, 13 Feb 2013 23:43:13 +0100 Subject: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients In-Reply-To: References: <511B8DB5.10105@active24.pl> <511BB23A.3050004@active24.pl> Message-ID: <511C1701.3070304@active24.pl> I configure director temporary in some tricky way because I have to serve md5 and ntlm authentication method and I do not want reconfigure pop/imap servers. Can you confirm I done all ok becouse I have that issue: # doveadm director map doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed user mail server ip expire time 192.168.111.24 2013-02-14 00:57:34 192.168.111.21 2013-02-14 00:57:46 yet everything else works ok My password_query: password_query = SELECT clear AS password, 'Y' as proxy, '%u' AS destuser, clear AS pass FROM postfix_users WHERE email = '%u' My director server config (dovecot -n): # dovecot -n # 2.1.13: /etc/dovecot/dovecot.conf # OS: Linux 3.7.1-hardened-r2 x86_64 Gentoo Base System release 2.1 auth_mechanisms = plain login digest-md5 cram-md5 ntlm apop auth_verbose = yes default_client_limit = 10000 default_process_limit = 10000 default_vsz_limit = 512 M deliver_log_format = from=%f, msgid=%m, psize=%p: %$ director_mail_servers = 192.168.111.21 192.168.111.22 192.168.111.23 192.168.111.24 192.168.111.25 192.168.111.26 director_servers = 195.78.67.29 director_user_expire = 2 hours disable_plaintext_auth = no passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = box msgid from size } protocols = imap pop3 service auth { client_limit = 10000 unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service director { inet_listener { port = 9090 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service pop3-login { executable = pop3-login director } ssl_ca = References: <511B8DB5.10105@active24.pl> <511BB23A.3050004@active24.pl> <511C1701.3070304@active24.pl> Message-ID: <511C1BAB.8040902@active24.pl> I apologize for asking a question too fast without checking , I forgot look at logs. Of course missing iterate_query iterate_query = SELECT email AS user FROM postfix_users From jtam.home at gmail.com Thu Feb 14 01:37:06 2013 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 13 Feb 2013 15:37:06 -0800 (PST) Subject: [Dovecot] Imap process crash: assertion failed: (full_fs_access) In-Reply-To: References: Message-ID: On Wed, 13 Feb 2013, dovecot-request at dovecot.org wrote: Timo Sirainen writes: >>>> Any guess at what would cause this? >>>>> Feb 7 21:20:53 server dovecot: imap(user): Panic: file >>>> mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion >>>> failed: (full_fs_access) >>> LIST command that had invalid parameters that somehow got through the >>> first validity check. What namespace config do you have? >> >> namespace { >> alias_for = >> hidden = yes >> list = no >> location = >> prefix = / >> separator = / >> } > > This one causes these crashes. Fixed in v2.2 now: http://hg.dovecot.org/dovecot-2.2/rev/fb367a977077 Thanks for diagnosing this. I'll probably remove this namespace or limp along until 2.2 becomes the latest stable release. I added this namespace alias in an abundance of caution: this was to prevent users from trying to peruse the entire file space e.g. /etc. A poor man's chroot. I didn't test this thoroughly to see if it actually did what it was supposed to. Joseph Tam From ya.mwork at yandex.ru Thu Feb 14 11:16:15 2013 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Thu, 14 Feb 2013 13:16:15 +0400 Subject: [Dovecot] Segfault with doveadm-server Message-ID: <511CAB5F.8050502@yandex.ru> Hello. With dovecot ver. from 20130211 I have a segfault on doveadm when sync. There are messages from server: 2013-02-14T12:42:35.508031+04:00 imap1 kernel: [870542.927083] doveadm-server[9483]: segfault at 4 ip 00000074b228bc24 sp 000003f30b8e61b0 error 4 in doveadm-server[74b226b000+3a000] or 2013-02-14T12:47:39.850942+04:00 imap1 kernel: [870847.279513] doveadm[9513]: segfault at 4 ip 00000006b31ac294 sp 000003dc5c669a90 error 4 in doveadm[6b3178000+5f000] I cannot catch process doveadm-server by strace because it starts with unknown PID on demand. From acasado at ual.es Thu Feb 14 11:48:25 2013 From: acasado at ual.es (Antonio Casado Rodriguez) Date: Thu, 14 Feb 2013 10:48:25 +0100 Subject: [Dovecot] proxy imaps Message-ID: <511CB2E9.4060402@ual.es> Hi all, I had a imaps dovecot server working normal. I wanted to isolate the Internet connection by placing a proxy imap(Intranet connections still connect directly to the imap server). Yesterday I set up oneproxy imaps dovecot and thereafter, imap dovecot logs on errors: imap-login: Error: OpenSSL malloc() failed. You may need to increase login_process_size Need dovecot more memory with imap proxy connections than direct connections? Regards -- Antonio Casado Rodr?guez Administrador de Servicios de Red y Seguridad TIC ?rea de Comunicaciones STIC (Servicio de las Tecnolog?as de la Informaci?n y las Comunicaciones) UNIVERSIDAD DE ALMER?A Este mensaje y los ficheros que puedan ser adjuntados son confidenciales. Los mismos contienen informaci?n reservada que no puede ser difundida. Si usted ha recibido este correo por error, tenga la amabilidad de eliminarlo de su sistema y avisar al remitente mediante reenv?o a su direcci?n electr?nica; no deber? copiar el mensaje ni divulgar su contenido a ninguna persona. Su direcci?n de correo electr?nico junto a sus datos personales constan en un fichero titularidad de La UNIVERSIDAD DE ALMER?A cuya finalidad es la de mantener el contacto con usted. Si quiere saber de qu? informaci?n disponemos de usted, modificarla, y en su caso, cancelarla, puede hacerlo enviando un escrito al efecto, acompa?ado de una fotocopia de su D.N.I. a la siguiente direcci?n: UNIVERSIDAD DE ALMER?A ?. Secretar?a General de La Universidad de Almer?a. Edificio Central, Planta baja. Ctra. Sacramento s/n, La Ca?ada de San Urbano. CP 04120 Almer?a From rob.redpath at heartinternet.co.uk Thu Feb 14 12:44:53 2013 From: rob.redpath at heartinternet.co.uk (Rob Redpath) Date: Thu, 14 Feb 2013 10:44:53 +0000 Subject: [Dovecot] Broken physical size caching in Dovecot 2.1.10 In-Reply-To: <1360030002.10326.52.camel@innu> References: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> <1360030002.10326.52.camel@innu> Message-ID: <511CC025.5060001@heartinternet.co.uk> On 05/02/13 02:06, Timo Sirainen wrote: > On Tue, 2013-01-29 at 19:58 +0000, Rob Redpath wrote: > >> I'm encountering two, probably related, errors in my Dovecot 2.1.10 >> install. The issue relates to compressed email stored in Maildir format >> directories. >> >> In some cases, a mailbox will become inaccessible, and the following >> will be logged from a doveadm fetch:- >> >> doveadm(info at example.com): Error: Corrupted index cache file >> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken >> physical size for mail UID 2777 >> doveadm(info at example.com): Error: Cached message size smaller than >> expected (18996 < 64624) >> doveadm(info at example.com): Error: Maildir filename has wrong S value, >> renamed the file from >> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z >> to >> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z >> >> Interestingly, the file mentioned in the error ( >> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't >> exist. >> >> The uncompressed size of the file is 64624 bytes, the compressed size is >> 18996. > Dovecot's zlib plugin requires that the maildir filenames > have ,S=uncompressed size. Otherwise you get those errors. > http://dovecot.org/tools/maildir-size-check.sh can be used to fix the > filenames. > > Thanks for this. The problem I'm encountering is that Dovecot renames files, and does so to their size on disk rather than their uncompressed size, undoing any corrective action that's been taken. For example:- [root at mailserver ~]# ls /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ [root at mailserver ~]# zcat /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ | wc 345 1530 24587 [root at mailserver ~]# mv /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ [root at mailserver ~]# doveadm fetch -u "info at example.com" text all > /dev/null doveadm(info at example.com): Error: Maildir filename has wrong S value, renamed the file from /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ to /var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ From aalberola at gtt.es Thu Feb 14 12:45:06 2013 From: aalberola at gtt.es (Antonio Alberola) Date: Thu, 14 Feb 2013 11:45:06 +0100 Subject: [Dovecot] Authentication with many Windows AD Message-ID: <003d01ce0aa0$5a731660$0f594320$@gtt.es> Hi, > You can set as many databases as you want. Check this: > > http://wiki2.dovecot.org/PasswordDatabase > http://wiki2.dovecot.org/Authentication/MultipleDatabases This solution is valid if I have only one domain. In case you have more than one domain, you need authenticate in the appropriate LDAP server. Is That correct? I'm going to explain my particular case. I have five Windows domains with some users who belong to those domains. I also have other users who are authenticated with local UNIX accounts. The only thing that I can do (I think) is using a Radius server. But, I'd rather not use it. From CMarcus at Media-Brokers.com Thu Feb 14 13:08:04 2013 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 14 Feb 2013 06:08:04 -0500 Subject: [Dovecot] Segfault with doveadm-server In-Reply-To: <511CAB5F.8050502@yandex.ru> References: <511CAB5F.8050502@yandex.ru> Message-ID: <511CC594.6050002@Media-Brokers.com> On 2013-02-14 4:16 AM, Evgeny Basov wrote: > With dovecot ver. from 20130211 I have a segfault on doveadm when sync. > There are messages from server: Since you are using hg version, shouldn't you always upgrade to latest before reporting a problem? Maybe it isn't expected, but I know I would... -- Best regards, */Charles/* From ya.mwork at yandex.ru Thu Feb 14 14:05:49 2013 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Thu, 14 Feb 2013 16:05:49 +0400 Subject: [Dovecot] Segfault with doveadm-server In-Reply-To: <511CC594.6050002@Media-Brokers.com> References: <511CAB5F.8050502@yandex.ru> <511CC594.6050002@Media-Brokers.com> Message-ID: <511CD31D.1030105@yandex.ru> 14.02.2013 15:08, Charles Marcus ?????: > On 2013-02-14 4:16 AM, Evgeny Basov wrote: >> With dovecot ver. from 20130211 > > Since you are using hg version, shouldn't you always upgrade to latest > before reporting a problem? Maybe it isn't expected, but I know I > would... > I'm sorry but words "ver. from XXXX" means that version 2013021 and later(!) has this bug. From CMarcus at Media-Brokers.com Thu Feb 14 14:31:47 2013 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 14 Feb 2013 07:31:47 -0500 Subject: [Dovecot] Segfault with doveadm-server In-Reply-To: <511CD31D.1030105@yandex.ru> References: <511CAB5F.8050502@yandex.ru> <511CC594.6050002@Media-Brokers.com> <511CD31D.1030105@yandex.ru> Message-ID: <511CD933.9060107@Media-Brokers.com> On 2013-02-14 7:05 AM, Evgeny Basov wrote: > I'm sorry but words "ver. from XXXX" means that version 2013021 and > later(!) has this bug. Ahh, ok, sorry for the confusion... -- Best regards, */Charles/* From joseba.torre at ehu.es Thu Feb 14 15:32:29 2013 From: joseba.torre at ehu.es (Joseba Torre) Date: Thu, 14 Feb 2013 14:32:29 +0100 Subject: [Dovecot] Authentication with many Windows AD In-Reply-To: <003d01ce0aa0$5a731660$0f594320$@gtt.es> References: <003d01ce0aa0$5a731660$0f594320$@gtt.es> Message-ID: <511CE76D.4020706@ehu.es> El 14/02/13 11:45, Antonio Alberola escribi?: > > Hi, > >> >You can set as many databases as you want. Check this: >> > >> >http://wiki2.dovecot.org/PasswordDatabase >> >http://wiki2.dovecot.org/Authentication/MultipleDatabases > This solution is valid if I have only one domain. In case you have more than > one domain, you need authenticate in the appropriate LDAP server. > Is That correct? If you don't give us more details, no, that's not correct. Whenever a user tries to authenticate dovecot will try the 1st passdb, if it fails the 2nd and so on. I can't see how the domain is relevant here. > > I'm going to explain my particular case. > I have five Windows domains with some users who belong to those domains. I > also have other users who are authenticated with local UNIX accounts. > The only thing that I can do (I think) is using a Radius server. But, I'd > rather not use it. From philq at qsystemsengineering.com Thu Feb 14 18:58:35 2013 From: philq at qsystemsengineering.com (PhilQ) Date: Thu, 14 Feb 2013 08:58:35 -0800 (PST) Subject: [Dovecot] Requested xxxx scheme, but we have a NULL password after upgrade In-Reply-To: <1360791027564-40123.post@n4.nabble.com> References: <1360791027564-40123.post@n4.nabble.com> Message-ID: <1360861115643-40136.post@n4.nabble.com> Digging further: I did some rc.d/init.d tweaking so that I could run either Dovecot version on demand and changed the configuration so that authentication for both is basically the same. 1.07 works and 1.2.17 still doesn't. I'd really like to understand what's going on here as opposed to just dropping back and declaring it "fixed". Here is the -n output for both versions, login/mail executables and plugins are present within the configured paths for both versions: [root at Server1 init.d]# dovecot -n # 1.2.17: /usr/local/etc/dovecot.conf # OS: Linux 2.6.18-348.el5.centos.plusxen x86_64 CentOS release 5.9 (Final) log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.log protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/pki/dovecot/certs/dovecot.pem ssl_key_file: /etc/pki/dovecot/private/dovecot.pem disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login max_mail_processes: 64 mail_location: maildir:~/Maildir maildir_very_dirty_syncs: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 mail_log_max_lines_per_sec: 100 imap_client_workarounds(default): outlook-idle imap_client_workarounds(imap): outlook-idle imap_client_workarounds(pop3): lda: mail_plugin_dir: /usr/local/lib/dovecot/lda auth default: mechanisms: ntlm plain cache_size: 16 cache_ttl: 90 verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot.users passdb: driver: pam args: cache_key=%u dovecot userdb: driver: passwd [root at Server1 init.d]# dovecot107 -n # 1.0.7: /etc/dovecot.conf login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_location: maildir:~/Maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 auth default: mechanisms: ntlm plain passdb: driver: passwd-file args: /etc/dovecot.users passdb: driver: pam args: cache_key=%u dovecot userdb: driver: passwd Many thanks, - Phil -- View this message in context: http://dovecot.2317879.n4.nabble.com/Requested-xxxx-scheme-but-we-have-a-NULL-password-after-upgrade-tp40123p40136.html Sent from the Dovecot mailing list archive at Nabble.com. From philq at qsystemsengineering.com Fri Feb 15 00:57:29 2013 From: philq at qsystemsengineering.com (PhilQ) Date: Thu, 14 Feb 2013 14:57:29 -0800 (PST) Subject: [Dovecot] Requested xxxx scheme, but we have a NULL password after upgrade In-Reply-To: <1360861115643-40136.post@n4.nabble.com> References: <1360791027564-40123.post@n4.nabble.com> <1360861115643-40136.post@n4.nabble.com> Message-ID: <1360882649369-40137.post@n4.nabble.com> This is definitely a PAM-related problem. I can get authentication to work with passdb shadow and userdb passwd. Looking around, I saw something about a config option and a certain library needing to be present at compile-time to support PAM authentication but I thought I read something about an error-message related to that being shown in the log. Perhaps that error message is shown at compile-time. If anyone is able to confirm my suspicions, please let me know. Cheers, - Phil -- View this message in context: http://dovecot.2317879.n4.nabble.com/Requested-xxxx-scheme-but-we-have-a-NULL-password-after-upgrade-tp40123p40137.html Sent from the Dovecot mailing list archive at Nabble.com. From alessio at skye.it Fri Feb 15 17:15:24 2013 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 15 Feb 2013 16:15:24 +0100 Subject: [Dovecot] Dict quota timeout error: Has anything changed in the latest versions? In-Reply-To: <511A1AC8.3070506@skye.it> References: <511A1AC8.3070506@skye.it> Message-ID: <511E510C.6020408@skye.it> Il 12/02/2013 11:34, Alessio Cecchi ha scritto: > Hi, > > I'm using dict quota with mysql for track them via SQL query instead of > read maildirsize file. Primary quota and enforcing is provided via > userdb and is "maildir" type. > > Since dovecot 2.1.10-13 (but perhaps even before) when a new user log in > via POP/IMAP or receive an email for the first time dovecot created the > entry for the users in the table "quota" of dovecot database with quota > usage and number of messages (before of this login/delivery the user has > no entry in the quota table of dovecot DB). > > After the update to 2.1.14 when a new user login or receive an email for > the first time dovecot was unable to create (immediately) the entry in > quota table and in the log print these errors: Anyone have this problem? Timo, can be fixed by creating the entry in the Dict database immediately (when is not already present) instead of wait the timeout seven time? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From AxelLuttgens at swing.be Fri Feb 15 18:32:25 2013 From: AxelLuttgens at swing.be (Axel Luttgens) Date: Fri, 15 Feb 2013 17:32:25 +0100 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up Message-ID: <6A5A5394-4325-47A5-A8A6-CE433FCDD7E3@swing.be> Hello, With the same basic configs as in my previous messages, I tried with the maildir format instead of the mbox format. That is, in both cases ("virtual" uids/gids vs a single "real" uid/gid), doveconf -n shows: mail_location = maildir:~/maildir instead of: mail_location = mbox:~/mboxes:INBOX=~/mboxes/inbox Here too, with "virtual" uids/gids, connections thru pop hang forever, while imap connections behave flawlessly. And with a single "real" uid/gid, connections thru both pop and imap behave as expected. As a result, it could be inferred that the problems I am encountering with pop connections do not depend on the mailbox format in use, and that they are entirely related to "virtual" uids/gids. To recap: with this very basic config # 2.1.14: /_ETC/dovecot/dovecot.conf # OS: Darwin 12.2.1 x86_64 auth_verbose = yes disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mboxes:INBOX=~/mboxes/inbox passdb { args = /_ETC/dovecot/db.conf driver = sql } ssl = no userdb { driver = prefetch } a pop connection hangs forever after the pass command $ telnet 127.0.0.1 110 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user user2 +OK pass pass2 and the log shows (here, when the user connects for the first time) dovecot[2997]: pop3-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=3113, secured, session= dovecot[2997]: pop3(user2): Debug: Effective uid=100003, gid=100003, home=/_Data/Mailstores/100003 dovecot[2997]: pop3(user2): Debug: Home dir not found: /_Data/Mailstores/100003 dovecot[2997]: pop3(user2): Debug: fs: root=/_Data/Mailstores/100003/mboxes, index=, control=, inbox=/_Data/Mailstores/100003/mboxes/inbox, alt= dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes doesn't exist yet, using default permissions dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes/inbox doesn't exist yet, using default permissions dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes/inbox doesn't exist yet, using default permissions dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 With exactly the same config, imap connections are fully functional. Does someone have an explanation? TIA, Axel From ncjeffgus at zimage.com Fri Feb 15 23:02:56 2013 From: ncjeffgus at zimage.com (Jeff Gustafson) Date: Fri, 15 Feb 2013 13:02:56 -0800 Subject: [Dovecot] accumulating orphaned processes Message-ID: <1360962176.11085.4.camel@maclinux> I'm getting orphaned processes on our mail servers. Some machines has processes hanging around from a week ago. I'm running the current stable of dovecot 2.1.15, but the problem has been happening for a while now. I thought it would be good to report the issue. I ran an strace on one of them and this is the output: Process 21370 attached - interrupt to quit open("/var/run/dovecot/stats-mail", O_WRONLY) = 7 write(7, "DISCONNECT\ta09ade0350d81d517a530"..., 44) = 44 munmap(0x7f1ad8a8d000, 266240) = 0 close(13) = 0 close(12) = 0 epoll_ctl(9, EPOLL_CTL_DEL, 11, {0, {u32=17384128, u64=17384128}}) = 0 close(11) = 0 close(7) = 0 munmap(0x7f1ada5d6000, 2106272) = 0 munmap(0x7f1ada7d9000, 2161360) = 0 munmap(0x7f1ada3cf000, 2124136) = 0 munmap(0x323c800000, 2162768) = 0 munmap(0x7f1ad9fc8000, 2102976) = 0 munmap(0x7f1ada1ca000, 2114728) = 0 epoll_ctl(9, EPOLL_CTL_DEL, 4, {0, {u32=17319120, u64=17319120}}) = 0 close(4) = 0 close(8) = 0 close(9) = 0 exit_group(0) = ? Process 21370 detached As you can see, the process exited at that point. Any ideas? ...Jeff From siefke_listen at web.de Sat Feb 16 05:13:10 2013 From: siefke_listen at web.de (Silvio Siefke) Date: Sat, 16 Feb 2013 04:13:10 +0100 Subject: [Dovecot] Virtual Users Dovecot and Postfix Message-ID: <20130216041310.f5e6fb2097db6e8e776526ad@web.de> Hello, i want setup my Mailserver with the Tutorial on Dovecot Wiki. http://wiki2.dovecot.org/HowTo/VirtualUserFlatFilesPostfix I have one question for this, must i set the Mailadresses in /etc/postfix/virtual too or is the passwd File from dovecot enough? Thank you for help & Greetings Silvio From ben at morrow.me.uk Sat Feb 16 05:49:37 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Sat, 16 Feb 2013 03:49:37 +0000 Subject: [Dovecot] Virtual Users Dovecot and Postfix In-Reply-To: <20130216041310.f5e6fb2097db6e8e776526ad@web.de> References: <20130216041310.f5e6fb2097db6e8e776526ad@web.de> Message-ID: <20130216034936.GA8029@anubis.morrow.me.uk> At 4AM +0100 on 16/02/13 you (Silvio Siefke) wrote: > > i want setup my Mailserver with the Tutorial on Dovecot Wiki. > > http://wiki2.dovecot.org/HowTo/VirtualUserFlatFilesPostfix > > I have one question for this, must i set the Mailadresses in > /etc/postfix/virtual too or is the passwd File from dovecot > enough? The setup given in that howto will accept mail for any user at one of the domains listed in virtual_mailbox_domains, and mail for nonexistent users will then be bounced by Dovecot. This is a very bad idea, since you'll end up becoming a backscatter source; you should set up a separate Postfix table listing the valid users at those domains, and put that table in virtual_mailbox_maps. (You don't want to use /etc/postfix/virtual for this, you want a separate table.) See the Postfix ADDRESS_CLASS_README. It might instead be possible to use the facilities described in the ADDRESS_VERIFICATION_README to verify addresses with the LMTP server rather than consulting a preprepared list, but IMHO that approach is wasteful and not entirely reliable. (IMHO you should also use smtpd_restriction_classes to get the smtpd_*_restrictions stuff out of master.cf, but that's a separate issue.) Ben From tss at iki.fi Sat Feb 16 06:01:36 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 06:01:36 +0200 Subject: [Dovecot] accumulating orphaned processes In-Reply-To: <1360962176.11085.4.camel@maclinux> References: <1360962176.11085.4.camel@maclinux> Message-ID: On 15.2.2013, at 23.02, Jeff Gustafson wrote: > I'm getting orphaned processes on our mail servers. Some machines has > processes hanging around from a week ago. I'm running the current stable > of dovecot 2.1.15, but the problem has been happening for a while now. I > thought it would be good to report the issue. > I ran an strace on one of them and this is the output: > > Process 21370 attached - interrupt to quit .. > exit_group(0) = ? > Process 21370 detached > > As you can see, the process exited at that point. Any ideas? Try getting gdb backtrace instead? gdb -p bt full Also doveconf -n output could be useful. From tlx at leuxner.net Sat Feb 16 09:22:55 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 16 Feb 2013 08:22:55 +0100 Subject: [Dovecot] Virtual Users Dovecot and Postfix In-Reply-To: <20130216034936.GA8029@anubis.morrow.me.uk> References: <20130216041310.f5e6fb2097db6e8e776526ad@web.de> <20130216034936.GA8029@anubis.morrow.me.uk> Message-ID: <20130216072255.GA5171@nihlus.leuxner.net> * Ben Morrow 2013.02.16 04:49: > > http://wiki2.dovecot.org/HowTo/VirtualUserFlatFilesPostfix > > The setup given in that howto will accept mail for any user at one of > the domains listed in virtual_mailbox_domains, and mail for nonexistent > users will then be bounced by Dovecot. This is a very bad idea, since > you'll end up becoming a backscatter source; you should set up a > separate Postfix table listing the valid users at those domains, and put > that table in virtual_mailbox_maps. (You don't want to use > /etc/postfix/virtual for this, you want a separate table.) It will verify the recipients before accepting the mail: reject_unverified_recipient http://www.postfix.org/postconf.5.html#reject_unverified_recipient I don't think the overhead of duplicating user checks on both ends is justified until this process is automated. Above will instruct the SMTP Server to verify the recipient address before accepting mail. This is what you want and it works dynamic out of the box. A better reason to use separate aliasing on the Postfix side is to rewrite addresses for the Dovecot Mailboxes. If you'd want to have a mailbox with several addresses in Dovecot you'd need to create dummy users for this while you can just alias them in Postfix. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ben at morrow.me.uk Sat Feb 16 10:26:17 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Sat, 16 Feb 2013 08:26:17 +0000 Subject: [Dovecot] Virtual Users Dovecot and Postfix In-Reply-To: <20130216072255.GA5171@nihlus.leuxner.net> References: <20130216041310.f5e6fb2097db6e8e776526ad@web.de> <20130216034936.GA8029@anubis.morrow.me.uk> <20130216072255.GA5171@nihlus.leuxner.net> Message-ID: <20130216082616.GB8029@anubis.morrow.me.uk> At 8AM +0100 on 16/02/13 you (Thomas Leuxner) wrote: > * Ben Morrow 2013.02.16 04:49: > > > > http://wiki2.dovecot.org/HowTo/VirtualUserFlatFilesPostfix > > > > The setup given in that howto will accept mail for any user at one of > > the domains listed in virtual_mailbox_domains, and mail for nonexistent > > users will then be bounced by Dovecot. This is a very bad idea, since > > you'll end up becoming a backscatter source; you should set up a > > separate Postfix table listing the valid users at those domains, and put > > that table in virtual_mailbox_maps. (You don't want to use > > /etc/postfix/virtual for this, you want a separate table.) > > It will verify the recipients before accepting the mail: > > reject_unverified_recipient > http://www.postfix.org/postconf.5.html#reject_unverified_recipient Hang on; the first listing in the howto uses that option, the second doesn't. I was looking at the second, as generally the cleaner. Hmmph. In any case, I referred to that option, but I don't really think it's a good idea. It seems like unnecessary overhead just to avoid building an extra map. > I don't think the overhead of duplicating user checks on both ends is > justified until this process is automated. The SMTP server has to make user checks of some sort. reject_unverified_recipient makes those checks more expensive, not less, since some of the time it will have to call out to the LMTP server to verify an address. Building a Postfix map from a passwd-format file is trivial. It can easily be incorporated into whatever script is used for adding and removing users already, or into /etc/postfix/Makefile or whatever is used for rebuilding the Postfix maps. > Above will instruct the > SMTP Server to verify the recipient address before accepting mail. > This is what you want and it works dynamic out of the box. A better > reason to use separate aliasing on the Postfix side is to rewrite > addresses for the Dovecot Mailboxes. If you'd want to have a mailbox > with several addresses in Dovecot you'd need to create dummy users for > this while you can just alias them in Postfix. This would be done with virtual aliases in Postfix, which are separate from the virtual mailboxes which end up delivered to Dovecot. (In fact, under many circumstances, it's more convenient to use local_transport for delivery to Dovecot.) Ben From chris.laif at googlemail.com Sat Feb 16 13:44:00 2013 From: chris.laif at googlemail.com (Chris Laif) Date: Sat, 16 Feb 2013 12:44:00 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) Message-ID: Hi, I would like to add a feature to my email-client (Thunderbird) which stops emails from showing up until a specific time (e.g. having buttons for: show again this evening, tomorrow, next week, next month ...). As far as I've learned this is not possible with IMAP: - storing the 'do-not-show-until-time' within a custom header-field requires down- and uploading of the whole email and therefore causes problems especially with large emails - IMAP keywords are limited are limited in quantity and can not be compared with datetime functions Any suggestions will be greatly appreciated! Chris From h.reindl at thelounge.net Sat Feb 16 13:49:17 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 16 Feb 2013 12:49:17 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: Message-ID: <511F723D.2030107@thelounge.net> Am 16.02.2013 12:44, schrieb Chris Laif: > Hi, I would like to add a feature to my email-client (Thunderbird) > which stops emails from showing up until a specific time (e.g. having > buttons for: show again this evening, tomorrow, next week, next month > ...). > > As far as I've learned this is not possible with IMAP: > - storing the 'do-not-show-until-time' within a custom header-field > requires down- and uploading of the whole email and therefore causes > problems especially with large emails sounds more like a social problem then a technical touching the e-mails is a very very bad idea because you may break them what does it help DOWNLOAD the mails, hide them and touch them you said "I would like to add a feature to my email-client" so why do you need to add headers to them? what has the server to do with this all and why would you touch the mails for this? if the user is not ready to deal with mails -> close TB -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From rs at sys4.de Sat Feb 16 14:19:45 2013 From: rs at sys4.de (Robert Schetterer) Date: Sat, 16 Feb 2013 13:19:45 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: Message-ID: <511F7961.6070202@sys4.de> Am 16.02.2013 12:44, schrieb Chris Laif: > Hi, I would like to add a feature to my email-client (Thunderbird) > which stops emails from showing up until a specific time (e.g. having > buttons for: show again this evening, tomorrow, next week, next month > ...). yeah ,there are lot of nice new ideas with mail these days > > As far as I've learned this is not possible with IMAP: > - storing the 'do-not-show-until-time' within a custom header-field > requires down- and uploading of the whole email and therefore causes > problems especially with large emails > - IMAP keywords are limited are limited in quantity and can not be > compared with datetime functions > > Any suggestions will be greatly appreciated! however some server coders will may include new features, that does not solve the client problem, the only existing free major cross os imap client recent is thunderbird, but i dont see mozilla willing to push more development in it. After all ,there is lot of "old" bleeding edge stuff for left implement in Thunderbird, before including new ideas. > > Chris > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From chris.laif at googlemail.com Sat Feb 16 14:46:49 2013 From: chris.laif at googlemail.com (Chris Laif) Date: Sat, 16 Feb 2013 13:46:49 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: <511F723D.2030107@thelounge.net> References: <511F723D.2030107@thelounge.net> Message-ID: On Sat, Feb 16, 2013 at 12:49 PM, Reindl Harald wrote: > you said "I would like to add a feature to my email-client" so why > do you need to add headers to them? what has the server to do with > this all and why would you touch the mails for this? > People are accessing their (IMAP-)inboxes from multiple computers/devices these days, so all information has to be stored on the server. Chris From h.reindl at thelounge.net Sat Feb 16 14:51:01 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 16 Feb 2013 13:51:01 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: <511F723D.2030107@thelounge.net> Message-ID: <511F80B5.8070308@thelounge.net> Am 16.02.2013 13:46, schrieb Chris Laif: > On Sat, Feb 16, 2013 at 12:49 PM, Reindl Harald wrote: >> you said "I would like to add a feature to my email-client" so why >> do you need to add headers to them? what has the server to do with >> this all and why would you touch the mails for this? >> > > People are accessing their (IMAP-)inboxes from multiple > computers/devices these days, so all information has to be stored on > the server. sorry but this all is nonsense and the context "i want to implement a feature in TB but multiple devices" it makes no sense if you do not want to see new mails do not check for them - no need to mangle messages switch TB in the offline-mode or clsoe the mail-client based on WHAT will wahtever you think you need to implement dec ide "if the user is ready" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From rs at sys4.de Sat Feb 16 14:57:46 2013 From: rs at sys4.de (Robert Schetterer) Date: Sat, 16 Feb 2013 13:57:46 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: <511F723D.2030107@thelounge.net> Message-ID: <511F824A.8020609@sys4.de> Am 16.02.2013 13:46, schrieb Chris Laif: > On Sat, Feb 16, 2013 at 12:49 PM, Reindl Harald wrote: >> you said "I would like to add a feature to my email-client" so why >> do you need to add headers to them? what has the server to do with >> this all and why would you touch the mails for this? >> > > People are accessing their (IMAP-)inboxes from multiple > computers/devices these days, so all information has to be stored on > the server. > > Chris > perhaps http://tools.ietf.org/html/rfc3864 may helpfull Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Sat Feb 16 16:50:52 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 16:50:52 +0200 Subject: [Dovecot] Broken physical size caching in Dovecot 2.1.10 In-Reply-To: <511B7DAF.30109@heartinternet.co.uk> References: <79BDD400309473478C82105B02CD6AA6112D9F@EXCH2010.heartoffice.co.uk> <1360030002.10326.52.camel@innu> <511B7DAF.30109@heartinternet.co.uk> Message-ID: <1361026252.3230.58.camel@hurina> On Wed, 2013-02-13 at 11:49 +0000, Rob Redpath wrote: > > Dovecot's zlib plugin requires that the maildir filenames > > have ,S=uncompressed size. Otherwise you get those errors. > > http://dovecot.org/tools/maildir-size-check.sh can be used to fix the > > filenames. > > > Thanks for this. The problem I'm encountering is that Dovecot renames > files, and does so to their size on disk rather than their uncompressed > size, undoing any corrective action that's been taken. Yeah. It's a bit difficult to change the automated fixing code to work properly with compressed files without some very ugly kludges. BTW. http://dovecot.org/tools/maildir-size-fix.pl has been updated to work with compressed files also, making maildir-size-check.sh obsolete. From tss at iki.fi Sat Feb 16 17:20:22 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 17:20:22 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> Message-ID: <1361028022.3230.61.camel@hurina> I did a bunch of dsync fixes today in hg. With the new locking behavior (and other fixes) you shouldn't be able to break it anymore. On Fri, 2013-02-01 at 21:53 +0100, Michael Grimm wrote: > [Sorry Oli for my previous mail to your address, only. Resent here] > > Oli Schacher wrote: > > > There still seems to be a problem when changes to both mailboxes at > > the same time are involved > > I can confirm your observation, although triggered by a different test > scenario, similar to the one I did use with 2.1 replicator before > (http://www.dovecot.org/list/dovecot/2012-March/064354.html). > > This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test" > at both servers "mx1" and "mx2", and replicator uses ssh for remote > access. Both servers run a recent postfix, use lmtp for local delivery, > and "test" is a virtual user. > > Test script to produce local testmails of equal size at mx1: > | #!/bin/csh > | set INDEX = 101 > | set endINDEX = 200 > | while ( $INDEX <= $endINDEX ) > | echo $INDEX > | echo "test" | mail -s $INDEX test at mx1 > | if ( $INDEX % 1000 == 0 ) then > | sleep 1 > | endif > | @ INDEX = $INDEX + 1 > |end > |exit 0 > > Test script to produce testmails of equal size at mx2: > | #!/bin/csh > | set INDEX = 1101 > | set endINDEX = 1200 > | while ( $INDEX <= $endINDEX ) > | echo $INDEX > | echo "test" | mail -s $INDEX test at mx2 > | if ( $INDEX % 1000 == 0 ) then > | sleep 1 > | endif > | @ INDEX = $INDEX + 1 > |end > |exit 0 > > All tests are run with vanilla mailboxes, after restarting dovecot, and > without imap connections by MUA: > > 1) Simultaneous mailbomb approach: run both scripts simultaneously, and > you'll end up with numerous duplicates in mailboxes "test". Very often > you'll find multiples. > > 2) Mailbomb approach: run one script at one server only, and all mails > will become perfectly well synchronised. > > 3) Mofify both scripts to "( $INDEX % 1 == 0 )" to add a second waiting > between every mail injection, and run them simultaneously at both > servers, and you'll end up with significantly less duplicates and no > more multiples. > > > Feb 1 07:12:52 doco1 dovecot: dsync-local(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=7a30ff22af5b0b510f0c0000960042f4 (UID=211) > > > Feb 1 07:12:54 doco2 dovecot: dsync-local(user1): Error: Importing mailbox INBOX failed > > > Feb 1 07:13:24 doco2 dovecot: dsync-local(user1): Error: Remote command process isn't dying, killing it > > I do see those error messages as well, and in addition numerous of those: > > | dovecot: dsync-local(test): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=7153: 82c5df0a4ffa0b5141e300006a0d5a02 != 29cc9f284ffa0b5141c2000036abecbd > > | doveadm: Error: dsync-remote(test): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=7153: 82c5df0a4ffa0b5141e300006a0d5a02 != 29cc9f284ffa0b5141c2000036abecbd > > | dovecot: lmtp(49752, test): Error: Corrupted index cache file /.../test/mailboxes/INBOX/dbox-Mails/dovecot.index.cache: File too small > > | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 > | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 > | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: mdbox /.../test/mailboxes/INBOX/dbox-Mails: Storage keeps breaking > | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Error: Mailbox INBOX: Corrupted index, uidvalidity=0 > | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:16 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes > | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes > | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:17 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes > | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes > | Feb 1 18:35:18 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: mdbox /.../test/storage: rebuilding indexes > | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Warning: fscking index file /.../test/storage/dovecot.map.index > | Feb 1 18:35:27 mx1 dovecot: imap(test) BXeiKq3UBgBd3DLy: Disconnected: Logged out in=425 out=1100 > > JFYI, and regards, > Michael > From rs at sys4.de Sat Feb 16 18:16:15 2013 From: rs at sys4.de (Robert Schetterer) Date: Sat, 16 Feb 2013 17:16:15 +0100 Subject: [Dovecot] doveadm -D quota get -u user Message-ID: <511FB0CF.6050902@sys4.de> Hi Timo, just running into problems with doveadm -D quota get -u user doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_autocreate_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_expire_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_listescape_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_lucene_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so doveadm(root): Fatal: Unknown command 'quota', but plugin quota exists. Try to set mail_plugins=quota should mail_plugins=quota really be set ? dovecot --version 2.1.12 (abcc9959b757) Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From daniel.parthey at informatik.tu-chemnitz.de Sat Feb 16 18:27:17 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 16 Feb 2013 17:27:17 +0100 Subject: [Dovecot] doveadm -D quota get -u user In-Reply-To: <511FB0CF.6050902@sys4.de> References: <511FB0CF.6050902@sys4.de> Message-ID: <20130216162717.GA8322@daniel.localdomain> Hi Robert, Robert Schetterer wrote: > dovecot --version > 2.1.12 (abcc9959b757) > > just running into problems with > > doveadm -D quota get -u user > doveadm(root): Fatal: Unknown command 'quota', but plugin quota exists. > Try to set mail_plugins=quota > > should mail_plugins=quota really be set ? Sure, at least in the global setting, might be different for protocol imap or protocol lmtp. http://wiki2.dovecot.org/Tools/Doveadm/Quota The quota get and quota recalc commands are only available when the global mail_plugins setting contains the quota plugin. Regards Daniel -- https://plus.google.com/103021802792276734820 From tss at iki.fi Sat Feb 16 18:29:21 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 18:29:21 +0200 Subject: [Dovecot] Dsync converting dbox->Maildir In-Reply-To: References: Message-ID: <1361032161.3230.63.camel@hurina> On Wed, 2013-02-13 at 12:28 +0400, Anes Mukhametov wrote: > I've changed my mailbox format from dbox to Maildir. The problem is maildir > message filename extension has no S=/W= field after migration. > Using dovecot 2.1.15. How did you change it? dsync is the only tool that can properly do it, and it definitely adds the S/W fields. Anyway, Dovecot can handle maildir files without S/W files anyway. But if you really want to add them, use http://dovecot.org/tools/maildir-size-fix.pl with -a parameter. It adds only S though, no W support yet. From tss at iki.fi Sat Feb 16 18:36:48 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 18:36:48 +0200 Subject: [Dovecot] dovecot-2.2: dsync to imapc not working In-Reply-To: <5118A6D7.9090206@yandex.ru> References: <5111F388.9030305@yandex.ru> <1360538149.10326.70.camel@innu> <5118A6D7.9090206@yandex.ru> Message-ID: <1361032608.3230.65.camel@hurina> On Mon, 2013-02-11 at 12:07 +0400, Evgeny Basov wrote: > 11.02.2013 03:15, Timo Sirainen ?????: > > Should work better now in latest hg. > > # dovecot --version > 20130211 (ed5fa76f0dc7+) > > Thanks. Work better but not fine. Not all emails was syncrornized: I did several dsync fixes today. See if it happens to work now? "doveadm backup -R imapc:" works at least in my quick tests. From tss at iki.fi Sat Feb 16 18:43:19 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 18:43:19 +0200 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: Message-ID: <1361032999.3230.69.camel@hurina> On Sat, 2013-02-16 at 12:44 +0100, Chris Laif wrote: > Hi, I would like to add a feature to my email-client (Thunderbird) > which stops emails from showing up until a specific time (e.g. having > buttons for: show again this evening, tomorrow, next week, next month > ...). > > As far as I've learned this is not possible with IMAP: > - storing the 'do-not-show-until-time' within a custom header-field > requires down- and uploading of the whole email and therefore causes > problems especially with large emails > - IMAP keywords are limited are limited in quantity and can not be > compared with datetime functions http://tools.ietf.org/html/rfc5257 could be used for this. Of course, Dovecot doesn't support it, and it's not easy to implement efficiently. Probably not too difficult to implement inefficiently with v2.2's mailbox_attribute_* API. Just create /annotations// mailbox attributes and delete them when the mails get expunged.. From tss at iki.fi Sat Feb 16 18:47:25 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 18:47:25 +0200 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up In-Reply-To: <6A5A5394-4325-47A5-A8A6-CE433FCDD7E3@swing.be> References: <6A5A5394-4325-47A5-A8A6-CE433FCDD7E3@swing.be> Message-ID: <1361033245.3230.72.camel@hurina> On Fri, 2013-02-15 at 17:32 +0100, Axel Luttgens wrote: > a pop connection hangs forever after the pass command > > $ telnet 127.0.0.1 110 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > +OK Dovecot ready. > user user2 > +OK > pass pass2 > > and the log shows (here, when the user connects for the first time) > > dovecot[2997]: pop3-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=3113, secured, session= > dovecot[2997]: pop3(user2): Debug: Effective uid=100003, gid=100003, home=/_Data/Mailstores/100003 > dovecot[2997]: pop3(user2): Debug: Home dir not found: /_Data/Mailstores/100003 > dovecot[2997]: pop3(user2): Debug: fs: root=/_Data/Mailstores/100003/mboxes, index=, control=, inbox=/_Data/Mailstores/100003/mboxes/inbox, alt= > dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes doesn't exist yet, using default permissions > dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 > dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes/inbox doesn't exist yet, using default permissions > dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 > dovecot[2997]: pop3(user2): Debug: Namespace : /_Data/Mailstores/100003/mboxes/inbox doesn't exist yet, using default permissions > dovecot[2997]: pop3(user2): Debug: Namespace : Using permissions from /_Data/Mailstores/100003/mboxes: mode=0700 gid=-1 > > With exactly the same config, imap connections are fully functional. > > Does someone have an explanation? a) dtruss the pop3 process to see which syscall hangs. Maybe this works: service pop3 { executable = /usr/bin/dtruss /usr/local/libexec/dovecot/pop3 } It logs to Dovecot's error log. b) get gdb backtrace: gdb -p bt full From chris.laif at googlemail.com Sat Feb 16 18:50:21 2013 From: chris.laif at googlemail.com (Chris Laif) Date: Sat, 16 Feb 2013 17:50:21 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: <511F80B5.8070308@thelounge.net> References: <511F723D.2030107@thelounge.net> <511F80B5.8070308@thelounge.net> Message-ID: > if you do not want to see new mails do not check for > them - no need to mangle messages > > switch TB in the offline-mode or clsoe the mail-client > based on WHAT will wahtever you think you need to implement > dec ide "if the user is ready" > Sorry, but that's not what I meant. After reading a message, I would like to decide that this very specific message should disappear from my mailbox for some time (I'am not able to handle/answer this email today, so it should reappear tomorrow/next week/...). I agree that this transforms email into a kind of todo/task-list. But many people I know are using email-accounts this way. Gmail and mailboxapp.com (and probably more) offer similar features, just google for 'snooze email'. Very likely it is not necessary to modifiy Thunderbird (or any other email-client) for this. You just have to set up a filter folder 'do-not-show-until-time'>now(). Chris From h.reindl at thelounge.net Sat Feb 16 18:54:14 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 16 Feb 2013 17:54:14 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: <511F723D.2030107@thelounge.net> <511F80B5.8070308@thelounge.net> Message-ID: <511FB9B6.6020104@thelounge.net> Am 16.02.2013 17:50, schrieb Chris Laif: >> if you do not want to see new mails do not check for >> them - no need to mangle messages >> >> switch TB in the offline-mode or clsoe the mail-client >> based on WHAT will wahtever you think you need to implement >> dec ide "if the user is ready" >> > > Sorry, but that's not what I meant. After reading a message, I would > like to decide that this very specific message should disappear from > my mailbox for some time (I'am not able to handle/answer this email > today, so it should reappear tomorrow/next week/...) and this is why you would touch the message-headers? what the hell - IMAP supports folders and subfolders under my inbox are 30 subfolders and i get between 300 and 500 messages per day with a usual response time of 5 minutes as you can see becuase some of them are important and some of them are archived for reminders -> thunderbird lightning as said - this is a social and not a technical problem -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Sat Feb 16 19:03:32 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 19:03:32 +0200 Subject: [Dovecot] Segfault with doveadm-server In-Reply-To: <511CAB5F.8050502@yandex.ru> References: <511CAB5F.8050502@yandex.ru> Message-ID: <1361034212.3230.74.camel@hurina> On Thu, 2013-02-14 at 13:16 +0400, Evgeny Basov wrote: > Hello. > > With dovecot ver. from 20130211 I have a segfault on doveadm when sync. > There are messages from server: > > 2013-02-14T12:42:35.508031+04:00 imap1 kernel: [870542.927083] > doveadm-server[9483]: segfault at 4 ip 00000074b228bc24 sp > 000003f30b8e61b0 error 4 in doveadm-server[74b226b000+3a000] > > I cannot catch process doveadm-server by strace because it starts with > unknown PID on demand. strace isn't useful for debugging crashes, gdb backtrace is. You should be able to get a core dump from doveadm-server (I've definitely been debugging those today many times). http://dovecot.org/bugreport.html Anyway, the crash is probably fixed by today's changes. From tss at iki.fi Sat Feb 16 19:05:23 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 19:05:23 +0200 Subject: [Dovecot] proxy imaps In-Reply-To: <511CB2E9.4060402@ual.es> References: <511CB2E9.4060402@ual.es> Message-ID: <1361034323.3230.75.camel@hurina> On Thu, 2013-02-14 at 10:48 +0100, Antonio Casado Rodriguez wrote: > Hi all, > > I had a imaps dovecot server working normal. I wanted to isolate the > Internet connection by placing a proxy imap(Intranet connections still > connect directly to the imap server). > Yesterday I set up oneproxy imaps dovecot and thereafter, imap dovecot > logs on errors: > > imap-login: Error: OpenSSL malloc() failed. You may need to increase > login_process_size > > Need dovecot more memory with imap proxy connections than direct > connections? Yes, SSL needs more memory. The error message is obsolete though, it should have said to increase service imap-login { vsz_limit }. Fixed in hg. From tss at iki.fi Sat Feb 16 19:10:36 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 19:10:36 +0200 Subject: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients In-Reply-To: <511C1701.3070304@active24.pl> References: <511B8DB5.10105@active24.pl> <511BB23A.3050004@active24.pl> <511C1701.3070304@active24.pl> Message-ID: <1361034636.3230.78.camel@hurina> On Wed, 2013-02-13 at 23:43 +0100, Piotr Rotter wrote: > I configure director temporary in some tricky way because I have to > serve md5 and ntlm authentication method and I do not want reconfigure > pop/imap servers. > > Can you confirm I done all ok becouse I have that issue: > > # doveadm director map > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > user All is working. If you want "doveadm director map" command to work you need to either: a) Make "doveadm user '*'" command work by providing appropriate userdb configuration or b) use doveadm director map -f userdb parameter where userdb is a file that lists all the usernames you have. There's no need to use doveadm director map command at all though. From tss at iki.fi Sat Feb 16 19:12:34 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 19:12:34 +0200 Subject: [Dovecot] "Renaming not supported across conflicting directory permissions" In-Reply-To: <04C4B084-A082-443E-A54A-1E7A147EC3BE@tucows.com> References: <04C4B084-A082-443E-A54A-1E7A147EC3BE@tucows.com> Message-ID: <1361034754.3230.80.camel@hurina> On Wed, 2013-02-13 at 13:04 -0500, Richard Platel wrote: > Is it possible to disable this check in 2.1.7? (without a patch and re-compile?) Nope. I know there are some fixes related to that check in a newer v2.1. If it doesn't work with v2.1.15 either the way you want, let me know what exactly is the problem. From mstevens at imt-systems.com Sat Feb 16 19:21:43 2013 From: mstevens at imt-systems.com (Morten Stevens) Date: Sat, 16 Feb 2013 18:21:43 +0100 Subject: [Dovecot] dovecot index errors since 2.1.12 In-Reply-To: <1359475051.12608.55.camel@hurina> References: <50D0E224.2050508@imt-systems.com> <20130114151644.GA31122@daniel.localdomain> <1358865535.12608.28.camel@hurina> <1359475051.12608.55.camel@hurina> Message-ID: <511FC027.8030307@imt-systems.com> On 29.01.2013 16:57, Timo Sirainen wrote: > On Tue, 2013-01-22 at 16:38 +0200, Timo Sirainen wrote: >> On Mon, 2013-01-14 at 16:16 +0100, Daniel Parthey wrote: >>> Jan 12 16:48:35 10.129.3.233 dovecot: mailbox: mail: >>> imap(user1 at example.org): : Error: Cached message >>> size larger than expected (5194 > 4399) >> >> I wonder if this patch causes your lmtp/lda processes to start >> assert-crashing (which at least with lmtp causes MTA to retry, with lda >> might reject mail depending on config): >> http://hg.dovecot.org/dovecot-2.1/rev/2b76d357a56a >> >> Since the bug appears to be in the mail delivery part, it would be much >> easier to debug&fix it by looking at what goes wrong there. > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/0b0399f1b6aa Hi Timo, Thank you. I can confirm the bug has been fixed since 2.1.14. Best regards, Morten From chris.laif at googlemail.com Sat Feb 16 19:33:22 2013 From: chris.laif at googlemail.com (Chris Laif) Date: Sat, 16 Feb 2013 18:33:22 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: <1361032999.3230.69.camel@hurina> References: <1361032999.3230.69.camel@hurina> Message-ID: On Sat, Feb 16, 2013 at 5:43 PM, Timo Sirainen wrote: >> As far as I've learned this is not possible with IMAP: >> - storing the 'do-not-show-until-time' within a custom header-field >> requires down- and uploading of the whole email and therefore causes >> problems especially with large emails >> - IMAP keywords are limited are limited in quantity and can not be >> compared with datetime functions > > http://tools.ietf.org/html/rfc5257 could be used for this. Of course, > Dovecot doesn't support it, and it's not easy to implement efficiently. > > Probably not too difficult to implement inefficiently with v2.2's > mailbox_attribute_* API. Just create /annotations// mailbox > attributes and delete them when the mails get expunged.. > Thanks Timo, at least one person understands me ;-) Unfortunately, both solutions require programming skills far better than I have. So I start enjoying my weekend doing other things ... thank you for your great work on dovecot! Chris From rs at sys4.de Sat Feb 16 19:48:31 2013 From: rs at sys4.de (Robert Schetterer) Date: Sat, 16 Feb 2013 18:48:31 +0100 Subject: [Dovecot] doveadm -D quota get -u user In-Reply-To: <20130216162717.GA8322@daniel.localdomain> References: <511FB0CF.6050902@sys4.de> <20130216162717.GA8322@daniel.localdomain> Message-ID: <511FC66F.4010700@sys4.de> Am 16.02.2013 17:27, schrieb Daniel Parthey: > Hi Robert, > > Robert Schetterer wrote: >> dovecot --version >> 2.1.12 (abcc9959b757) >> >> just running into problems with >> >> doveadm -D quota get -u user >> doveadm(root): Fatal: Unknown command 'quota', but plugin quota exists. >> Try to set mail_plugins=quota >> >> should mail_plugins=quota really be set ? > > Sure, at least in the global setting, > might be different for protocol imap or protocol lmtp. > > http://wiki2.dovecot.org/Tools/Doveadm/Quota > > The quota get and quota recalc commands are only available when the global > mail_plugins setting contains the quota plugin. guilty, thx , i overread this it works now > > Regards > Daniel > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From daniel.parthey at informatik.tu-chemnitz.de Sat Feb 16 19:56:08 2013 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 16 Feb 2013 18:56:08 +0100 Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: <511F723D.2030107@thelounge.net> <511F80B5.8070308@thelounge.net> Message-ID: <20130216175608.GA9333@daniel.localdomain> Hi Chris, Chris Laif wrote: > After reading a message, I would like to decide that this very specific > message should disappear from my mailbox for some time (I'am not able to > handle/answer this email today, so it should reappear tomorrow/next > week/...). Well, I simply move some mail into a folder called "Todo" and as soon as I've got time, handle all mails in this folder. You could even write a script, which marks the messages in the Todo folder via IMAP as "new" or "unread" after a specified time, but this is rather a client-side issue and should not be handled by Dovecot. Regards Daniel -- https://plus.google.com/103021802792276734820 From tss at iki.fi Sat Feb 16 20:19:31 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 20:19:31 +0200 Subject: [Dovecot] Private message flags in shared mailboxes Message-ID: <0D916D2F-C59F-4C77-8887-34BFED364A00@iki.fi> I'm wondering a bit how these should work. Comments welcome. In v2.2 the recommended way to enable private flags for shared mailboxes is to create private indexes with :INDEXPVT=. This creates dovecot.index.pvt[.log] files which only contain message UIDs and flags. The indexes are updated somewhat lazily by updating them whenever user has the shared mailbox opened and it gets synced. 1. What would be a good place to configure which flags are shared and which are private? Currently it forces \Seen flag to be private and others not. With Maildir I used to have the idea of configuring these in dovecot-shared file, but that's a rather annoying extra file. Then again it could be cached into dovecot.index. Another possibility would be in dovecot-acl file, but public mailboxes might be enabled without ACL plugin. Mailbox attributes (for URLAUTH and for future METADATA) also exist in v2.2, but they require configuring a dict for them, which isn't very efficient either to read every time a mailbox is accessed. Mailbox's admin user should be able to do this somehow via IMAP protocol also. METADATA would pretty much be required for that I guess. 2. Private flags aren't currently inherited from the primary (shared) flags. With user-shared mailboxes this is good, because the owner doesn't have private indexes for the mailboxes, so when owner reads a mail it gets a \Seen flag in the primary index. If that mailbox was shared to someone else, the \Seen flag shouldn't be set. But this also means that for truly shared (public) mailboxes that have no owner you can't set initial flags with Sieve's addflag command (e.g. \Seen flag to high-scored mails in Spam). I guess it could be possible to change this so that the inheritance is done for public namespaces but not for shared namespaces, but is that really a good idea either?.. 3. Currently COPY/APPEND ignore any private flags (which also means dsync won't set them always). Fixing this would require remembering these flags, committing the save to primary index, syncing the private index to get the new mails added there, then updating their flags (or doing it directly at sync stage with some kludging code). To avoid race conditions the private index would need to be locked before committing the save. I guess this is doable.. 4. Private keywords aren't supported. I guess not much worse than system flags, but more code complexity. Wonder if anyone would care about them. From trashcan at odo.in-berlin.de Sat Feb 16 20:26:50 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 16 Feb 2013 19:26:50 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <1361028022.3230.61.camel@hurina> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> Message-ID: Timo Sirainen wrote: > I did a bunch of dsync fixes today in hg. With the new locking > behavior (and other fixes) you shouldn't be able to break it anymore. Sorry to say, but I am still able to break replicator with v2.2.beta1 (35194cf0693e) under the conditions outlined below. > On 2013-02-01 Michael Grimm wrote: > >> This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test" >> at both servers "mx1" and "mx2", and replicator uses ssh for remote >> access. Both servers run a recent postfix, use lmtp for local delivery, >> and "test" is a virtual user. I might add that both servers run inside FreeBSD jails (if that might make the difference to your test setup. >> All tests are run with vanilla mailboxes, after restarting dovecot, and >> without imap connections by MUA: This time I did even restart both service jails before every test. And, I did use both Mail.app and roundcube as MUA to check the results (if Mail.app might have screwed INBOX ...) >> 1) Simultaneous mailbomb approach: run both scripts simultaneously, and >> you'll end up with numerous duplicates in mailboxes "test". Very often >> you'll find multiples. Still a lot of duplicates and multiples. Those numbers are not reproducable, 240 (best case) up to 340 (worst case) instead of 200 messages (after 10 tests). Here is one logfile example of a triplicated mail injected at mx1: logfile at mx1: | Feb 16 19:03:12 mx1 postfix/pickup[33958]: 3Z7fMh1PYMz5Ng: uid=0 from= | Feb 16 19:03:12 mx1 postfix/cleanup[34320]: 3Z7fMh1PYMz5Ng: message-id=<3Z7fMh1PYMz5Ng at test.mx1.invalid> | Feb 16 19:03:12 mx1 postfix/qmgr[33959]: 3Z7fMh1PYMz5Ng: from=, size=310, nrcpt=1 (queue active) | Feb 16 19:03:12 mx1 dovecot: lmtp(34456, test): copy from : box=INBOX, uid=12, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=() | Feb 16 19:03:12 mx1 dovecot: lmtp(34456, test): nVlIDeDJH1GYhgAAag1aAg: sieve: msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>: stored mail into mailbox 'INBOX' | Feb 16 19:03:12 mx1 postfix/lmtp[34453]: 3Z7fMh1PYMz5Ng: to=, orig_to=, relay=test.mx1.invalid[private/dovecot-lmtp], delay=0.29, delays=0.08/0/0/0.21, dsn=2.0.0, status=sent (250 2.0.0 nVlIDeDJH1GYhgAAag1aAg Saved) | Feb 16 19:03:12 mx1 postfix/qmgr[33959]: 3Z7fMh1PYMz5Ng: removed | Feb 16 19:03:13 mx1 dovecot: dsync-local(test): copy from INBOX: box=INBOX, uid=42, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=() | Feb 16 19:03:13 mx1 dovecot: dsync-local(test): expunge: box=INBOX, uid=12, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=(\Recent) | Feb 16 19:03:16 mx1 dovecot: dsync-local(test): copy from INBOX: box=INBOX, uid=164, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=() | Feb 16 19:03:16 mx1 dovecot: dsync-local(test): copy from INBOX: box=INBOX, uid=263, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=() | Feb 16 19:03:16 mx1 dovecot: dsync-local(test): expunge: box=INBOX, uid=118, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=(\Recent) | Feb 16 19:03:16 mx1 dovecot: dsync-local(test): expunge: box=INBOX, uid=42, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=(\Recent) after reading those three messages at mx1: | Feb 16 19:04:22 mx1 dovecot: imap(test) hQjfUNvVPwBd3Cqw: flag_change: box=INBOX, uid=372, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=(\Seen \Recent) | Feb 16 19:05:40 mx1 dovecot: imap(test) hQjfUNvVPwBd3Cqw: flag_change: box=INBOX, uid=263, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=(\Seen \Recent) | Feb 16 19:05:41 mx1 dovecot: imap(test) hQjfUNvVPwBd3Cqw: flag_change: box=INBOX, uid=164, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=(\Seen \Recent) logfile at mx2: | Feb 16 19:03:13 mx2 dovecot: dsync-local(test): save: box=INBOX, uid=50, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=() | Feb 16 19:03:17 mx2 dovecot: dsync-local(test): copy from INBOX: box=INBOX, uid=372, msgid=<3Z7fMh1PYMz5Ng at test.mx1.invalid>, size=544, from=root at mx1.invalid (admin), flags=() >> 2) Mailbomb approach: run one script at one server only, and all mails >> will become perfectly well synchronised. Same results here. >> 3) Modify both scripts to "( $INDEX % 1 == 0 )" to add a second waiting >> between every mail injection, and run them simultaneously at both >> servers, and you'll end up with significantly less duplicates and no >> more multiples. Same results here. Good: I cannot find any "Error:" entries in both logfiles any longer. Regards, Michael From dovecot at lists.wgwh.ch Sat Feb 16 20:32:44 2013 From: dovecot at lists.wgwh.ch (Oli Schacher) Date: Sat, 16 Feb 2013 19:32:44 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <1361028022.3230.61.camel@hurina> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> Message-ID: <20130216193244.03b3c5a6@boscos.leetdreams.ch> On Sat, 16 Feb 2013 17:20:22 +0200 Timo Sirainen wrote: > I did a bunch of dsync fixes today in hg. With the new locking > behavior (and other fixes) you shouldn't be able to break it anymore. > Thanks for the fixes, Timo! I can confirm I'm no longer able to break anything with the tests I've mentioned so far(mass appending, simultaneous append and delete on both mailboxes), no more errors, no more dupes. I can also confirm the doveadm-server crash I reported in http://dovecot.markmail.org/thread/fb3qjnsdhtcpirg3 is now gone. There seems to be an issue left when expunging a large amount of messages from the Trash. I managed to get it twice so far by expunging ~3k messages. I'll try to create a reproducible test script for this scenario. I can currently only provide my "clicking around" log output. Version is current hg, e63d1cf19ec7. First time it happened: Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir /mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID (old uid=1221, file=1361035457.M728795P6220.doco1,S=2476,W=2555:2,Sa) Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir /mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID (old uid=1222, file=1361035458.M501466P6220.doco1,S=2477,W=2556:2,Sa) Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir /mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID (old uid=1223, file=1361035458.M988177P6220.doco1,S=2520,W=2599:2,Sa) Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir /mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID (old uid=1224, file=1361035459.M254031P6220.doco1,S=2483,W=2562:2,Sa) Feb 16 18:49:49 doco2 dovecot: imap(user1): Warning: Maildir /mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID (old uid=1225, file=1361035459.M431911P6220.doco1,S=2490,W=2569:2,Sa) Feb 16 18:49:49 doco2 dovecot: imap(user1): Warning: Maildir /mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID (old uid=1226, file=1361035459.M959244P6220.doco1,S=2482,W=2561:2,Sa) Feb 16 18:50:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: Couldn't lock /mailstore/user1/.dovecot-sync.lock: Interrupted system call Feb 16 18:50:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: dsync(local): Received unexpected input S != H Feb 16 18:50:14 doco2 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.61) failed: EOF Feb 16 18:50:14 doco2 dovecot: dsync-local(user1): Error: Remote command returned error 75 Feb 16 18:50:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: Couldn't lock /mailstore/user1/.dovecot-sync.lock: Interrupted system call Feb 16 18:50:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: dsync(local): Received unexpected input N != H Feb 16 18:50:44 doco2 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.61) failed: EOF Feb 16 18:50:44 doco2 dovecot: dsync-local(user1): Error: Remote command returned error 75 2nd time: (no "reappeared" messages this time) Feb 16 19:08:13 doco2 dovecot: imap-login: Login: user=, method=PLAIN, rip=192.168.23.130, lip=192.168.23.62, mpid=4794, session= Feb 16 19:08:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: Couldn't lock /mailstore/user1/.dovecot-sync.lock: Interrupted system call Feb 16 19:08:44 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: dsync(local): Received unexpected input S != H Feb 16 19:08:44 doco2 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.61) failed: EOF Feb 16 19:08:44 doco2 dovecot: dsync-local(user1): Error: Remote command returned error 75 A while later on the other server: Feb 16 19:13:08 doco1 dovecot: doveadm: Error: dsync-remote(user1): Panic: file mail-transaction-log-view.c: line 72 (mail_transaction_log_view_set): assertion failed: (min_file_seq <= max_file_seq) Feb 16 19:13:08 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x5dc2a) [0x7f305f325c2a] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f305f325d12] -> /usr/lib64/dovecot/libdovecot.so.0(+0x1f80a) [0x7f305f2e780a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_transaction_log_view_set+0x580) [0x7f305f64e3f0] -> /usr/bin/doveadm() [0x43786b] -> /usr/bin/doveadm(dsync_transaction_log_scan_init+0x8c) [0x43791c] -> /usr/bin/doveadm(dsync_brain_sync_mailbox_open+0x5e) [0x42724e] -> /usr/bin/doveadm(dsync_brain_slave_recv_mailbox+0x123) [0x427c63] -> /usr/bin/doveadm(dsync_brain_run+0x178) [0x425ff8] -> /usr/bin/doveadm() [0x4265d1] -> /usr/bin/doveadm() [0x4357f0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f305f334bd6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f305f335c67] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f305f334b78] -> /usr/bin/doveadm() [0x424294] -> /usr/bin/doveadm() [0x40ffaf] -> /usr/bin/doveadm() [0x4107dd] -> /usr/bin/doveadm(doveadm_mail_try_run+0x141) [0x410d01] -> /usr/bin/doveadm(main+0x3f1) [0x417d21] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f305ef53cdd] -> /usr/bin/doveadm() [0x40f999] Feb 16 19:13:08 doco1 dovecot: dsync-local(user1): Error: read(vmail at 192.168.23.62) failed: EOF Feb 16 19:13:08 doco1 dovecot: dsync-local(user1): Error: Remote command returned error 255 -- message transmitted on 100% recycled electrons From tss at iki.fi Sat Feb 16 21:09:19 2013 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 Feb 2013 21:09:19 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> Message-ID: <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> On 16.2.2013, at 20.26, Michael Grimm wrote: > Timo Sirainen wrote: > >> I did a bunch of dsync fixes today in hg. With the new locking >> behavior (and other fixes) you shouldn't be able to break it anymore. > > Sorry to say, but I am still able to break replicator with v2.2.beta1 > (35194cf0693e) under the conditions outlined below. I wonder if locking is working correctly in your setup. Your users have home directories, right? Dovecot should be creating .dovecot-sync.lock files in there during the sync. >>> This is v2.2.beta1 (78bdcb6642c7) with freshly created mailboxes "test" >>> at both servers "mx1" and "mx2", and replicator uses ssh for remote >>> access. Both servers run a recent postfix, use lmtp for local delivery, >>> and "test" is a virtual user. > > I might add that both servers run inside FreeBSD jails (if that might make > the difference to your test setup. Inside jail Dovecot sees two different hostnames (same as "hostname" command)? > Good: I cannot find any "Error:" entries in both logfiles any longer. What about Warning? From piotr.rotter at active24.pl Sat Feb 16 23:41:08 2013 From: piotr.rotter at active24.pl (Piotr Rotter) Date: Sat, 16 Feb 2013 22:41:08 +0100 Subject: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients In-Reply-To: <1361034636.3230.78.camel@hurina> References: <511B8DB5.10105@active24.pl> <511BB23A.3050004@active24.pl> <511C1701.3070304@active24.pl> <1361034636.3230.78.camel@hurina> Message-ID: <511FFCF4.2050200@active24.pl> Thank you for your assistant. Broken uidlist errors appears more than 10 time less frequently then in the past and still reduce. I have question about director and no plain text auth methods. Now I have tricky query in director configuration. password_query = SELECT clear AS password, 'Y' as proxy, '%u' AS destuser, clear AS pass FROM postfix_users WHERE email = '%u' Is there some method to make it better. Master password is a bit dangerous ( man in the middle ). Is director can relay no plain text authorization with out checking. I have one more question, which is not concerned subject above. Is there any chance to set dovecot to be policy demon for postfix to check mailbox quota before mail are checking by (amavis, spamassassin, clamav) and before it is sending by lmtp. I think that it could reduce load. From trashcan at odo.in-berlin.de Sun Feb 17 00:12:25 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 16 Feb 2013 23:12:25 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> Message-ID: <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> On 16.02.2013, at 20:09, Timo Sirainen wrote: > On 16.2.2013, at 20.26, Michael Grimm wrote: >> Sorry to say, but I am still able to break replicator with v2.2.beta1 >> (35194cf0693e) under the conditions outlined below. > > I wonder if locking is working correctly in your setup. Your users have home directories, right? Yes, I do have homedirs, ... > Dovecot should be creating .dovecot-sync.lock files in there during the sync. ... and I double-checked that a .dovecot-sync.lock lockfile is being created during replication, and yes, it is. >> I might add that both servers run inside FreeBSD jails (if that might make >> the difference to your test setup. > > Inside jail Dovecot sees two different hostnames (same as "hostname" command)? Hmm. Both jails run at distinct servers. ssh replication uses different domains, though. But, both jails are named identically "test", and both jails resolve to identical hostnames "test" if using "hostname". But, a "hostname -f" is lacking to return "test.mx1.invalid" and "test.mx2.invalid", respectively (although a "nslookup test" does). Hmm, do you think I should need to provide different hostnames in both jails? >> Good: I cannot find any "Error:" entries in both logfiles any longer. > > What about Warning? I do see only those few messages at both servers: | dovecot: doveadm(test): Warning: fscking index file /.../test/storage/dovecot.map.index | dovecot: doveadm(test): Warning: fscking index file /.../test/storage/dovecot.map.index | dovecot: doveadm(test): Warning: mdbox /.../test/storage: rebuilding indexes Please let me know what you want me to test next. I really to appreciate your efforts and with kind regards, Michael From AxelLuttgens at swing.be Sun Feb 17 00:43:28 2013 From: AxelLuttgens at swing.be (Axel Luttgens) Date: Sat, 16 Feb 2013 23:43:28 +0100 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up In-Reply-To: <1361033245.3230.72.camel@hurina> References: <6A5A5394-4325-47A5-A8A6-CE433FCDD7E3@swing.be> <1361033245.3230.72.camel@hurina> Message-ID: <91205620-143D-42E7-9DD5-4A14DA994B5E@swing.be> Le 16 f?vr. 2013 ? 17:47, Timo Sirainen a ?crit : > [...] > a) dtruss the pop3 process to see which syscall hangs. Maybe this works: > > service pop3 { > executable = /usr/bin/dtruss /usr/local/libexec/dovecot/pop3 > } > > It logs to Dovecot's error log. Hello Timo, As usual, once again, many thanks for your reply. :-) I already tried the dtruss way on hanging pop3 processes, but without any valuable output: they were just sleeping, and dtruss came without any output. Anyway, yes, above proposal proved functional. Interestingly. ;-) I tried five consecutive pop logins (RUN 1 to 5) with the same user ("user1", 100002/100002), yet without being able to discern some conclusive pattern. Just to be sure, I turned flocking off for two other subsequent logins (RUN 6 and 7). Again without any "Aha" effect on my side. In all cases, I had to send a KILL signal to the hanging pop3 process, hence the last empty line in the log ("pop3: Error: "). Outputs are in Pop3Hangs.txt.zip. > b) get gdb backtrace: > > gdb -p > bt full I thus tried an eigth run, still with flocking off. Output is in Pop3HangsGdb.txt.zip. Wishing you all the best, Axel -------------- next part -------------- A non-text attachment was scrubbed... Name: Pop3Hangs.txt.zip Type: application/zip Size: 16181 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Pop3HangsGdb.txt.zip Type: application/zip Size: 4023 bytes Desc: not available URL: From siefke_listen at web.de Sun Feb 17 01:34:53 2013 From: siefke_listen at web.de (Silvio Siefke) Date: Sun, 17 Feb 2013 00:34:53 +0100 Subject: [Dovecot] Virtual Users Dovecot and Postfix In-Reply-To: <20130216034936.GA8029@anubis.morrow.me.uk> References: <20130216041310.f5e6fb2097db6e8e776526ad@web.de> <20130216034936.GA8029@anubis.morrow.me.uk> Message-ID: <20130217003453.94d841ec7742e1bf3b6b5f09@web.de> On Sat, 16 Feb 2013 03:49:37 +0000 Ben Morrow wrote: > The setup given in that howto will accept mail for any user at one of > the domains listed in virtual_mailbox_domains, and mail for > nonexistent users will then be bounced by Dovecot. This is a very bad > idea, since you'll end up becoming a backscatter source; you should > set up a separate Postfix table listing the valid users at those > domains, and put that table in virtual_mailbox_maps. (You don't want > to use /etc/postfix/virtual for this, you want a separate table.) my main.cf > http://nopaste.info/23ee74a772.html When i understand correct the passwd file from Dovecot is enough when i delivered over dovecot lmtp? Or must i set the mailboxes in the /etc/postfix/virtual too? Sorry for question, its long time i work with postfix and dovecot and in last years dovecot change much in config. I want not run a database for the mailserver, i have not so much boxes at end running. Thank You for Help & Greetings Silvio From ben at morrow.me.uk Sun Feb 17 01:49:03 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Sat, 16 Feb 2013 23:49:03 +0000 Subject: [Dovecot] Virtual Users Dovecot and Postfix In-Reply-To: <20130217003453.94d841ec7742e1bf3b6b5f09@web.de> References: <20130216041310.f5e6fb2097db6e8e776526ad@web.de> <20130216034936.GA8029@anubis.morrow.me.uk> <20130217003453.94d841ec7742e1bf3b6b5f09@web.de> Message-ID: <20130216234902.GC8029@anubis.morrow.me.uk> At 12AM +0100 on 17/02/13 you (Silvio Siefke) wrote: > On Sat, 16 Feb 2013 03:49:37 +0000 Ben Morrow wrote: > > > The setup given in that howto will accept mail for any user at one of > > the domains listed in virtual_mailbox_domains, and mail for > > nonexistent users will then be bounced by Dovecot. This is a very bad > > idea, since you'll end up becoming a backscatter source; you should > > set up a separate Postfix table listing the valid users at those > > domains, and put that table in virtual_mailbox_maps. (You don't want > > to use /etc/postfix/virtual for this, you want a separate table.) > > my main.cf > http://nopaste.info/23ee74a772.html Excerpts: | virtual_mailbox_domains = silvio-siefke.de, silviosiefke.com | virtual_mailbox_maps = hash:/etc/postfix/maps/mailbox | virtual_transport = lmtp:unix:private/dovecot-lmtp | | smtpd_recipient_restrictions = | reject_unverified_recipient, > When i understand correct the passwd file from Dovecot is enough when > i delivered over dovecot lmtp? Since you have 'reject_unverified_recipient', Postfix will check with the LMTP server whether or not the recipient exists. > Or must i set the mailboxes in the /etc/postfix/virtual too? Did you read the text you quoted above? I specifically said you do *not* want to use /etc/postfix/virtual for this. What do you have in hash:/etc/postfix/maps/mailbox? If this is a complete list of all addresses at your virtual_mailbox_domains, you can remove reject_unverified_recipient; otherwise, you want to remove virtual_mailbox_maps. > Sorry for question, its long time i work with postfix and dovecot > and in last years dovecot change much in config. This is not really a Dovecot problem. Ben From ben at morrow.me.uk Sun Feb 17 05:14:50 2013 From: ben at morrow.me.uk (Ben Morrow) Date: Sun, 17 Feb 2013 03:14:50 +0000 Subject: [Dovecot] Private message flags in shared mailboxes In-Reply-To: <0D916D2F-C59F-4C77-8887-34BFED364A00@iki.fi> References: <0D916D2F-C59F-4C77-8887-34BFED364A00@iki.fi> Message-ID: <20130217031449.GD8029@anubis.morrow.me.uk> At 8PM +0200 on 16/02/13 you (Timo Sirainen) wrote: > > 1. What would be a good place to configure which flags are shared and > which are private? [...] Another possibility would > be in dovecot-acl file, but public mailboxes might be enabled without > ACL plugin. [...] > > 4. Private keywords aren't supported. I guess not much worse than > system flags, but more code complexity. Wonder if anyone would care > about them. Does this mean non-owners can't set keywords, or that keywords they set are set for everyone? I can see the latter being irritating, since some clients (like Thunderbird) tend to set keywords without being asked to. Denying 'w' would presumably prevent this, but that rather reduces the point of having shared flags other than \Seen in the first place. I think ideally one would want an extension to ACL, which supports a per-flag (or keyword) per-mailbox per-user permission which can be either no access, read, write shared, or write private. I don't think this can sensibly be stuffed into the existing ACL extension (the only reasonable place to put the flag name is as part of the user identifier, which is confusing and crude), so this would mean a new GETFLAGACL/ SETFLAGACL/MYFLAGRIGHTS IMAP extension. Unless you think this can be sensibly set via METADATA (I'm not familiar with that extension, so I don't know whether using it for access control would be reasonable)? Personally I don't think it's worth worrying about the case of shared mailboxes without ACL. Ben From tss at iki.fi Sun Feb 17 06:47:56 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 06:47:56 +0200 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up In-Reply-To: <91205620-143D-42E7-9DD5-4A14DA994B5E@swing.be> References: <6A5A5394-4325-47A5-A8A6-CE433FCDD7E3@swing.be> <1361033245.3230.72.camel@hurina> <91205620-143D-42E7-9DD5-4A14DA994B5E@swing.be> Message-ID: <617547CE-B563-4154-9B22-97402921D061@iki.fi> On 17.2.2013, at 0.43, Axel Luttgens wrote: >> b) get gdb backtrace: >> >> gdb -p >> bt full > > I thus tried an eigth run, still with flocking off. > > Output is in Pop3HangsGdb.txt.zip. #21 0x00007fff8f57b086 in gethostbyname () #22 0x000000010b75571c in my_hostdomain () at hostpid.c:39 Where the code is: hent = gethostbyname(my_hostname); The gethostbyname() call is used to figure out the current system's domain. Why it would be hanging there, I don't really know. That call anyway isn't fully required, you could just replace it with "hent = NULL;" From tss at iki.fi Sun Feb 17 06:58:21 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 06:58:21 +0200 Subject: [Dovecot] dovecot-uidlist crashing and re retriving messages by pop3 clients In-Reply-To: <511FFCF4.2050200@active24.pl> References: <511B8DB5.10105@active24.pl> <511BB23A.3050004@active24.pl> <511C1701.3070304@active24.pl> <1361034636.3230.78.camel@hurina> <511FFCF4.2050200@active24.pl> Message-ID: <9B9022CA-2A8B-4335-BA00-EAC197ECD99B@iki.fi> On 16.2.2013, at 23.41, Piotr Rotter wrote: > Thank you for your assistant. > > Broken uidlist errors appears more than 10 time less frequently then in the past and still reduce. > > I have question about director and no plain text auth methods. Now I have tricky query in director configuration. > password_query = SELECT clear AS password, 'Y' as proxy, '%u' AS destuser, clear AS pass FROM postfix_users WHERE email = '%u' > > Is there some method to make it better. %u as destuser isn't necessary, since it's the default. > Master password is a bit dangerous ( man in the middle ). Is director can relay no plain text authorization with out checking. If you set NULL as password, director won't verify the password. You could even use passdb static. But this increases director's memory usage since users are verified by backends and director needs to remember all the unknown user login attempts (from brute force bots). If you want non-plaintext auth: Master password is slightly annoying, but you can set up the system so that director won't accept it and backends can't be connected directly. It would also in theory be possible to use a per-user master password by using e.g. md5(username | master_password) as the password, so the master password would never be actually visible in the network (actually adding some non-plaintext SASL auth client support for Dovecot would basically do that). Or the director <-> backend auth could simply have its own list of randomly generated passwords for users. > I have one more question, which is not concerned subject above. Is there any chance to set dovecot to be policy demon for postfix to check mailbox quota before mail are checking by (amavis, spamassassin, clamav) and before it is sending by lmtp. I think that it could reduce load. That's something I'm planning on creating. From tss at iki.fi Sun Feb 17 07:06:02 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 07:06:02 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> Message-ID: <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> On 17.2.2013, at 0.12, Michael Grimm wrote: >>> I might add that both servers run inside FreeBSD jails (if that might make >>> the difference to your test setup. >> >> Inside jail Dovecot sees two different hostnames (same as "hostname" command)? > > Hmm. Both jails run at distinct servers. ssh replication uses different domains, though. But, both jails are named identically "test", and both jails resolve to identical hostnames "test" if using "hostname". But, a "hostname -f" is lacking to return "test.mx1.invalid" and "test.mx2.invalid", respectively (although a "nslookup test" does). Hmm, do you think I should need to provide different hostnames in both jails? That's the problem most likely. I'd guess Dovecot sees both servers as having "test" as the hostname and each server thinks it's the one that should be doing the locking and not the other. See if this helps: http://hg.dovecot.org/dovecot-2.2/rev/e7aabd79c9d5 From tss at iki.fi Sun Feb 17 07:23:21 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 07:23:21 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> Message-ID: On 17.2.2013, at 7.06, Timo Sirainen wrote: > On 17.2.2013, at 0.12, Michael Grimm wrote: > >> Hmm. Both jails run at distinct servers. ssh replication uses different domains, though. But, both jails are named identically "test", and both jails resolve to identical hostnames "test" if using "hostname". But, a "hostname -f" is lacking to return "test.mx1.invalid" and "test.mx2.invalid", respectively (although a "nslookup test" does). Hmm, do you think I should need to provide different hostnames in both jails? > > That's the problem most likely. I'd guess Dovecot sees both servers as having "test" as the hostname and each server thinks it's the one that should be doing the locking and not the other. > > See if this helps: http://hg.dovecot.org/dovecot-2.2/rev/e7aabd79c9d5 Although even if it does, other parts of Dovecot still use only the hostname part to guarantee global uniqueness of things. So better to have unique hostnames. From tss at iki.fi Sun Feb 17 10:21:37 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 10:21:37 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: <20130216193244.03b3c5a6@boscos.leetdreams.ch> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <20130216193244.03b3c5a6@boscos.leetdreams.ch> Message-ID: <1361089297.3230.84.camel@hurina> On Sat, 2013-02-16 at 19:32 +0100, Oli Schacher wrote: > There seems to be an issue left when expunging a large amount of > messages from the Trash. I managed to get it twice so far by expunging > ~3k messages. I'll try to create a reproducible test script for this > scenario. I can currently only provide my "clicking around" log output. > Version is current hg, e63d1cf19ec7. > > First time it happened: > Feb 16 18:49:48 doco2 dovecot: imap(user1): Warning: Maildir /mailstore/user1/maildir/.Trash: Expunged message reappeared, giving a new UID (old uid=1221, file=1361035457.M728795P6220.doco1,S=2476,W=2555:2,Sa) These errors should be gone now in hg. Although there's still some mail duplication problem with maildir that doesn't log any errors about it. I'm not sure why that happens. > Feb 16 18:50:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Error: dsync(local): Received unexpected input S != H Fixed also this error that happened on locking failure. > Feb 16 19:13:08 doco1 dovecot: doveadm: Error: dsync-remote(user1): Panic: file mail-transaction-log-view.c: line 72 (mail_transaction_log_view_set): assertion failed: (min_file_seq <= max_file_seq) Not sure about this one. But usually this happens only once and retry works. From trashcan at odo.in-berlin.de Sun Feb 17 11:44:54 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 17 Feb 2013 10:44:54 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> Message-ID: On 17.02.2013, at 06:23, Timo Sirainen wrote: > On 17.2.2013, at 7.06, Timo Sirainen wrote: >> On 17.2.2013, at 0.12, Michael Grimm wrote: >>> Hmm. Both jails run at distinct servers. ssh replication uses different domains, though. But, both jails are named identically "test", and both jails resolve to identical hostnames "test" if using "hostname". But, a "hostname -f" is lacking to return "test.mx1.invalid" and "test.mx2.invalid", respectively (although a "nslookup test" does). Hmm, do you think I should need to provide different hostnames in both jails? >> >> That's the problem most likely. I'd guess Dovecot sees both servers as having "test" as the hostname and each server thinks it's the one that should be doing the locking and not the other. >> >> See if this helps: http://hg.dovecot.org/dovecot-2.2/rev/e7aabd79c9d5 Good news! Those identical hostnames at both servers broke replicator. Now, with v2.2.beta1 (1dd1e88ba0a2) I cannot break replicator any longer how many messages I do inject at both servers simultaneously. (Tested a couple of times up to 2000 mails at every server.) > Although even if it does, other parts of Dovecot still use only the hostname part to guarantee global uniqueness of things. So better to have unique hostnames. What parts of Dovecot would be involved? I'm curious because my production mailservers use identical hostnames in their jails ever since running Dovecot (starting 1.x). Thanks for the new replicator code, I really appreciate your work! And, from my point of view I will consider replicator v2.2 ready for production. With kind regards, Michael From tss at iki.fi Sun Feb 17 12:08:28 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 12:08:28 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> Message-ID: <1361095708.3230.91.camel@hurina> On Sun, 2013-02-17 at 10:44 +0100, Michael Grimm wrote: > > Although even if it does, other parts of Dovecot still use only the hostname part to guarantee global uniqueness of things. So better to have unique hostnames. > > What parts of Dovecot would be involved? I'm curious because my production mailservers use identical hostnames in their jails ever since running Dovecot (starting 1.x). Mainly that maildir filenames are used as GUIDs. If two have the same name, they are assumed to be identical. That's why the maildir filenames include the hostname in them, to make sure that the GUID is different even if two mails happen to be delivered at exactly the same time with the same PID and same size to two different servers. So pretty unlikely, but better to be safe. :) There may be some other features that require unique hostnames in future. Anything where multiple Dovecot servers need to communicate between each others. If some day there is such generic communication between Dovecot servers I'm planning on enforcing this requirement. From trashcan at odo.in-berlin.de Sun Feb 17 12:19:18 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 17 Feb 2013 11:19:18 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <1361095708.3230.91.camel@hurina> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> Message-ID: <6411E134-618E-495C-A968-4230CD3A8D43@odo.in-berlin.de> On 17.02.2013, at 11:08, Timo Sirainen wrote: > On Sun, 2013-02-17 at 10:44 +0100, Michael Grimm wrote: >>> Although even if it does, other parts of Dovecot still use only the hostname part to guarantee global uniqueness of things. So better to have unique hostnames. >> >> What parts of Dovecot would be involved? I'm curious because my production mailservers use identical hostnames in their jails ever since running Dovecot (starting 1.x). > > Mainly that maildir filenames are used as GUIDs. If two have the same > name, they are assumed to be identical. That's why the maildir filenames > include the hostname in them, to make sure that the GUID is different > even if two mails happen to be delivered at exactly the same time with > the same PID and same size to two different servers. So pretty unlikely, > but better to be safe. :) Ok, that won't hit me for the time being because I am using mdbox. > There may be some other features that require unique hostnames in > future. Anything where multiple Dovecot servers need to communicate > between each others. If some day there is such generic communication > between Dovecot servers I'm planning on enforcing this requirement. Thanks for that clarification. Thus I will need to think about different hostnames, although that implies "no more just copying config files between both servers that imply identical hostnames at both sites" ;-) Regards, Michael From dovecot at anes.su Sun Feb 17 12:25:27 2013 From: dovecot at anes.su (Anes Mukhametov) Date: Sun, 17 Feb 2013 14:25:27 +0400 Subject: [Dovecot] Dsync converting dbox->Maildir In-Reply-To: <1361032161.3230.63.camel@hurina> References: <1361032161.3230.63.camel@hurina> Message-ID: <8394cff6ce441b172b85a22af6e23a8e@logol.ru> I used dsync mirror: http://wiki2.dovecot.org/Migration/MailFormat AFAIK without S field there is big overhead with quota recalculation. Timo Sirainen ????? 2013-02-16 20:29: > On Wed, 2013-02-13 at 12:28 +0400, Anes Mukhametov wrote: > >> I've changed my mailbox format from dbox to Maildir. The problem is >> maildir >> message filename extension has no S=/W= field after migration. >> Using dovecot 2.1.15. > > How did you change it? dsync is the only tool that can properly do it, > and it definitely adds the S/W fields. > > Anyway, Dovecot can handle maildir files without S/W files anyway. But > if you really want to add them, use > http://dovecot.org/tools/maildir-size-fix.pl with -a parameter. It > adds > only S though, no W support yet. From tss at iki.fi Sun Feb 17 12:30:36 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 12:30:36 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: <6411E134-618E-495C-A968-4230CD3A8D43@odo.in-berlin.de> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> <6411E134-618E-495C-A968-4230CD3A8D43@odo.in-berlin.de> Message-ID: <89A60A45-639C-4818-9071-849E9F6A9868@iki.fi> On 17.2.2013, at 12.19, Michael Grimm wrote: > On 17.02.2013, at 11:08, Timo Sirainen wrote: >> On Sun, 2013-02-17 at 10:44 +0100, Michael Grimm wrote: > >>>> Although even if it does, other parts of Dovecot still use only the hostname part to guarantee global uniqueness of things. So better to have unique hostnames. >>> >>> What parts of Dovecot would be involved? I'm curious because my production mailservers use identical hostnames in their jails ever since running Dovecot (starting 1.x). >> >> Mainly that maildir filenames are used as GUIDs. If two have the same >> name, they are assumed to be identical. That's why the maildir filenames >> include the hostname in them, to make sure that the GUID is different >> even if two mails happen to be delivered at exactly the same time with >> the same PID and same size to two different servers. So pretty unlikely, >> but better to be safe. :) > > Ok, that won't hit me for the time being because I am using mdbox. It's basically the same with mdbox, except instead of using actual hostname it's using a 32bit hash of it. (So yeah, ideally there should be checks for detecting hostname hash collisions..) From tss at iki.fi Sun Feb 17 12:32:03 2013 From: tss at iki.fi (Timo Sirainen) Date: Sun, 17 Feb 2013 12:32:03 +0200 Subject: [Dovecot] Dsync converting dbox->Maildir In-Reply-To: <8394cff6ce441b172b85a22af6e23a8e@logol.ru> References: <1361032161.3230.63.camel@hurina> <8394cff6ce441b172b85a22af6e23a8e@logol.ru> Message-ID: <08A11847-1F95-4C5C-A8AF-FDA49509A22C@iki.fi> Ah, right. dsync preserves the dbox GUIDs by storing them into filenames instead. Anyway, if you use dict-file quota backend there's no slowness (but yes, with maildir quota backend is). On 17.2.2013, at 12.25, Anes Mukhametov wrote: > I used dsync mirror: http://wiki2.dovecot.org/Migration/MailFormat > AFAIK without S field there is big overhead with quota recalculation. > > Timo Sirainen ????? 2013-02-16 20:29: >> On Wed, 2013-02-13 at 12:28 +0400, Anes Mukhametov wrote: >>> I've changed my mailbox format from dbox to Maildir. The problem is maildir >>> message filename extension has no S=/W= field after migration. >>> Using dovecot 2.1.15. >> How did you change it? dsync is the only tool that can properly do it, >> and it definitely adds the S/W fields. >> Anyway, Dovecot can handle maildir files without S/W files anyway. But >> if you really want to add them, use >> http://dovecot.org/tools/maildir-size-fix.pl with -a parameter. It adds >> only S though, no W support yet. > > From h.reindl at thelounge.net Sun Feb 17 12:36:33 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 17 Feb 2013 11:36:33 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <1361095708.3230.91.camel@hurina> References: <20130131112752.642beb56@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> Message-ID: <5120B2B1.7090306@thelounge.net> Am 17.02.2013 11:08, schrieb Timo Sirainen: >> What parts of Dovecot would be involved? I'm curious because my production mailservers use identical hostnames in their jails ever since running Dovecot (starting 1.x). > > Mainly that maildir filenames are used as GUIDs. If two have the same > name, they are assumed to be identical. That's why the maildir filenames > include the hostname in them, to make sure that the GUID is different > even if two mails happen to be delivered at exactly the same time with > the same PID and same size to two different servers. So pretty unlikely, > but better to be safe. :) > > There may be some other features that require unique hostnames in > future. Anything where multiple Dovecot servers need to communicate > between each others. If some day there is such generic communication > between Dovecot servers I'm planning on enforcing this requirement. Postfix is enforcing this since forever "Greeted me with my own hostname" hostnames inside a network should always be unique -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From AxelLuttgens at swing.be Sun Feb 17 13:24:19 2013 From: AxelLuttgens at swing.be (Axel Luttgens) Date: Sun, 17 Feb 2013 12:24:19 +0100 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up In-Reply-To: <617547CE-B563-4154-9B22-97402921D061@iki.fi> References: <6A5A5394-4325-47A5-A8A6-CE433FCDD7E3@swing.be> <1361033245.3230.72.camel@hurina> <91205620-143D-42E7-9DD5-4A14DA994B5E@swing.be> <617547CE-B563-4154-9B22-97402921D061@iki.fi> Message-ID: <534308E6-579C-4CAA-AB14-CFB8105C04DD@swing.be> Le 17 f?vr. 2013 ? 05:47, Timo Sirainen a ?crit : > [...] > The gethostbyname() call is used to figure out the current system's domain. Why it would be hanging there, I don't really know. That call anyway isn't fully required, you could just replace it with "hent = NULL;" Yes, looks like those gethostbyname() require some investigations here... I'll have a look and provide my "findings" here. In the meantime, I don't really like the idea of messing with a general purpose function, my_hostdomain(), just to avoid a local problem... Wouldn't you have a better idea? ;-) And, BTW, imap processes may hang as well, for the same underlying causes. It is just a matter of selecting INBOX (up to now, I just performed a login/logout sequence). TIA, Axel From trashcan at odo.in-berlin.de Sun Feb 17 22:04:15 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 17 Feb 2013 21:04:15 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <1361095708.3230.91.camel@hurina> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> Message-ID: <621CF682-5E43-414D-A69D-5A1089A34B6A@odo.in-berlin.de> On 17.02.2013, at 11:08, Timo Sirainen wrote: > There may be some other features that require unique hostnames in > future. Anything where multiple Dovecot servers need to communicate > between each others. I'd like to come back to that issue in order to understand your statement cited below. First of all: whenever you referred to "hostname" in this thread you have been using it as a synonym for the local part [1] of a FQDN, right? I have both servers of mine configured to use identical local parts ("test") but different FQDN (aka "test.domainA.tldA" and "test.domainB.tldB"). Your fix has been to replace "my_hostname" by "my_hostdomain()", thus using "test.domainA.tldA" and "test.domainB.tldB" instead of "test", right? > If some day there is such generic communication between Dovecot servers > I'm planning on enforcing this requirement. Given that all my interpretations of your statements are correct I do have difficulties in understanding why a "generic communication between Dovecot servers" should be limited to enforcing different local parts of all Dovecot servers implied instead of different FQDN? That would make much more sense regarding uniqueness in hostnames, IMHO. Two servers like "dovecot.forget-about.it" and "dovecot.you-name.it" should be able to communicate generically, again: IMHO. BTW: I had had defined "hostname=" in dovecot.conf identically using completely different *but* identical FQDNs "mail.my-domain.tld" because of: | conf.d/15-lda.conf: | # Hostname to use in various parts of sent mails, eg. in Message-Id. | # Default is the system's real hostname. | #hostname = At least my_hostdomain() doesn't care about that setting, right? Again, I can live with mandatory different local hostname parts, but I would love to understand why ... With kind regards, Michael [1] http://en.wikipedia.org/wiki/Hostname From h.reindl at thelounge.net Sun Feb 17 22:10:38 2013 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 17 Feb 2013 21:10:38 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <621CF682-5E43-414D-A69D-5A1089A34B6A@odo.in-berlin.de> References: <20130131112752.642beb56@boscos> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> <621CF682-5E43-414D-A69D-5A1089A34B6A@odo.in-berlin.de> Message-ID: <5121393E.1040404@thelounge.net> Am 17.02.2013 21:04, schrieb Michael Grimm: > On 17.02.2013, at 11:08, Timo Sirainen wrote: > >> There may be some other features that require unique hostnames in >> future. Anything where multiple Dovecot servers need to communicate >> between each others. > > I'd like to come back to that issue in order to understand your statement cited below. > > First of all: whenever you referred to "hostname" in this thread you have been using it as a synonym for the local part [1] of a FQDN, right? > > I have both servers of mine configured to use identical local parts ("test") but different FQDN (aka "test.domainA.tldA" and "test.domainB.tldB"). Your fix has been to replace "my_hostname" by "my_hostdomain()", thus using "test.domainA.tldA" and "test.domainB.tldB" instead of "test", right? > >> If some day there is such generic communication between Dovecot servers >> I'm planning on enforcing this requirement. > > Given that all my interpretations of your statements are correct I do have difficulties in understanding why a "generic communication between Dovecot servers" should be limited to enforcing different local parts of all Dovecot servers implied instead of different FQDN? That would make much more sense regarding uniqueness in hostnames, IMHO. Two servers like "dovecot.forget-about.it" and "dovecot.you-name.it" should be able to communicate generically, again: IMHO. the better design would be if doveot generates some UUID at the first startup in a /etc/dovecot/uuid.conf if the file does not exist becasue it would make hostnames meaningless at all AND give you the option if you are knowing what you are doing to replace a machine with a newer one by rsync datadirs and the whole /etc/dovecot/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From trashcan at odo.in-berlin.de Sun Feb 17 22:10:46 2013 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 17 Feb 2013 21:10:46 +0100 Subject: [Dovecot] dsync replication errors In-Reply-To: <621CF682-5E43-414D-A69D-5A1089A34B6A@odo.in-berlin.de> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> <621CF682-5E43-414D-A69D-5A1089A34B6A@odo.in-berlin.de> Message-ID: <02F50CDE-FD44-4DA6-993B-960EC612CC83@odo.in-berlin.de> On 17.02.2013, at 21:04, Michael Grimm wrote: > BTW: I had had defined "hostname=" in dovecot.conf identically using completely different *but* identical FQDNs "mail.my-domain.tld" because of: s/using completely different/using completely different to locally reported by resolver/g Regards, Michael From tss at iki.fi Mon Feb 18 07:21:02 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 18 Feb 2013 07:21:02 +0200 Subject: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up In-Reply-To: <534308E6-579C-4CAA-AB14-CFB8105C04DD@swing.be> References: <6A5A5394-4325-47A5-A8A6-CE433FCDD7E3@swing.be> <1361033245.3230.72.camel@hurina> <91205620-143D-42E7-9DD5-4A14DA994B5E@swing.be> <617547CE-B563-4154-9B22-97402921D061@iki.fi> <534308E6-579C-4CAA-AB14-CFB8105C04DD@swing.be> Message-ID: <1361164862.3230.93.camel@hurina> On Sun, 2013-02-17 at 12:24 +0100, Axel Luttgens wrote: > Le 17 f?vr. 2013 ? 05:47, Timo Sirainen a ?crit : > > > [...] > > The gethostbyname() call is used to figure out the current system's domain. Why it would be hanging there, I don't really know. That call anyway isn't fully required, you could just replace it with "hent = NULL;" > > Yes, looks like those gethostbyname() require some investigations here... > I'll have a look and provide my "findings" here. > > In the meantime, I don't really like the idea of messing with a general purpose function, my_hostdomain(), just to avoid a local problem... > Wouldn't you have a better idea? ;-) Solution for v2.2: http://hg.dovecot.org/dovecot-2.2/rev/a6b40687c0a4 From tss at iki.fi Mon Feb 18 08:07:42 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 18 Feb 2013 08:07:42 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: <621CF682-5E43-414D-A69D-5A1089A34B6A@odo.in-berlin.de> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> <621CF682-5E43-414D-A69D-5A1089A34B6A@odo.in-berlin.de> Message-ID: <7BFB54D5-FE53-4328-A339-F752D330F2CF@iki.fi> On 17.2.2013, at 22.04, Michael Grimm wrote: > On 17.02.2013, at 11:08, Timo Sirainen wrote: > >> There may be some other features that require unique hostnames in >> future. Anything where multiple Dovecot servers need to communicate >> between each others. > > I'd like to come back to that issue in order to understand your statement cited below. > > First of all: whenever you referred to "hostname" in this thread you have been using it as a synonym for the local part [1] of a FQDN, right? I mean what gethostname() function returns, which is what "hostname" command usually also returns. And yes, I think it's the local part always. > I have both servers of mine configured to use identical local parts ("test") but different FQDN (aka "test.domainA.tldA" and "test.domainB.tldB"). Your fix has been to replace "my_hostname" by "my_hostdomain()", thus using "test.domainA.tldA" and "test.domainB.tldB" instead of "test", right? Yes. >> If some day there is such generic communication between Dovecot servers >> I'm planning on enforcing this requirement. > > Given that all my interpretations of your statements are correct I do have difficulties in understanding why a "generic communication between Dovecot servers" should be limited to enforcing different local parts of all Dovecot servers implied instead of different FQDN? That would make much more sense regarding uniqueness in hostnames, IMHO. Two servers like "dovecot.forget-about.it" and "dovecot.you-name.it" should be able to communicate generically, again: IMHO. I think systems named those would belong to different clusters and wouldn't need to communicate with each others. I looked through the code. The hostname (without domain) are currently used for: * maildir filenames * temporary filenames * authentication challenge strings in some auth mechanisms * logging So I think the hostname uniqueness matters mainly when using a shared filesystem (e.g. NFS). > BTW: I had had defined "hostname=" in dovecot.conf identically using completely different *but* identical FQDNs "mail.my-domain.tld" because of: > > | conf.d/15-lda.conf: > > | # Hostname to use in various parts of sent mails, eg. in Message-Id. > | # Default is the system's real hostname. > | #hostname = > > At least my_hostdomain() doesn't care about that setting, right? Right. I updated the comment a bit: http://hg.dovecot.org/dovecot-2.2/rev/6a67a1440e15 lda_hostname would have been a better name for the settings. From tss at iki.fi Mon Feb 18 08:49:06 2013 From: tss at iki.fi (Timo Sirainen) Date: Mon, 18 Feb 2013 08:49:06 +0200 Subject: [Dovecot] dsync replication errors In-Reply-To: <89A60A45-639C-4818-9071-849E9F6A9868@iki.fi> References: <20130131112752.642beb56@boscos> <20130131130639.260eb4b8@boscos> <8430F751-4A7E-4CCF-AFAF-20E4C9F2A608@iki.fi> <20130131134624.2c515aa8@boscos> <4ABF8E21-B7D4-47A7-B79F-81E97BE21A7C@iki.fi> <20130131141018.44b30d71@boscos> <8114CD6C-3825-4BAB-919D-EE3AD92FFF4F@iki.fi> <20130131143639.3cfdf41c@boscos> <20130131173736.3015b6cb@boscos> <3221CB48-750F-493F-AFE0-32E6FDD18360@iki.fi> <20130131184125.49d35f5d@boscos> <1359663448.3230.0.camel@hurina> <20130201132825.5363b1f4@boscos> <1361028022.3230.61.camel@hurina> <996B9676-78E8-44FA-B464-70A32C531038@iki.fi> <05DB5831-C3AE-4853-AF93-984B133F2231@odo.in-berlin.de> <24C03F6F-F069-4AEA-9F26-A1C9D1C93F2F@iki.fi> <1361095708.3230.91.camel@hurina> <6411E134-618E-495C-A968-4230CD3A8D43@odo.in-berlin.de> <89A60A45-639C-4818-9071-849E9F6A9868@iki.fi> Message-ID: <1361170146.3230.96.camel@hurina> On Sun, 2013-02-17 at 12:30 +0200, Timo Sirainen wrote: > (So yeah, ideally there should be checks for detecting hostname hash collisions..) Added to v2.2 hg: % doveconf -H dovecot%d No duplicate host hashes in dovecot0 .. dovecot9 % doveconf -H dovecot%2d No duplicate host hashes in dovecot0 .. dovecot99 % doveconf -H dovecot%02d No duplicate host hashes in dovecot00 .. dovecot99 doveconf -H without the template it attempts to detect it from the current hostname. From tlx at leuxner.net Mon Feb 18 10:58:01 2013 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 18 Feb 2013 09:58:01 +0100 Subject: [Dovecot] Quota Problems with LMTP in HG 62a930eb22b5 Message-ID: <20130218085801.GA14082@nihlus.leuxner.net> Since updating to the latest HG these errors occur. Nothing else changed in the config: $ dovecot --version 2.2.beta1 (62a930eb22b5) ==> /var/log/dovecot/dovecot.log <== Feb 18 09:47:32 spectre dovecot: lmtp(14340): Connect from local Feb 18 09:47:32 spectre dovecot: lmtp(14340, tlx at leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, tlx at leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, tlx at leuxner.net): Error: dict quota: Quota update failed, it's now desynced $ doveconf -n # 2.2.beta1 (62a930eb22b5): /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-0.bpo.4-amd64 x86_64 Debian 6.0.6 auth_cache_size = 16 k auth_verbose = yes mail_location = mdbox:~/mdbox mail_plugins = acl quota stats mailbox_list_index = yes namespace { list = yes location = mdbox:/var/vmail/public:INDEXPVT=~/mdbox/public prefix = Public/ separator = / subscriptions = no type = public } namespace inbox { hidden = no inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = username_format=%u /var/vmail/auth.d/%d/passwd driver = passwd-file } plugin { acl = vfile:/var/vmail/conf.d/%d/acls:cache_secs=300 mail_log_events = expunge mailbox_delete quota = dict:user::file:%h/mdbox/dovecot-quota quota_rule = *:storage=1GB quota_rule2 = Trash:storage=+10%% sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /var/vmail/conf.d/%d/sieve stats_refresh = 30s stats_track_cmds = yes } protocols = " imap lmtp" quota_full_tempfail = yes service auth-worker { unix_listener auth-worker { user = doveauth } user = doveauth } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = doveauth } service imap-login { inet_listener imap { address = 1.2.3.4 port = 143 } inet_listener imaps { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_ca = From skdovecot at smail.inf.fh-brs.de Mon Feb 18 11:03:39 2013 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 18 Feb 2013 10:03:39 +0100 (CET) Subject: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails) In-Reply-To: References: <1361032999.3230.69.camel@hurina> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 16 Feb 2013, Chris Laif wrote: > > Thanks Timo, at least one person understands me ;-) Unfortunately, > both solutions require programming skills far better than I have. So I > start enjoying my weekend doing other things ... thank you for your > great work on dovecot! I like Daniel's idea: + Have Thunderbird move unwanted messages to Delayed/