From stan at hardwarefreak.com Mon Oct 1 00:37:17 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 30 Sep 2012 16:37:17 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <5068BB8D.8020302@hardwarefreak.com> On 9/30/2012 8:02 AM, Charles Marcus wrote: > Hi Timo/everyone, > > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? > > The reason I ask is, we are changing ISPs, and I would like to see in > the logs when an external connection is coming from our OLD ISP > connection, and when it is coming through our new one. Traffic monitoring during a switchover of this nature is typically done at the edge router, not inside an individual server application. Is your router able to compile or export a daily traffic report per physical port, or raw data to your network monitoring software, showing packets/connections for TCP/UDP. Most can. This would give you the information you seek, including all traffic for both the new and old ISP, not just IMAP. -- Stan From eugene at raptor.kiev.ua Mon Oct 1 02:07:25 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 02:07:25 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50641AE7.6040201@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> Message-ID: On Thu, 27 Sep 2012 12:22:47 +0300, Davide wrote: > Hi to all, sorry in advance for my poor english, this is the first time > that i wrote to a list if i make mistake .... excuseme. > My problem is this: i have dovecot 2.1.8 installed and functioning from > 2 years one week ago i have installed crm114 for my last spam detection > filter "version 20100106-BlameMichelson (TRE 0.8.0 (BSD))" > My mail system is qmail that through .qmail default correctly tag with > X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-27CA1CFB > X-CRM114-CacheID: sfid-20120927_105129_798028_B0035817 > X-CRM114-Status: GOOD ( 28.64 ) headers ... > "Operation failed over folder 'UNSURE'. Server for account > davide.marchi at mail.cgilfe.it said: [CANNOT] Failed to call crm114 > binary.." Did you see anything in syslog? If not, let's begin with the attached patch. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix -------------- next part -------------- A non-text attachment was scrubbed... Name: crm_debug.patch Type: application/octet-stream Size: 694 bytes Desc: not available URL: From eugene at raptor.kiev.ua Mon Oct 1 02:16:25 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 02:16:25 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: References: <50641AE7.6040201@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich wrote: > Did you see anything in syslog? If not, let's begin with the attached > patch. Ouch... too sleepy. Here's the correct patch. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix -------------- next part -------------- A non-text attachment was scrubbed... Name: crm_debug.patch Type: application/octet-stream Size: 716 bytes Desc: not available URL: From tss at iki.fi Mon Oct 1 05:41:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 01 Oct 2012 05:41:00 +0300 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <1349059260.18782.42.camel@innu> On Sun, 2012-09-30 at 09:02 -0400, Charles Marcus wrote: > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? Dovecot only sees one remote IP address (%r) and one local IP address (% l) for connections. %r is already logged, but you can add %l if that helps. Other than that, I can't really help. From david.ledger at ivdcs.co.uk Mon Oct 1 10:58:30 2012 From: david.ledger at ivdcs.co.uk (David Ledger) Date: Mon, 1 Oct 2012 08:58:30 +0100 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <5068582A.6030507@brightok.net> References: <506842CB.8080501@Media-Brokers.com> <5068582A.6030507@brightok.net> Message-ID: At 09:33 -0500 30/9/12, Jack Bates wrote: >On 9/30/2012 8:02 AM, Charles Marcus wrote: >> Hi Timo/everyone, >> >> Currently we are logging the remote IP, but is there a way to show >>the IP address that the NAT connection is coming from? >> >> The reason I ask is, we are changing ISPs, and I would like to see >>in the logs when an external connection is coming from our OLD ISP >>connection, and when it is coming through our new one. >> >> We have a Watchguard firewall, and I have both External >>connections setup and working, and have just pointed our DNS >>records to the new public IP, and would like to be able to see >>which WAN connection/IP they are coming from. > >You could bind 2 internal IP Addresses to the server and have each >NAT translation go to a different internal IP. > >Jack From my remembrances of the packet layout there is nowhere in the packet for the pre-NAT address to live. The only place the mapping is stored is in the internal tables of the NAT router which has to know where to send the reply packets. David -- David Ledger - Freelance Unix Sysadmin in the UK. david.ledger at ivdcs.co.uk www.ivdcs.co.uk From CMarcus at Media-Brokers.com Mon Oct 1 11:58:15 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 01 Oct 2012 04:58:15 -0400 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <1349059260.18782.42.camel@innu> References: <506842CB.8080501@Media-Brokers.com> <1349059260.18782.42.camel@innu> Message-ID: <50695B27.8090802@Media-Brokers.com> On 2012-09-30 10:41 PM, Timo Sirainen wrote: > Dovecot only sees one remote IP address (%r) and one local IP address (% > l) for connections. %r is already logged, but you can add %l if that > helps. Other than that, I can't really help. Yeah, but that is the IP of the mail server, and since I have only one, it doesn't help any. No worries, I did see how to see this on my perimeter firewall (thanks Stan), so I can see what I'm looking for now. Thanks Timo, -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From davide.marchi at mail.cgilfe.it Mon Oct 1 12:00:14 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:00:14 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_011708_663506_8DEC5391) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) Message-ID: <50695B9E.3090502@mail.cgilfe.it> Thank you very much for the reply, i' have installed the supplied patch with the following command: - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) i compile the plugin and all go easy i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell me ..blah..blah.. [CANNOT] Failed to call crm114 binary I cant see in any log what's the problem .... Il 01/10/2012 01:16, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich > wrote: > >> Did you see anything in syslog? If not, let's begin with the attached >> patch. > > Ouch... too sleepy. Here's the correct patch. > From davide.marchi at mail.cgilfe.it Mon Oct 1 12:04:30 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:04:30 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) Message-ID: <50695C9E.5040101@mail.cgilfe.it> I downloaded and applaied the patch in the message 01:16 of 716 bytes: diff -r 7f94cc6b4d8e src/crm114.c --- a/src/crm114.c Fri May 11 04:05:59 2012 +0300 +++ b/src/crm114.c Mon Oct 01 02:15:40 2012 +0300 @@ -56,11 +56,17 @@ * really only needs the signature. */ if (pipe(pipes)) + { + i_debug("Failed to create pipes"); return -1; + } pid = fork(); if (pid < 0) + { + i_debug("Couldn't fork"); return -1; + } if (pid) { @@ -117,6 +123,7 @@ argv[i++] = spam ? cfg->spam : cfg->non_spam; + i_debug("Executing '%s %s'", cfg->binary, t_strarray_join(argv, " ")); execv(cfg->binary, (char *const *) argv); /* fall through if reaver can't be found */ i_debug("executing %s failed: %d (uid=%d, gid=%d)", cfg->binary, errno, Il 01/10/2012 11:00, Davide ha scritto: > Thank you very much for the reply, i' have installed the supplied patch > with the following command: > > - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) > i compile the plugin and all go easy > > i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell > me ..blah..blah.. [CANNOT] Failed to call crm114 binary > I cant see in any log what's the problem .... > > Il 01/10/2012 01:16, Eugene Paskevich ha scritto: >> On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich >> wrote: >> >>> Did you see anything in syslog? If not, let's begin with the attached >>> patch. >> >> Ouch... too sleepy. Here's the correct patch. >> From davide.marchi at mail.cgilfe.it Mon Oct 1 12:08:33 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:08:33 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695C9E.5040101@mail.cgilfe.it> (sfid-20121001_110525_429072_9195F28B) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) <50695C9E.5040101@mail.cgilfe.it> (sfid-20121001_110525_429072_9195F28B) Message-ID: <50695D91.3090204@mail.cgilfe.it> I have recived an empty message in response Il 01/10/2012 11:04, Davide ha scritto: > I downloaded and applaied the patch in the message 01:16 of 716 bytes: > > diff -r 7f94cc6b4d8e src/crm114.c > --- a/src/crm114.c Fri May 11 04:05:59 2012 +0300 > +++ b/src/crm114.c Mon Oct 01 02:15:40 2012 +0300 > @@ -56,11 +56,17 @@ > * really only needs the signature. > */ > if (pipe(pipes)) > + { > + i_debug("Failed to create pipes"); > return -1; > + } > > pid = fork(); > if (pid < 0) > + { > + i_debug("Couldn't fork"); > return -1; > + } > > if (pid) > { > @@ -117,6 +123,7 @@ > > argv[i++] = spam ? cfg->spam : cfg->non_spam; > > + i_debug("Executing '%s %s'", cfg->binary, t_strarray_join(argv, " ")); > execv(cfg->binary, (char *const *) argv); > /* fall through if reaver can't be found */ > i_debug("executing %s failed: %d (uid=%d, gid=%d)", cfg->binary, > errno, > > > Il 01/10/2012 11:00, Davide ha scritto: >> Thank you very much for the reply, i' have installed the supplied patch >> with the following command: >> >> - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) >> i compile the plugin and all go easy >> >> i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell >> me ..blah..blah.. [CANNOT] Failed to call crm114 binary >> I cant see in any log what's the problem .... >> >> Il 01/10/2012 01:16, Eugene Paskevich ha scritto: >>> On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich >>> wrote: >>> >>>> Did you see anything in syslog? If not, let's begin with the attached >>>> patch. >>> >>> Ouch... too sleepy. Here's the correct patch. >>> From fabio.ferrari at unimore.it Mon Oct 1 12:15:14 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Mon, 1 Oct 2012 11:15:14 +0200 Subject: [Dovecot] Problem with process_limit Message-ID: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Hello, i have a problem with the process_limit configuration. Occasionally, it happens that the dovecot.log shows this line: master: Warning: service(imap): process_limit reached, client connections are being dropped So I checked, the process number, with the command: ps auxwww | grep imap | grep -v login | wc -l and it shows 1024. Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line process_limit = 1500 I checked if the dovecot had accepted the change with the command doveconf -n | grep process_limit and it shows process_limit = 1500 But it is dropping the connections when they reach 1024 anyway, the configuration parameter is totally ignored. Can anyone help? Am I editing the right parameter? thanks in advance Fabio Ferrari P.S. The version of the dovecot is dovecot-2.0.1-1_118.el5 on Red Hat Enterprise Linux Server release 5.8 (Tikanga). From dovecot at tvetc.de Mon Oct 1 12:17:48 2012 From: dovecot at tvetc.de (Karim 'Kasi Mir' Senoucci) Date: Mon, 01 Oct 2012 11:17:48 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders Message-ID: <50695FBC.6080403@tvetc.de> Hello everyone, I recently updated to dovecot 2.0.19 (in fact, I updated the whole system to Ubuntu 12.04 LTS, I am using the dovecot from the Ubuntu packages) and just today found out that the dovecot-lda for some reason doesn't auto-create missing folders anymore as it did with my old installation (1.0.10). I have a "system users" layout and send my mails through a user-specific procmail filter. Every delivery in those filters is done via dovecot-lda using a line list this: > |$DELIVER -d archive -m lists.hylafax-`date "+%Y%m"` where $DELIVER is set to > DELIVER="/usr/bin/sudo /usr/lib/dovecot/dovecot-lda" (I use the sudo because some of the procmail lines deliver the mail to folders in a different user's mailbox.) This works fine for all users, unless the mail folder to be delivered to doesn't exist yet. Here's an example of such a delivery attempt: > Oct 01 10:57:09 lda: Debug: auth input: archive > system_groups_user=archive uid=1002 gid=1002 home=/home/archive > Oct 01 10:57:09 lda(archive): Debug: Effective uid=1002, gid=1002, > home=/home/archive > Oct 01 10:57:09 lda(archive): Debug: Namespace : type=private, > prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes > location=maildir:/home/archive/Maildir > Oct 01 10:57:09 lda(archive): Debug: maildir++: > root=/home/archive/Maildir, index=, control=, > inbox=/home/archive/Maildir, alt= > Oct 01 10:57:09 lda(archive): Debug: none: root=, index=, control=, > inbox=, alt= > Oct 01 10:57:09 lda(archive): Debug: Destination address: xxx at xxxx.de > (source: user at hostname) > Oct 01 10:57:09 lda(archive): Debug: Namespace : > /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using > default permissions > Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions > from /home/archive/Maildir: mode=0755 gid=-1 > Oct 01 10:57:09 lda(archive): Debug: Namespace : > /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using > default permissions > Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions > from /home/archive/Maildir: mode=0755 gid=-1 > Oct 01 10:57:09 lda(archive): Info: > msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: save failed > to open mailbox lists.hylafax-201210: Mailbox doesn't exist: > lists.hylafax-201210 > Oct 01 10:57:09 lda(archive): Info: > msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: saved mail to > INBOX Can anyone tell me what goes wrong here and how to fix it? From what I could find out dovecot-lda should auto-create those mailbox folders, but somehow it doesn't. Any help is appreciated. Thanks in advance Kasi Mir From eugene at raptor.kiev.ua Mon Oct 1 12:20:06 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 12:20:06 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695B9E.3090502@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 12:00:14 +0300, Davide wrote: > Thank you very much for the reply, i' have installed the supplied patch > with the following command: > > - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) > i compile the plugin and all go easy > > i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell > me ..blah..blah.. [CANNOT] Failed to call crm114 binary > I cant see in any log what's the problem .... Weird... Did you configure anything specific about logging? doveconf -n might show. You should also check your syslog configuration as to where debugging logging should go to. Otherwise I have no idea. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From robert at schetterer.org Mon Oct 1 12:46:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 01 Oct 2012 11:46:58 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders In-Reply-To: <50695FBC.6080403@tvetc.de> References: <50695FBC.6080403@tvetc.de> Message-ID: <50696692.3050700@schetterer.org> Am 01.10.2012 11:17, schrieb Karim 'Kasi Mir' Senoucci: > Hello everyone, > I recently updated to dovecot 2.0.19 (in fact, I updated the whole > system to Ubuntu 12.04 LTS, I am using the dovecot from the Ubuntu > packages) and just today found out that the dovecot-lda for some reason > doesn't auto-create missing folders anymore as it did with my old > installation (1.0.10). > > I have a "system users" layout and send my mails through a user-specific > procmail filter. Every delivery in those filters is done via dovecot-lda > using a line list this: > >> |$DELIVER -d archive -m lists.hylafax-`date "+%Y%m"` > > where $DELIVER is set to > >> DELIVER="/usr/bin/sudo /usr/lib/dovecot/dovecot-lda" > > (I use the sudo because some of the procmail lines deliver the mail to > folders in a different user's mailbox.) > > This works fine for all users, unless the mail folder to be delivered to > doesn't exist yet. Here's an example of such a delivery attempt: > >> Oct 01 10:57:09 lda: Debug: auth input: archive >> system_groups_user=archive uid=1002 gid=1002 home=/home/archive >> Oct 01 10:57:09 lda(archive): Debug: Effective uid=1002, gid=1002, >> home=/home/archive >> Oct 01 10:57:09 lda(archive): Debug: Namespace : type=private, >> prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes >> location=maildir:/home/archive/Maildir >> Oct 01 10:57:09 lda(archive): Debug: maildir++: >> root=/home/archive/Maildir, index=, control=, >> inbox=/home/archive/Maildir, alt= >> Oct 01 10:57:09 lda(archive): Debug: none: root=, index=, control=, >> inbox=, alt= >> Oct 01 10:57:09 lda(archive): Debug: Destination address: xxx at xxxx.de >> (source: user at hostname) >> Oct 01 10:57:09 lda(archive): Debug: Namespace : >> /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using >> default permissions >> Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions >> from /home/archive/Maildir: mode=0755 gid=-1 >> Oct 01 10:57:09 lda(archive): Debug: Namespace : >> /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using >> default permissions >> Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions >> from /home/archive/Maildir: mode=0755 gid=-1 >> Oct 01 10:57:09 lda(archive): Info: >> msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: save failed >> to open mailbox lists.hylafax-201210: Mailbox doesn't exist: >> lists.hylafax-201210 >> Oct 01 10:57:09 lda(archive): Info: >> msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: saved mail to >> INBOX > > Can anyone tell me what goes wrong here and how to fix it? From what I > could find out dovecot-lda should auto-create those mailbox folders, but > somehow it doesn't. Any help is appreciated. > > Thanks in advance > Kasi Mir my bet goes here http://wiki2.dovecot.org/LDA parameters lda_mailbox_autocreate lda_mailbox_autosubscribe -- Best Regards MfG Robert Schetterer From davide.marchi at mail.cgilfe.it Mon Oct 1 12:54:01 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:54:01 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_112051_451409_99CBB428) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_112051_451409_99CBB428) Message-ID: <50696839.7090901@mail.cgilfe.it> this is my dovecot configuration for antispam plugin logging: antispam_verbose_debug = 1 antispam_debug_target = syslog and this is info_log_path = /var/log/dovecot/dovecot.log log_path = /var/log/dovecot/dovecot-err.log debug_log_path = Il 01/10/2012 11:20, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:00:14 +0300, Davide > wrote: > >> Thank you very much for the reply, i' have installed the supplied >> patch with the following command: >> >> - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) >> i compile the plugin and all go easy >> >> i move my wrongly tagged mail from UNSURE to INBOX and thunderbird >> tell me ..blah..blah.. [CANNOT] Failed to call crm114 binary >> I cant see in any log what's the problem .... > > Weird... Did you configure anything specific about logging? > doveconf -n might show. You should also check your syslog configuration > as to where debugging logging should go to. Otherwise I have no idea. > From dovecot at tvetc.de Mon Oct 1 13:01:39 2012 From: dovecot at tvetc.de (Karim 'Kasi Mir' Senoucci) Date: Mon, 01 Oct 2012 12:01:39 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders In-Reply-To: <50696692.3050700@schetterer.org> References: <50695FBC.6080403@tvetc.de> <50696692.3050700@schetterer.org> Message-ID: <50696A03.3080605@tvetc.de> Hello everyone, Am 01.10.2012 11:46, schrieb Robert Schetterer: >> Can anyone tell me what goes wrong here and how to fix it? From what I >> could find out dovecot-lda should auto-create those mailbox folders, but >> somehow it doesn't. Any help is appreciated. > my bet goes here > > parameters > > lda_mailbox_autocreate > lda_mailbox_autosubscribe D*mn, I fell into the trap again. I was searching for such parameters before posting my question here, but couldn't find them - because I was looking in the 1.x docmentation, not the 2.x one. Thanks for pointing that out to me. Greetings Kasi Mir From eugene at raptor.kiev.ua Mon Oct 1 13:07:48 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 13:07:48 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50696839.7090901@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 12:54:01 +0300, Davide wrote: > this is my dovecot configuration for antispam plugin logging: > antispam_verbose_debug = 1 > antispam_debug_target = syslog Neither my plugin nor (I believe) main dovecot reads those two. You can remove them. > and this is > info_log_path = /var/log/dovecot/dovecot.log > log_path = /var/log/dovecot/dovecot-err.log > debug_log_path = Did you notice that patch is mainly for i_debug() invocations? :-) Try sitting debug_log_path to something meaningful. And of course restart dovecot. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From davide.marchi at mail.cgilfe.it Mon Oct 1 13:57:14 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 12:57:14 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_120839_719900_279FDE63) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> (sfid-20121001_120839_719900_279FDE63) Message-ID: <5069770A.5030007@mail.cgilfe.it> I deleted antispam_verbose_debug = 1 antispam_debug_target = syslog from dovecot.conf and i added debug_log_path = /var/log/dovecot/dovecot_debug.log restarted dovecot and now i see in the debug log 2012-10-01 12:33:31 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib90_antispam_plugin.so but nothing inherit [CANNOT] Failed to call crm114 binary Il 01/10/2012 12:07, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:54:01 +0300, Davide > wrote: > >> this is my dovecot configuration for antispam plugin logging: >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog > > Neither my plugin nor (I believe) main dovecot reads those two. You can > remove them. > >> and this is >> info_log_path = /var/log/dovecot/dovecot.log >> log_path = /var/log/dovecot/dovecot-err.log >> debug_log_path = > > Did you notice that patch is mainly for i_debug() invocations? :-) > Try sitting debug_log_path to something meaningful. And of course > restart dovecot. > From davide.marchi at mail.cgilfe.it Mon Oct 1 13:58:23 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 12:58:23 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_120839_719900_279FDE63) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> (sfid-20121001_120839_719900_279FDE63) Message-ID: <5069774F.30209@mail.cgilfe.it> Parameters i.e. antispam_crm_binary = /opt/crm114/mailreaver.crm must be enclosed on ""? so antispam_crm_binary = "/opt/crm114/mailreaver.crm" Il 01/10/2012 12:07, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:54:01 +0300, Davide > wrote: > >> this is my dovecot configuration for antispam plugin logging: >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog > > Neither my plugin nor (I believe) main dovecot reads those two. You can > remove them. > >> and this is >> info_log_path = /var/log/dovecot/dovecot.log >> log_path = /var/log/dovecot/dovecot-err.log >> debug_log_path = > > Did you notice that patch is mainly for i_debug() invocations? :-) > Try sitting debug_log_path to something meaningful. And of course > restart dovecot. > From eugene at raptor.kiev.ua Mon Oct 1 14:00:22 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 14:00:22 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069774F.30209@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069774F.30209@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 13:58:23 +0300, Davide wrote: > Parameters i.e. antispam_crm_binary = /opt/crm114/mailreaver.crm > must be enclosed on ""? so > antispam_crm_binary = "/opt/crm114/mailreaver.crm" No. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From eugene at raptor.kiev.ua Mon Oct 1 14:01:14 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 14:01:14 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069770A.5030007@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 13:57:14 +0300, Davide wrote: > I deleted > antispam_verbose_debug = 1 > antispam_debug_target = syslog > from dovecot.conf and i added > > debug_log_path = /var/log/dovecot/dovecot_debug.log > > restarted dovecot and now i see in the debug log > > 2012-10-01 12:33:31 imap: Debug: Module loaded: > /usr/local/lib/dovecot/lib90_antispam_plugin.so > > but nothing inherit [CANNOT] Failed to call crm114 binary Could you please post the contents of the debug file somewhere? -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From davide.marchi at mail.cgilfe.it Mon Oct 1 14:48:31 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 13:48:31 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_130205_017099_DD5462AE) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> (sfid-20121001_130205_017099_DD5462AE) Message-ID: <5069830F.4060401@mail.cgilfe.it> Can i attach compressed log in a post? I can produce output replacing crm binary with a script bash to ouput command,user etc etc Il 01/10/2012 13:01, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 13:57:14 +0300, Davide > wrote: > >> I deleted >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog >> from dovecot.conf and i added >> >> debug_log_path = /var/log/dovecot/dovecot_debug.log >> >> restarted dovecot and now i see in the debug log >> >> 2012-10-01 12:33:31 imap: Debug: Module loaded: >> /usr/local/lib/dovecot/lib90_antispam_plugin.so >> >> but nothing inherit [CANNOT] Failed to call crm114 binary > > Could you please post the contents of the debug file somewhere? > From davide.marchi at mail.cgilfe.it Mon Oct 1 14:53:08 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 13:53:08 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_134931_162985_54BB8C15) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> (sfid-20121001_130205_017099_DD5462AE) <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_134931_162985_54BB8C15) Message-ID: <50698424.8080208@mail.cgilfe.it> Restarting Dovecot instead to use doveadm reload i hav strange error: managesieve(root): Fatal: getcwd() failed: No such file or directory doveconf: Error: managesieve-login: dump-capability process returned 89 Il 01/10/2012 13:48, Davide ha scritto: > Can i attach compressed log in a post? > I can produce output replacing crm binary with a script bash to ouput > command,user etc etc > > > Il 01/10/2012 13:01, Eugene Paskevich ha scritto: >> On Mon, 01 Oct 2012 13:57:14 +0300, Davide >> wrote: >> >>> I deleted >>> antispam_verbose_debug = 1 >>> antispam_debug_target = syslog >>> from dovecot.conf and i added >>> >>> debug_log_path = /var/log/dovecot/dovecot_debug.log >>> >>> restarted dovecot and now i see in the debug log >>> >>> 2012-10-01 12:33:31 imap: Debug: Module loaded: >>> /usr/local/lib/dovecot/lib90_antispam_plugin.so >>> >>> but nothing inherit [CANNOT] Failed to call crm114 binary >> >> Could you please post the contents of the debug file somewhere? >> From eugene at raptor.kiev.ua Mon Oct 1 15:04:50 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 15:04:50 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069830F.4060401@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 14:48:31 +0300, Davide wrote: > Can i attach compressed log in a post? If it's of the sane size :-) But I'm actually interested in the output which was triggered by the mail move itself. > I can produce output replacing crm binary with a script bash to ouput > command,user etc etc Wait a second... So the script is ran correctly but reaver isn't? That probably means that either reaver dislikes its arguments or its stdin... -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From jbates at brightok.net Mon Oct 1 15:23:38 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 07:23:38 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: References: <506842CB.8080501@Media-Brokers.com> <5068582A.6030507@brightok.net> Message-ID: <50698B4A.7090604@brightok.net> On 10/1/2012 2:58 AM, David Ledger wrote: > At 09:33 -0500 30/9/12, Jack Bates wrote: >> On 9/30/2012 8:02 AM, Charles Marcus wrote: >>> Hi Timo/everyone, >>> >>> Currently we are logging the remote IP, but is there a way to show >>> the IP address that the NAT connection is coming from? >>> >>> The reason I ask is, we are changing ISPs, and I would like to see >>> in the logs when an external connection is coming from our OLD ISP >>> connection, and when it is coming through our new one. >>> >>> We have a Watchguard firewall, and I have both External connections >>> setup and working, and have just pointed our DNS records to the new >>> public IP, and would like to be able to see which WAN connection/IP >>> they are coming from. >> >> You could bind 2 internal IP Addresses to the server and have each >> NAT translation go to a different internal IP. >> >> Jack > > From my remembrances of the packet layout there is nowhere in the > packet for the pre-NAT address to live. The only place the mapping is > stored is in the internal tables of the NAT router which has to know > where to send the reply packets. > > David > > Public IP1 -> 192.168.0.33 Public IP2 -> 192.168.0.34 Now we just track the internal address in our logs, since each public IP is mapped to a different internal IP. Jack From jbates at brightok.net Mon Oct 1 15:35:03 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 07:35:03 -0500 Subject: [Dovecot] Problem with process_limit In-Reply-To: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: <50698DF7.60209@brightok.net> On 10/1/2012 4:15 AM, FABIO FERRARI wrote: > Hello, > > i have a problem with the process_limit configuration. > > Occasionally, it happens that the dovecot.log shows this line: > master: Warning: service(imap): process_limit reached, client connections > are being dropped > > So I checked, the process number, with the command: > ps auxwww | grep imap | grep -v login | wc -l > and it shows 1024. > > Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line > process_limit = 1500 > > I checked if the dovecot had accepted the change with the command > doveconf -n | grep process_limit > and it shows > process_limit = 1500 > > But it is dropping the connections when they reach 1024 anyway, the > configuration parameter is totally ignored. > > Can anyone help? Am I editing the right parameter? > > thanks in advance > > Fabio Ferrari > > P.S. The version of the dovecot is dovecot-2.0.1-1_118.el5 on Red Hat > Enterprise Linux Server release 5.8 (Tikanga). Don't forget to change the operating system limit as well. I added these lines to my /etc/sysconfig/dovecot on rhel6. I compiled dovecot myself, but I package similar to the system version. ulimit -n 4096 ulimit -u 5120 This increases the filehandles allowed by dovecot to 4096 and increases the number of processes per user to 5120. This is a proxy server, so I needed to support much larger numbers. than a silly 1024. Jack From davide.marchi at mail.cgilfe.it Mon Oct 1 18:46:53 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 17:46:53 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_143220_258050_A12B6D1E) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_143220_258050_A12B6D1E) Message-ID: <5069BAED.8020206@mail.cgilfe.it> I made an experiment because iwasn't able to output some error to syslog or stderr about my configuration... I've installed antispam plugin in a parallel server with the program of johannes Bergs that seem now upgraded to function with 2.X and i have same problem ("Failed to call crm114 binary.." in antispam dovecot from your program, Eugene, and "Failed to call reaver.." from Johannes program) but now i'm able to output to log with this config: antispam_debug_target = syslog antispam_debug_target = stderr antispam_verbose_debug = 1 antispam_backend = crm114 antispam_spam_pattern_ignorecase = spam* antispam_trash_pattern_ignorecase = trash* antispam_unsure_pattern_ignorecase = unsure* antispam_signature = X-CRM114-CacheID antispam_signature_missing = error antispam_crm_binary = /usr/local/bin/piper_log.sh antispam_crm_spam_arg = --spam antispam_crm_notspam_arg = --good # antispam_crm_binary = /opt/crm114/mailreaver.crm antispam_crm_args = -u;%h;--fileprefix=/opt/crm114/ # antispam_signature = X-CRM114-CacheID in dovecot-err.log now i have 2012-10-01 17:39:35 imap: Error: antispam: plugin initialising (2.0-4-g83b0b4b-dirty) 2012-10-01 17:39:35 imap: Error: antispam: verbose debug enabled 2012-10-01 17:39:35 imap: Error: antispam: "SPAM" is exact match spam folder 2012-10-01 17:39:35 imap: Error: antispam: "UNSURE" is exact match unsure folder 2012-10-01 17:39:35 imap: Error: antispam: "Trash" is exact match trash folder 2012-10-01 17:39:35 imap: Error: antispam: reaver binary set to /usr/local/bin/piper_log.sh 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg -u 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg /home/vpopmail/domains/mail.cgilfe.it/davide.marchi 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg --fileprefix=/opt/crm114/ 2012-10-01 17:39:35 imap: Error: antispam: signature header line is "X-CRM114-CacheID" 2012-10-01 17:39:39 imap: Error: antispam: plugin initialising (2.0-4-g83b0b4b-dirty) 2012-10-01 17:39:39 imap: Error: antispam: verbose debug enabled . . . 2012-10-01 17:39:42 imap: Error: antispam: mail copy: from trash: 0, to trash: 0 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_spam(SPAM): 1 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_spam(INBOX): 0 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_unsure(SPAM): 0 2012-10-01 17:39:42 imap: Error: antispam: mail copy: src spam: 1, dst spam: 0, src unsure: 0 (i moved a mail from SPAM to INBOX) this is the output for "call command crm args" /opt/crm114/mailreaver.crm --good -u /home/vpopmail/domains/mail.cgilfe.it/davide.marchi --fileprefix=/opt/crm114/ if i exec this command with user vpopmail in console the command is successful Il 01/10/2012 14:04, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 14:48:31 +0300, Davide > wrote: > >> Can i attach compressed log in a post? > > If it's of the sane size :-) > But I'm actually interested in the output which was triggered by the > mail move itself. > >> I can produce output replacing crm binary with a script bash to ouput >> command,user etc etc > > Wait a second... So the script is ran correctly but reaver isn't? > That probably means that either reaver dislikes its arguments or its > stdin... > From fabiodepin at gmail.com Mon Oct 1 22:20:50 2012 From: fabiodepin at gmail.com (Fabio Depin) Date: Mon, 1 Oct 2012 16:20:50 -0300 Subject: [Dovecot] BUG to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2 Message-ID: Hello, Today I needed to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2. When running 'make' getting the following error: -------------------------------------------------- -------- db-checkpassword.c: In function 'sigchld_handler': db-checkpassword.c: 426: error: assignment of read-only member '__in' db-checkpassword.c: 429: error: assignment of read-only member '__in' db-checkpassword.c: 431: error: assignment of read-only member '__in' db-checkpassword.c: 432: error: assignment of read-only member '__in' make [3]: ** [db-checkpassword.o] Error 1 make [3]: ** Waiting for other processes to finish. mv-f .deps / auth-worker-server.Tpo .deps / auth-worker-server.Po make [3]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src/auth ' make [2]: ** [all-recursive] Error 1 make [2]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src ' make [1]: ** [all-recursive] Error 1 make [1]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7 ' make: ** [all] Error 2 -------------------------------------------------- -------- -To work did the following: -------------------------------------------------- -------- 417a418 + Int stat = status-> status; 426c427 - If (WIFSIGNALED (status-> status)) { --- + If (WIFSIGNALED (stat)) { 429c430 - Dec2str (status-> pid), WTERMSIG (status-> status)); --- + Dec2str (status-> pid), WTERMSIG (stat)); 431.432 c432, 433 -} Else if (WIFEXITED (status-> status)) { - Request-> exit_status WEXITSTATUS = (status-> status); --- +} Else if (WIFEXITED (stat)) { + Request-> exit_status WEXITSTATUS = (stat); -------------------------------------------------- -------- With this change worked perfectly ntanto in debian 4 with gcc 4.1.2, as in debian 6 with gcc 4.4.5. -I wonder if I made the change may affect any function of dovecot, or if it is correct. Thank you for your attention. Fabio Depin From joe at netmusician.org Mon Oct 1 22:34:25 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 01 Oct 2012 15:34:25 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <506453CE.7000608@gmail.com> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> <506453CE.7000608@gmail.com> Message-ID: <5069F041.6060904@netmusician.org> Are performance issues a possible symptom of this problem? It was mentioned that this happens after disconnects, but does this break IDLE? > Birta Levente > September 27, 2012 9:25 AM > > Hi Timo > > I just want to ask you: this issue is still in your task list? > If you need more debug information please tell me how can I give you. > > Thanks, > Levi > > Timo Sirainen > September 24, 2012 10:58 AM > > Show your doveconf -n output at least. As for debugging information, > that would depend on how you installed Dovecot? From some RPM or sources? > > Birta Levente > September 24, 2012 10:55 AM > > > I have the same problem, but on centos 6.3 64bit. How can I give you > the debug information? > > Levi > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: postbox-contact.jpg Type: image/jpeg Size: 1305 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Mon Oct 1 23:13:50 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 01 Oct 2012 16:13:50 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <5069F041.6060904@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> <506453CE.7000608@gmail.com> <5069F041.6060904@netmusician.org> Message-ID: <5069F97E.5020900@netmusician.org> Are performance issues a possible symptom of this problem? It was mentioned that this happens after disconnects, but does this break IDLE? > > Birta Levente > September 27, 2012 9:25 AM > > Hi Timo > > I just want to ask you: this issue is still in your task list? > If you need more debug information please tell me how can I give you. > > Thanks, > Levi > > > Timo Sirainen > September 24, 2012 10:58 AM > > Show your doveconf -n output at least. As for debugging information, > that would depend on how you installed Dovecot? From some RPM or sources? > > > Birta Levente > September 24, 2012 10:55 AM > > > I have the same problem, but on centos 6.3 64bit. How can I give you > the debug information? > > Levi > > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------ From dovecot-in at keystealth.org Mon Oct 1 23:36:25 2012 From: dovecot-in at keystealth.org (Scott Neville) Date: Mon, 1 Oct 2012 13:36:25 -0700 (PDT) Subject: [Dovecot] Logging IP address for failed login Message-ID: Hi, I am trying to use the logs to show the IP that brute force activity comes from, but Im not succeeding. I have read the archives and seen the advice others have had. I can see logs for repeated bad logins, but I need the IP address from the attempts. dovecot 2.0.12 / CentOS 5.4 / imaps only (993) I have tried a bunch of different combinations of 10-logging.conf settings. This is what I have currently (that does not work the way I want): auth_verbose = yes #auth_verbose_passwords = no #auth_debug = yes #auth_debug_passwords = no #mail_debug = no I *dont* want to see the passwords, either failed or successful. I just want to see failed logins for whatever reason and the IP they came from. In /var/log/maillog I get lines like this: Oct 1 04:19:12 olive dovecot: auth: pam(marketing): unknown user Oct 1 04:19:17 olive dovecot: auth: pam(marketing): unknown user When i had debugging turned on, I would get lines like this: Sep 9 01:14:59 olive dovecot: auth: Debug: passwd(dbelan,62.128.300.94): lookup but only for successful logins. The brute force attempts dont log like that: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:58 olive dovecot: auth: pam(backup): unknown user No IP anywhere in that. fail2ban seems to rely on the pop-login or imap-login lines to pull the IP from. I get an imap-login for my real logins: Oct 1 12:38:56 olive dovecot: imap-login: Login: user=, method=PLAIN, rip=62.128.300.94, lip=204.152.189.165, mpid=20360, TLS but no similar line for the failed logins. So is this a dovecot logging configuration combination I need to find? Is it getting lost in pam? Is it specific to CentOS? Any help appreciated - happy to read up on it myself, but would need a pointer, since the docs so far either assume I get an imap-login line for failed logins which I dont, or they assume I just want to see the repeated attempts/passwords. Scott. From fabiodepin at gmail.com Mon Oct 1 23:51:39 2012 From: fabiodepin at gmail.com (Fabio Depin) Date: Mon, 1 Oct 2012 17:51:39 -0300 Subject: [Dovecot] BUG to compile Plugin - deleted-to-trash on dovecot 2.1+ Message-ID: Hello, Today I needed to compile the plugin deleted-to-trash for dovecot 1.2.10 (> 2.1) and had problems compiling. To solve the problem efetuei changes below: -------------------------------------------------- -------------------- deleted-to-trash-plugin.c 79.80 C79 - Mailbox_alloc box = (list, name, MAILBOX_FLAG_KEEP_RECENT | - MAILBOX_FLAG_NO_INDEX_FILES); --- + Box = mailbox_alloc (list, name, MAILBOX_FLAG_NO_INDEX_FILES); 136c135 - If (keywords! = NULL) mailbox_keywords_unref (trash_box, & keywords); --- + If (keywords! = NULL) mailbox_keywords_unref (& keywords); -------------------------------------------------- -------------------- I would like to verify that this is correct, or can generate a problem. Thank you for your attention. Fabio Depin From jbates at brightok.net Tue Oct 2 00:15:54 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 16:15:54 -0500 Subject: [Dovecot] Logging IP address for failed login In-Reply-To: References: Message-ID: <506A080A.60906@brightok.net> On 10/1/2012 3:36 PM, Scott Neville wrote: > > In /var/log/maillog I get lines like this: > Oct 1 04:19:12 olive dovecot: auth: pam(marketing): unknown user > Oct 1 04:19:17 olive dovecot: auth: pam(marketing): unknown user > I'm guessing you are using a centos package. This may be package version specific. Here is RHEL6's dovecot 2.0.9 default except for setting auth_verbose = yes. Sep 28 21:12:10 compiler dovecot: auth: pam(test,::1): unknown user Sep 28 21:12:24 compiler dovecot: auth: pam(validuser,::1): pam_authenticate() failed: Authentication failure (password mismatch?) 2.1.9/2.1.10 which I packaged shows similar. Since I connected localhost, the IP is IPv6, of course. Jack From jbates at brightok.net Tue Oct 2 04:42:36 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 20:42:36 -0500 Subject: [Dovecot] BUG to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2 In-Reply-To: References: Message-ID: <506A468C.10505@brightok.net> It looks like this might be a bug in glibc 2.3.3 http://sourceware.org/bugzilla/show_bug.cgi?id=1392 Jack On 10/1/2012 2:20 PM, Fabio Depin wrote: > Hello, > > Today I needed to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2. > When running 'make' getting the following error: > -------------------------------------------------- -------- > db-checkpassword.c: In function 'sigchld_handler': > db-checkpassword.c: 426: error: assignment of read-only member '__in' > db-checkpassword.c: 429: error: assignment of read-only member '__in' > db-checkpassword.c: 431: error: assignment of read-only member '__in' > db-checkpassword.c: 432: error: assignment of read-only member '__in' > make [3]: ** [db-checkpassword.o] Error 1 > make [3]: ** Waiting for other processes to finish. > mv-f .deps / auth-worker-server.Tpo .deps / auth-worker-server.Po > make [3]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src/auth ' > make [2]: ** [all-recursive] Error 1 > make [2]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src ' > make [1]: ** [all-recursive] Error 1 > make [1]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7 ' > make: ** [all] Error 2 > -------------------------------------------------- -------- > > -To work did the following: > -------------------------------------------------- -------- > 417a418 > + Int stat = status-> status; > 426c427 > - If (WIFSIGNALED (status-> status)) { > --- > + If (WIFSIGNALED (stat)) { > 429c430 > - Dec2str (status-> pid), WTERMSIG (status-> status)); > --- > + Dec2str (status-> pid), WTERMSIG (stat)); > 431.432 c432, 433 > -} Else if (WIFEXITED (status-> status)) { > - Request-> exit_status WEXITSTATUS = (status-> status); > --- > +} Else if (WIFEXITED (stat)) { > + Request-> exit_status WEXITSTATUS = (stat); > -------------------------------------------------- -------- > > With this change worked perfectly ntanto in debian 4 with gcc 4.1.2, as in > debian 6 with gcc 4.4.5. > -I wonder if I made the change may affect any function of dovecot, or if it is > correct. > > Thank you for your attention. > Fabio Depin > From amateo at um.es Tue Oct 2 11:41:51 2012 From: amateo at um.es (Angel L. Mateo) Date: Tue, 02 Oct 2012 10:41:51 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> Message-ID: <506AA8CF.8090605@um.es> Hello, I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: root at myotis30:/etc/dovecot/conf.d# doveadm user angel.luis at um.es userdb: angel.luis at um.es mail : mdbox:/home/alumnos/46/113246/mdbox:INDEX=/var/indexes/mdbox/angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=10G root at myotis30:/etc/dovecot/conf.d# doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=10G I guess I'm using different keys depending the user database used. I have configured three user databases, one for master-password, one for a ldap server and the other with pam (I need it because my webmail users authenticate in my SSO system through PAM). This is my config: passdb { driver = passwd-file master = yes args = /etc/dovecot/master-users # Unless you're using PAM, you probably still want the destination user to # be looked up from passdb that it really exists. pass=yes does that. pass = yes } passdb { driver = pam # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=] # [cache_key=] [] #args = dovecot args = session=yes cache_key=%n dovecot } passdb { driver = ldap # Path for LDAP configuration file, see example-config/dovecot-ldap.conf.ext args = /etc/dovecot/dovecot-ldap.conf.ext } # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # userdb { driver = prefetch } userdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf.ext # Default fields can be used to specify defaults that LDAP may override #default_fields = home=/home/virtual/%u } In my ldap configuration, I have a filter that looks for the uid of the user or the hole email: user_filter = (&()(|(uid=%u)(mail=%u))) I need this, because I have users that authenticate with just his/her login, not the complete email address. How can I unify those entries, so they use always just the login as key? El 18/09/12 18:31, Timo Sirainen escribi?: > On 18.9.2012, at 9.59, Angel L. Mateo wrote: > >>>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >>> >>> Works in my tests. >>> >> Is this cache the same than the user information cache? > > Yes. > >> The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: >> >> doveadm auth cache flush > > What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: > > doveadm auth cache flush foo # make sure it isn't there > doveadm user foo > doveadm auth cache flush foo > > Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. > > You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. > -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.4.0-030400-generic x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1000 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_exceeded_message = El mensaje no se ha entregado porque el destinatario del mismo tiene el buz?n lleno. quota_rule = *:storage=20G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 2003 } service auth { client_limit = 3000 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 2 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 2 } ssl = no ssl_cert = } From davide.marchi at mail.cgilfe.it Tue Oct 2 11:57:33 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Tue, 02 Oct 2012 10:57:33 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_143220_258050_A12B6D1E) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_143220_258050_A12B6D1E) Message-ID: <506AAC7D.8060408@mail.cgilfe.it> I'm unable to output nothing to syslog nor in other dovecot's log files about problem of plugin. If i use the test server the output is this: Oct 2 10:38:34 debnew imap: antispam: mailbox_is_unsure(SPAM): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_trash(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_trash(SPAM): 0 Oct 2 10:38:34 debnew imap: antispam: mail copy: from trash: 0, to trash: 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_spam(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_spam(SPAM): 1 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_unsure(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Oct 2 10:38:34 debnew imap: antispam: /usr/local/bin/piper_log.sh --spam -u /home/vpopmail/domains/mail.cgilfe.it/davide.marchi --fileprefix=/opt/crm114/ and this is correct but anyway thunderbird say [CANNOT] Failed to call reaver Il 01/10/2012 14:04, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 14:48:31 +0300, Davide > wrote: > >> Can i attach compressed log in a post? > > If it's of the sane size :-) > But I'm actually interested in the output which was triggered by the > mail move itself. > >> I can produce output replacing crm binary with a script bash to ouput >> command,user etc etc > > Wait a second... So the script is ran correctly but reaver isn't? > That probably means that either reaver dislikes its arguments or its > stdin... > From tss at iki.fi Tue Oct 2 21:37:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:37:09 +0300 Subject: [Dovecot] LTMP Proxy failure fix/hack In-Reply-To: <50660897.6040008@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> <506604BC.6050503@brightok.net> <50660897.6040008@brightok.net> Message-ID: On 28.9.2012, at 23.29, Jack Bates wrote: > On 9/28/2012 3:12 PM, Jack Bates wrote: >> >> Code needs to be written to handle the special case of us not having any proxy callbacks as they are all bad. >> > > > Timo, please check and approve. This was diff'd on 2.1.10 on my test server (2.1.9 and 2.1.10 at least had this callback issue). > > *** lmtp-proxy.c-orig 2012-09-28 20:17:36.138916678 +0000 > --- lmtp-proxy.c 2012-09-28 20:18:12.241940780 +0000 > *************** > *** 300,303 **** > --- 300,304 ---- > lmtp_client_send(conn->client, conn->data_input); > lmtp_client_send_more(conn->client); > } > + lmtp_proxy_try_finish(proxy); > } Looks ok. Added: http://hg.dovecot.org/dovecot-2.1/rev/38727d3e90ec From tss at iki.fi Tue Oct 2 21:41:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:41:12 +0300 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924184157.GA75341@kyoko.org> References: <20120924134810.GA62723@kyoko.org> <20120924150440.GA85969@kyoko.org> <20120924184157.GA75341@kyoko.org> Message-ID: On 24.9.2012, at 21.41, Philippe Chevalier wrote: > As for the ldap message, it errors if there's no domain in the login. > > In the doc, it says that %d is empty if there's no domain part. So I > guess it's an enhancement request : a configuration option to have it > filled out with a default domain if there's no one supplied by the > client. Maybe this is enough? auth_bind_userdn = dc=%Du,ou=Domains,ou=Mail,dc=dspnet,dc=fr See %D in http://wiki2.dovecot.org/Variables From listas at adminlinux.com.br Tue Oct 2 21:45:39 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Tue, 02 Oct 2012 15:45:39 -0300 Subject: [Dovecot] About ManageSieve and TLS Message-ID: <506B3653.5020804@adminlinux.com.br> Hi, I have a "ubuntu10.04 + dovecot-2.0.13" configuration in my server. It works fine with ~50k accounts. Recently I enabled TLS: $ cat /etc/dovecot/dovecot.conf ... # Use SSL ? ssl = yes ... The goal was to provide only IMAPS and POP3S. But Managesieve says "STARTTLS": $ telnet _MY_IP_ 2000 Trying _MY_IP_... Connected to _MY_IP_. Escape character is '^]'. "IMPLEMENTATION" "K8 ManageSieve" "SIEVE" "comparator-i;ascii-numeric copy envelope fileinto imapflags include notify regex reject relational subaddress vacation" "SASL" "PLAIN LOGIN" "STARTTLS" "VERSION" "1.0" OK "K8 IMAP/POP3 server" doveconf -a shows: service managesieve-login { ... inet_listener sieve { address = _MY_IP_ port = 4190 ssl = no } inet_listener sieve_deprecated { address = _MY_IP_ port = 2000 ssl = no } ... } I think there is something wrong there but I don't know. I think Managesieve should not say "STARTTLS". Can someone help me? Thanks. -- Thiago Henrique adminlinux.com.br From tss at iki.fi Tue Oct 2 21:56:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:56:33 +0300 Subject: [Dovecot] Spurious " Renaming not supported across conflicting directory permissions" In-Reply-To: <505DA946.90409@yahoo.com> References: <505DA946.90409@yahoo.com> Message-ID: On 22.9.2012, at 15.04, tlhackque wrote: > Dovecot 2.1.10 > Client = Thunderbird. Local disks. mbox format. > > Attempted to rename a folder, failed with: > > CANNOT Renaming not supported across conflicting directory permissions Fixed: http://hg.dovecot.org/dovecot-2.1/rev/83695d6d41aa From tss at iki.fi Tue Oct 2 22:28:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:28:22 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C393.5050209@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: On 19.9.2012, at 16.07, Alessio Cecchi wrote: > #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( > backend=0x1944240) at acl-backend-vfile-acllist.c:297 This backtrace is rather weird. Could you also do (instead of bt full): fr 1 p *ns p *ns.user p *auser It crashes because auser->dict = NULL, but it should never be NULL. From stephan at rename-it.nl Tue Oct 2 22:31:05 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 02 Oct 2012 21:31:05 +0200 Subject: [Dovecot] About ManageSieve and TLS In-Reply-To: <506B3653.5020804@adminlinux.com.br> References: <506B3653.5020804@adminlinux.com.br> Message-ID: <506B40F9.1000905@rename-it.nl> On 10/2/2012 8:45 PM, 3.listas at adminlinux.com.br wrote: > Hi, > > I have a "ubuntu10.04 + dovecot-2.0.13" configuration in my server. It > works fine with ~50k accounts. > > Recently I enabled TLS: > $ cat /etc/dovecot/dovecot.conf > ... > # Use SSL ? > ssl = yes > ... > > The goal was to provide only IMAPS and POP3S. But Managesieve says > "STARTTLS": > $ telnet _MY_IP_ 2000 > Trying _MY_IP_... > Connected to _MY_IP_. > Escape character is '^]'. > "IMPLEMENTATION" "K8 ManageSieve" > "SIEVE" "comparator-i;ascii-numeric copy envelope fileinto imapflags > include notify regex reject relational subaddress vacation" > "SASL" "PLAIN LOGIN" > "STARTTLS" > "VERSION" "1.0" > OK "K8 IMAP/POP3 server" > > I think there is something wrong there but I don't know. I think > Managesieve should not say "STARTTLS". > Can someone help me? The STARTTLS capability means that ManageSieve is prepared to accept a STARTTLS command that invokes the TLS handshake. Basically, the protocol starts in plaintext and switches to a TLS/SSL secured channel once the STARTTLS command is issued. However, the client can also choose not to use it. Therefore, it really shouldn't influence whether ManageSieve works properly (unless the client messes up TLS somehow). If you really want to, you can disable TLS for ManageSieve specifically by putting a ssl=no inside the protocol sieve {} section. Regards, Stephan. From tss at iki.fi Tue Oct 2 22:38:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:38:58 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <5062DF3C.3050601@um.es> References: <5062DF3C.3050601@um.es> Message-ID: On 26.9.2012, at 13.55, Angel L. Mateo wrote: > doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do > doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; > done -S auth-userdb? You've named it completely wrong if that works :) > The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: > > e62e0d3834ed094e5c7900007efb8a67 66 > e62e0d3834ed094e5c7900007efb8a67 71 > e62e0d3834ed094e5c7900007efb8a67 74 > e62e0d3834ed094e5c7900007efb8a67 > 75 e62e0d3834ed094e5c7900007efb8a67 > 77 e62e0d3834ed094e5c7900007efb8a67 > 78 e62e0d3834ed094e5c7900007efb8a67 Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/94c7e875f9b9 From tss at iki.fi Tue Oct 2 22:50:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:50:08 +0300 Subject: [Dovecot] Problem with process_limit In-Reply-To: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: On 1.10.2012, at 12.15, FABIO FERRARI wrote: > Occasionally, it happens that the dovecot.log shows this line: > master: Warning: service(imap): process_limit reached, client connections > are being dropped .. > Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line > process_limit = 1500 But did you set it inside service imap {}? All of the services have process_limit parameter. From tss at iki.fi Tue Oct 2 23:12:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:12:51 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <5060AE92.5040904@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> Message-ID: On 24.9.2012, at 22.03, Joe Auty wrote: > #2 0x00007ff30074633d in master_service_settings_cache_deinit (_cache=) > at master-service-settings-cache.c:86 Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 From tss at iki.fi Tue Oct 2 23:15:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:15:22 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <83B37619-1CE8-4C5D-8147-A3C0E1C99CDC@iki.fi> On 2.10.2012, at 22.28, Timo Sirainen wrote: > On 19.9.2012, at 16.07, Alessio Cecchi wrote: > >> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > > This backtrace is rather weird. Could you also do (instead of bt full): Also, can you reproduce the crash always by running "doveadm quota recalc -u user at domain"? From tss at iki.fi Tue Oct 2 23:18:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:18:48 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <506AA8CF.8090605@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> Message-ID: <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> On 2.10.2012, at 11.41, Angel L. Mateo wrote: > I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). You could flush the whole cache also. > I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: Flush both the user and user at domain entries? From florob at babelmonkeys.de Wed Oct 3 00:05:56 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 02 Oct 2012 23:05:56 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme Message-ID: <506B5734.30906@babelmonkeys.de> Hello, attached is an hg export on top of the current dovecot-2.2 branch, which adds support for a SCRAM-SHA-1 password scheme. Ideally I'd want doveadm pw's rounds flag to apply to this, but that's currently specific to the crypt password scheme, so I left it out for now. Regards, Florian Zeitz -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1348017219 -7200 # Node ID 21a0d1b4daa7bb924f1666f0bb7c7e697a19c950 # Parent 8802322d72573ee17c52ce5e972e77e6f8ad69d1 auth: Add and use SCRAM-SHA-1 password scheme diff --git a/src/auth/Makefile.am b/src/auth/Makefile.am --- a/src/auth/Makefile.am +++ b/src/auth/Makefile.am @@ -44,6 +44,7 @@ password-scheme.c \ password-scheme-crypt.c \ password-scheme-md5crypt.c \ + password-scheme-scram.c \ password-scheme-otp.c \ password-scheme-rpa.c diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -1,11 +1,13 @@ /* * SCRAM-SHA-1 SASL authentication, see RFC-5802 * - * Copyright (c) 2011 Florian Zeitz + * Copyright (c) 2011-2012 Florian Zeitz * * This software is released under the MIT license. */ +#include + #include "auth-common.h" #include "base64.h" #include "buffer.h" @@ -29,45 +31,22 @@ /* sent: */ const char *server_first_message; - unsigned char salt[16]; - unsigned char salted_password[SHA1_RESULTLEN]; + const char *snonce; /* received: */ const char *gs2_cbind_flag; const char *cnonce; - const char *snonce; const char *client_first_message_bare; const char *client_final_message_without_proof; buffer_t *proof; + + /* stored */ + buffer_t *stored_key; + buffer_t *server_key; }; -static void Hi(const unsigned char *str, size_t str_size, - const unsigned char *salt, size_t salt_size, unsigned int i, - unsigned char result[SHA1_RESULTLEN]) -{ - struct hmac_context ctx; - unsigned char U[SHA1_RESULTLEN]; - unsigned int j, k; - - /* Calculate U1 */ - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, salt, salt_size); - hmac_update(&ctx, "\0\0\0\1", 4); - hmac_final(&ctx, U); - - memcpy(result, U, SHA1_RESULTLEN); - - /* Calculate U2 to Ui and Hi */ - for (j = 2; j <= i; j++) { - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, U, sizeof(U)); - hmac_final(&ctx, U); - for (k = 0; k < SHA1_RESULTLEN; k++) - result[k] ^= U[k]; - } -} - -static const char *get_scram_server_first(struct scram_auth_request *request) +static const char *get_scram_server_first(struct scram_auth_request *request, + int iter, const char *salt) { unsigned char snonce[SCRAM_SERVER_NONCE_LEN+1]; string_t *str; @@ -84,12 +63,9 @@ snonce[sizeof(snonce)-1] = '\0'; request->snonce = p_strndup(request->pool, snonce, sizeof(snonce)); - random_fill(request->salt, sizeof(request->salt)); - - str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(request->salt))); - str_printfa(str, "r=%s%s,s=", request->cnonce, request->snonce); - base64_encode(request->salt, sizeof(request->salt), str); - str_printfa(str, ",i=%d", SCRAM_ITERATE_COUNT); + str = t_str_new(sizeof(snonce)); + str_printfa(str, "r=%s%s,s=%s,i=%d", request->cnonce, request->snonce, + salt, iter); return str_c(str); } @@ -105,15 +81,8 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Server Key", 10); - hmac_final(&ctx, server_key); - - safe_memset(request->salted_password, 0, - sizeof(request->salted_password)); - - hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_init(&ctx, request->server_key->data, request->server_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, server_signature); @@ -211,8 +180,7 @@ return TRUE; } -static bool verify_credentials(struct scram_auth_request *request, - const unsigned char *credentials, size_t size) +static bool verify_credentials(struct scram_auth_request *request) { struct hmac_context ctx; const char *auth_message; @@ -221,54 +189,76 @@ unsigned char stored_key[SHA1_RESULTLEN]; size_t i; - /* FIXME: credentials should be SASLprepped UTF8 data here */ - Hi(credentials, size, request->salt, sizeof(request->salt), - SCRAM_ITERATE_COUNT, request->salted_password); - - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Client Key", 10); - hmac_final(&ctx, client_key); - - sha1_get_digest(client_key, sizeof(client_key), stored_key); - auth_message = t_strconcat(request->client_first_message_bare, ",", request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_init(&ctx, request->stored_key->data, request->stored_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) - client_signature[i] ^= client_key[i]; + client_key[i] = + ((char*)request->proof->data)[i] ^ client_signature[i]; + + sha1_get_digest(client_key, sizeof(client_key), stored_key); safe_memset(client_key, 0, sizeof(client_key)); - safe_memset(stored_key, 0, sizeof(stored_key)); + safe_memset(client_signature, 0, sizeof(client_signature)); - return memcmp(client_signature, request->proof->data, - request->proof->used) == 0; + return memcmp(stored_key, request->stored_key->data, + request->stored_key->used) == 0; } static void credentials_callback(enum passdb_result result, const unsigned char *credentials, size_t size, struct auth_request *auth_request) { + const char *const *fields; + size_t len; + int iter; + const char *salt; struct scram_auth_request *request = (struct scram_auth_request *)auth_request; - const char *server_final_message; switch (result) { case PASSDB_RESULT_OK: - if (!verify_credentials(request, credentials, size)) { + fields = t_strsplit(t_strndup(credentials, size), ","); + + iter = atoi(fields[0]); + salt = fields[1]; + + len = strlen(fields[2]); + request->stored_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[2], len, NULL, + request->stored_key) < 0) { auth_request_log_info(auth_request, "scram-sha-1", - "password mismatch"); + "Invalid base64 encoding" + "of StoredKey in passdb"); auth_request_fail(auth_request); - } else { - server_final_message = get_scram_server_final(request); - auth_request_success(auth_request, server_final_message, - strlen(server_final_message)); + break; } + + len = strlen(fields[3]); + request->server_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[3], len, NULL, + request->server_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of ServerKey in passdb"); + auth_request_fail(auth_request); + break; + } + + request->server_first_message = p_strdup(request->pool, + get_scram_server_first(request, iter, salt)); + + auth_request_handler_reply_continue(auth_request, + request->server_first_message, + strlen(request->server_first_message)); break; case PASSDB_RESULT_INTERNAL_FAILURE: auth_request_internal_failure(auth_request); @@ -333,8 +323,6 @@ request->client_final_message_without_proof = p_strdup(request->pool, t_strarray_join(fields, ",")); - auth_request_lookup_credentials(&request->auth_request, "PLAIN", - credentials_callback); return TRUE; } @@ -345,22 +333,35 @@ struct scram_auth_request *request = (struct scram_auth_request *)auth_request; const char *error = NULL; + const char *server_final_message; + int len; if (!request->client_first_message_bare) { /* Received client-first-message */ if (parse_scram_client_first(request, data, data_size, &error)) { - request->server_first_message = p_strdup(request->pool, - get_scram_server_first(request)); - auth_request_handler_reply_continue(auth_request, - request->server_first_message, - strlen(request->server_first_message)); + auth_request_lookup_credentials(&request->auth_request, + "SCRAM-SHA1", + credentials_callback); return; } } else { /* Received client-final-message */ - if (parse_scram_client_final(request, data, data_size, &error)) - return; + if (parse_scram_client_final(request, data, data_size, + &error)) { + if (!verify_credentials(request)) { + auth_request_log_info(auth_request, + "scram-sha-1", + "password mismatch"); + } else { + server_final_message = + get_scram_server_final(request); + len = strlen(server_final_message); + auth_request_success(auth_request, + server_final_message, len); + return; + } + } } if (error != NULL) diff --git a/src/auth/password-scheme-scram.c b/src/auth/password-scheme-scram.c new file mode 100644 --- /dev/null +++ b/src/auth/password-scheme-scram.c @@ -0,0 +1,139 @@ +/* + * SCRAM-SHA-1 SASL authentication, see RFC-5802 + * + * Copyright (c) 2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include + +#include "lib.h" +#include "safe-memset.h" +#include "base64.h" +#include "buffer.h" +#include "hmac.h" +#include "randgen.h" +#include "sha1.h" +#include "str.h" +#include "password-scheme.h" + +/* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ +#define SCRAM_ITERATE_COUNT 4096 + +static void Hi(const unsigned char *str, size_t str_size, + const unsigned char *salt, size_t salt_size, unsigned int i, + unsigned char result[SHA1_RESULTLEN]) +{ + struct hmac_context ctx; + unsigned char U[SHA1_RESULTLEN]; + unsigned int j, k; + + /* Calculate U1 */ + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); + + memcpy(result, U, SHA1_RESULTLEN); + + /* Calculate U2 to Ui and Hi */ + for (j = 2; j <= i; j++) { + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); + for (k = 0; k < SHA1_RESULTLEN; k++) + result[k] ^= U[k]; + } +} + +/* password string format: iter,salt,stored_key,server_key */ + +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED) +{ + struct hmac_context ctx; + string_t *str; + const char *const *fields; + int iter; + const unsigned char *salt; + size_t salt_len; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + fields = t_strsplit(t_strndup(raw_password, size), ","); + iter = atoi(fields[0]); + salt = buffer_get_data(t_base64_decode_str(fields[1]), &salt_len); + str = t_str_new(strlen(fields[2])); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, salt_len, + iter, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + base64_encode(stored_key, sizeof(stored_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + return strcmp(fields[2], str_c(str)) == 0 ? 1 : 0; +} + +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r) +{ + string_t *str; + struct hmac_context ctx; + unsigned char salt[16]; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char server_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + random_fill(salt, sizeof(salt)); + + str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(salt))); + str_printfa(str, "%i,", SCRAM_ITERATE_COUNT); + base64_encode(salt, sizeof(salt), str); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, + sizeof(salt), SCRAM_ITERATE_COUNT, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + str_append_c(str, ','); + base64_encode(stored_key, sizeof(stored_key), str); + + /* Calculate ServerKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); + str_append_c(str, ','); + base64_encode(server_key, sizeof(server_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(server_key, 0, sizeof(server_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + *raw_password_r = (const unsigned char *)str_c(str); + *size_r = str_len(str); +} diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -822,6 +822,8 @@ { "PLAIN-TRUNC", PW_ENCODING_NONE, 0, plain_trunc_verify, plain_generate }, { "CRAM-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, + { "SCRAM-SHA1", PW_ENCODING_NONE, 0, scram_sha1_verify, + scram_sha1_generate}, { "HMAC-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, { "DIGEST-MD5", PW_ENCODING_HEX, MD5_RESULTLEN, diff --git a/src/auth/password-scheme.h b/src/auth/password-scheme.h --- a/src/auth/password-scheme.h +++ b/src/auth/password-scheme.h @@ -85,6 +85,12 @@ const unsigned char *raw_password, size_t size, const char **error_r); +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED); +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r); + /* check wich of the algorithms Blowfisch, SHA-256 and SHA-512 are supported by the used libc's/glibc's crypt() */ void password_scheme_register_crypt(void); From joe at netmusician.org Wed Oct 3 00:09:19 2012 From: joe at netmusician.org (Joe Auty) Date: Tue, 02 Oct 2012 17:09:19 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> Message-ID: <506B57FF.40809@netmusician.org> > > Timo Sirainen > October 2, 2012 4:12 PM > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 Awesome! Will this fix make it into 2.1.11, or should I toy with incorporating your change and compiling a new copy of 2.1.10 by hand? Also, will the seg fault have caused performance issues in breaking IDLE connections? Just wondering what sort of impact this fix might have... Thanks for working on this, it is most appreciated, I'm a big fan of Dovecot! > > > Joe Auty > September 24, 2012 3:03 PM >> >> Timo Sirainen >> September 24, 2012 10:32 AM >> >> Well, the good news is that it crashes only after it has already >> disconnected the client anyway. But I thought I fixed this bug in >> v2.1.10 and I'm not able to reproduce it myself.. Having debugging >> information available might show something useful. Try installing >> dovecot-dbg package and getting the bt full again? > > Thanks Timo, I have done so. Here is the results of my debugging info > now: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...Reading symbols > from /usr/lib/debug/usr/lib/dovecot/imap-login...done. > (no debugging symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...Reading > symbols from > /usr/lib/debug/usr/lib/dovecot/libdovecot-login.so.0.0.0...done. > (no debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...Reading > symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot.so.0.0.0...done. > (no debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 hash_table_destroy (_table=0x28) at hash.c:106 > 106 hash.c: No such file or directory. > in hash.c > (gdb) bt full > #0 hash_table_destroy (_table=0x28) at hash.c:106 > table = > #1 0x00007ff300721054 in settings_parser_deinit (_ctx= optimized out>) at settings-parser.c:237 > ctx = 0x0 > #2 0x00007ff30074633d in master_service_settings_cache_deinit > (_cache=) > at master-service-settings-cache.c:86 > cache = 0x9f9a60 > entry = 0xa016e0 > next = 0x0 > __FUNCTION__ = "master_service_settings_cache_deinit" > #3 0x00007ff3009a5018 in main_deinit (binary=, > argc=2, argv=0x9f8370) at main.c:355 > No locals. > #4 login_binary_run (binary=, argc=2, > argv=0x9f8370) at main.c:407 > set_pool = 0x9f8a30 > allow_core_dumps = > login_socket = > c = > #5 0x00007ff3003c0c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #6 0x0000000000402459 in _start () > No symbol table info available. > (gdb) > > >> >> >> Joe Auty >> September 23, 2012 7:05 AM >>> >>> Timo Sirainen >>> September 23, 2012 5:58 AM >>> >>> >>> You should have a similar log line about the crash in mail.log (or >>> wherever "doveadm log find" says that errors get logged). Find those >>> lines, then configure login processes to dump core files. This >>> probably should work: >>> >>> service imap-login { >>> executable = imap-login -D >>> } >>> >>> Next time it crashes hopefully you'll have >>> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >>> send it: >>> >>> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >>> bt full >> >> I hope I'm doing this correctly! >> >> # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> GNU gdb (GDB) 7.0.1-debian >> Copyright (C) 2009 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later >> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show >> copying" >> and "show warranty" for details. >> This GDB was configured as "x86_64-linux-gnu". >> For bug reporting instructions, please see: >> ... >> Reading symbols from /usr/lib/dovecot/imap-login...(no debugging >> symbols found)...done. >> >> warning: Can't read pathname for load map: Input/output error. >> Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no >> debugging symbols found)...done. >> Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 >> Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging >> symbols found)...done. >> Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 >> Reading symbols from /lib/libc.so.6...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libc.so.6 >> Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols >> found)...done. >> Loaded symbols for /usr/lib/libssl.so.0.9.8 >> Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging >> symbols found)...done. >> Loaded symbols for /usr/lib/libcrypto.so.0.9.8 >> Reading symbols from /lib/librt.so.1...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/librt.so.1 >> Reading symbols from /lib/libdl.so.2...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libdl.so.2 >> Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging >> symbols found)...done. >> Loaded symbols for /lib64/ld-linux-x86-64.so.2 >> Reading symbols from /usr/lib/libz.so.1...(no debugging symbols >> found)...done. >> Loaded symbols for /usr/lib/libz.so.1 >> Reading symbols from /lib/libpthread.so.0...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libpthread.so.0 >> Core was generated by `dovecot/imap-login ?'. >> Program terminated with signal 11, Segmentation fault. >> #0 0x00007f789cd08e14 in hash_table_destroy () from >> /usr/lib/dovecot/libdovecot.so.0 >> (gdb) bt full >> #0 0x00007f789cd08e14 in hash_table_destroy () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00007f789ccda054 in settings_parser_deinit () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () >> from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #3 0x00007f789cf5e018 in login_binary_run () from >> /usr/lib/dovecot/libdovecot-login.so.0 >> No symbol table info available. >> #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 >> No symbol table info available. >> #5 0x0000000000402459 in ?? () >> No symbol table info available. >> #6 0x00007fff8a9c65f8 in ?? () >> No symbol table info available. >> #7 0x000000000000001c in ?? () >> No symbol table info available. >> #8 0x0000000000000002 in ?? () >> No symbol table info available. >> #9 0x00007fff8a9c7e6a in ?? () >> No symbol table info available. >> #10 0x00007fff8a9c7e7d in ?? () >> No symbol table info available. >> #11 0x0000000000000000 in ?? () >> No symbol table info available. >> >> >>> >>> >>> Joe Auty >>> September 22, 2012 3:57 PM >>> Hello, >>> >>> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >>> I suspect this might be causing performance issues. Any suggestions >>> what I can try to fix this? >>> >>> >>> I'm using the 2.1.10 packages obtained with the following in my >>> sources.list: >>> >>> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >>> >>> >>> I need to use Dovecot 2.1.x because I need to support handling >>> multiple SSL certs. >>> ------------------------------------------------------------------------ >>> >> >> >> >> >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full > > ------------------------------------------------------------------------ From tss at iki.fi Wed Oct 3 00:13:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:13:10 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <506B57FF.40809@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> <506B57FF.40809@netmusician.org> Message-ID: <9AC6F07E-8751-47C0-AF6C-82722D1BAE5D@iki.fi> On 3.10.2012, at 0.09, Joe Auty wrote: >> Timo Sirainen >> October 2, 2012 4:12 PM >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 > > Awesome! > > Will this fix make it into 2.1.11, or should I toy with incorporating your change and compiling a new copy of 2.1.10 by hand? All changes added to dovecot-2.1 hg go to the next 2.1 release. But I don't know when v2.1.11 will be released, probably a few weeks at least. > Also, will the seg fault have caused performance issues in breaking IDLE connections? Just wondering what sort of impact this fix might have... It shouldn't have caused any user-visible problems. From tss at iki.fi Wed Oct 3 00:27:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:27:13 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B5734.30906@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> Message-ID: On 3.10.2012, at 0.05, Florian Zeitz wrote: > attached is an hg export on top of the current dovecot-2.2 branch, which > adds support for a SCRAM-SHA-1 password scheme. > > Ideally I'd want doveadm pw's rounds flag to apply to this, but that's > currently specific to the crypt password scheme, so I left it out for now. Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. From cor at xs4all.nl Wed Oct 3 00:39:39 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 2 Oct 2012 23:39:39 +0200 Subject: [Dovecot] possible nfs issue Message-ID: Hi all, we've started receiving complaints from users that seemingly use more quota than they actually have. We noticed that these users have (in some cases many) .nfs files in their mailspool. Some of our admins checked their own dirs, and noticed them there as well. This could of course be unrelated to dovecot (kernel issue, netapp issue) but maybe somehow has an idea about if dovecot could cause this. This has been going on for at least a year, not really enough to notice before now. root at userimap1# find . -type f|grep -i .nfs ./cur/.nfs00000000003967ad003c0603 ./cur/.nfs000000000757b44b003be609 ./cur/.nfs00000000035e89bd003be60b ./cur/.nfs000000000796251c003be60c ./cur/.nfs000000000796251f003be60e ./cur/.nfs000000000262f9a1003be33a ./cur/.nfs00000000096513f3003be524 ./cur/.nfs0000000007962525003be60f ./cur/.nfs0000000003e7d8ab003be62b ./cur/.nfs00000000026f4fad003be50d ./cur/.nfs0000000000bdaeab003c0611 ./cur/.nfs0000000005da42c7003be525 ./cur/.nfs0000000003d74729003be526 ./cur/.nfs000000000229769e003be535 ./cur/.nfs000000000440969e003be516 With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. Anyone else with NFS mailspools seeing this? Cor From tss at iki.fi Wed Oct 3 00:45:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:45:35 +0300 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: On 3.10.2012, at 0.39, Cor Bosman wrote: > With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) From florob at babelmonkeys.de Wed Oct 3 01:12:29 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 00:12:29 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: References: <506B5734.30906@babelmonkeys.de> Message-ID: <506B66CD.6010302@babelmonkeys.de> Am 02.10.2012 23:27, schrieb Timo Sirainen: > On 3.10.2012, at 0.05, Florian Zeitz wrote: > >> attached is an hg export on top of the current dovecot-2.2 branch, which >> adds support for a SCRAM-SHA-1 password scheme. >> >> Ideally I'd want doveadm pw's rounds flag to apply to this, but that's >> currently specific to the crypt password scheme, so I left it out for now. > > Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. > I shall. For the iteration count the endless loop should be fixed by restricting the largest value to UINT_MAX-1, right? I'm not too fond of stopping people from wasting their CPU time on Hi calculation beyond this. I can try to guestimate a "sane" upper limit, but given time I have an icky feeling that it will end up being too low. Thoughts? From tss at iki.fi Wed Oct 3 01:31:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 01:31:51 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B66CD.6010302@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <506B66CD.6010302@babelmonkeys.de> Message-ID: On 3.10.2012, at 1.12, Florian Zeitz wrote: > Am 02.10.2012 23:27, schrieb Timo Sirainen: >> On 3.10.2012, at 0.05, Florian Zeitz wrote: >> >>> attached is an hg export on top of the current dovecot-2.2 branch, which >>> adds support for a SCRAM-SHA-1 password scheme. >>> >>> Ideally I'd want doveadm pw's rounds flag to apply to this, but that's >>> currently specific to the crypt password scheme, so I left it out for now. >> >> Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. >> > I shall. For the iteration count the endless loop should be fixed by > restricting the largest value to UINT_MAX-1, right? Yeah. > I'm not too fond of > stopping people from wasting their CPU time on Hi calculation beyond > this. I can try to guestimate a "sane" upper limit, but given time I > have an icky feeling that it will end up being too low. Thoughts? Looks like RFC 5802 doesn't give any kind of a limit. But since it gets sent to various client implementations, INT_MAX is probably a good limit? Also 0 isn't a valid iteration count. From tss at iki.fi Wed Oct 3 01:35:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 01:35:20 +0300 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> On 3.10.2012, at 0.45, Timo Sirainen wrote: > On 3.10.2012, at 0.39, Cor Bosman wrote: > >> With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. > > How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) You could see if this old test program leaves .nfs files lying around: http://dovecot.org/tmp/readdir.c Just comment out the line: close(fd); From jtam.home at gmail.com Wed Oct 3 01:42:23 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 2 Oct 2012 15:42:23 -0700 (PDT) Subject: [Dovecot] Logging IP address for failed login In-Reply-To: References: Message-ID: Scott Neville writes: > I am trying to use the logs to show the IP that brute force activity > comes from, but Im not succeeding. I have read the archives and seen > the advice others have had. I can see logs for repeated bad logins, > but I need the IP address from the attempts. > > ... > but only for successful logins. The brute force attempts dont log like that: > > Sep 16 00:02:58 olive dovecot: auth: pam(backup): unknown user This was similar to another complaint several months ago. I conjectured that these attempts are SMTP AUTH, not IMAP, brute forcing. Are you using the dovecot's SASL feature to authenticate outgoing Email (i.e. via Postfix?). Maybe you verify this hypothesis by checking the Postfix logs. Joseph Tam From cor at xs4all.nl Wed Oct 3 01:53:15 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 3 Oct 2012 00:53:15 +0200 Subject: [Dovecot] possible nfs issue In-Reply-To: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> References: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> Message-ID: On Oct 3, 2012, at 12:35 AM, Timo Sirainen wrote: > On 3.10.2012, at 0.45, Timo Sirainen wrote: > >> On 3.10.2012, at 0.39, Cor Bosman wrote: >> >>> With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. >> >> How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) > > You could see if this old test program leaves .nfs files lying around: > > http://dovecot.org/tmp/readdir.c > > Just comment out the line: > > close(fd); > I meant the .nfs1234 indeed, but it seemed very unlikely. Thanks for clarifying. The readdir program leaves no .nfs files. We'll have to explore other possibilities. Cor From tss at iki.fi Wed Oct 3 02:42:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 02:42:17 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B5734.30906@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> Message-ID: <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> On 3.10.2012, at 0.05, Florian Zeitz wrote: > attached is an hg export on top of the current dovecot-2.2 branch, which > adds support for a SCRAM-SHA-1 password scheme. Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. From florob at babelmonkeys.de Wed Oct 3 02:54:21 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 01:54:21 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> Message-ID: <506B7EAD.1080108@babelmonkeys.de> Am 03.10.2012 01:42, schrieb Timo Sirainen: > On 3.10.2012, at 0.05, Florian Zeitz wrote: > >> attached is an hg export on top of the current dovecot-2.2 branch, which >> adds support for a SCRAM-SHA-1 password scheme. > > Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. > Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the RFC, and SHA-1 is the hash name registered with IANA [1]. I did call the password scheme SCRAM-SHA1 to be consistent with other current password schemes. I'm not 100% sure which one to use, or whether a mix might even be the way to go ("correct" messages, but minimum user confusion for password schemes). [1] https://www.iana.org/assignments/hash-function-text-names/hash-function-text-names.xml From tss at iki.fi Wed Oct 3 02:58:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 02:58:25 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B7EAD.1080108@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> Message-ID: On 3.10.2012, at 2.54, Florian Zeitz wrote: > Am 03.10.2012 01:42, schrieb Timo Sirainen: >> On 3.10.2012, at 0.05, Florian Zeitz wrote: >> >>> attached is an hg export on top of the current dovecot-2.2 branch, which >>> adds support for a SCRAM-SHA-1 password scheme. >> >> Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. >> > Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the > RFC, and SHA-1 is the hash name registered with IANA [1]. > I did call the password scheme SCRAM-SHA1 to be consistent with other > current password schemes. I'm not 100% sure which one to use, or whether > a mix might even be the way to go ("correct" messages, but minimum user > confusion for password schemes). Hmm. Probably not worth it to have both SCRAM-SHA1 and SCRAM-SHA-1. And now I see that the user-visible strings are about SCRAM-SHA-1 mechanism, not the hash. So yeah, I guess the best way to avoid confusion is to call it SCRAM-SHA-1 everywhere. From florob at babelmonkeys.de Wed Oct 3 03:10:41 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 02:10:41 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> Message-ID: <506B8281.9060703@babelmonkeys.de> Am 03.10.2012 01:58, schrieb Timo Sirainen: > On 3.10.2012, at 2.54, Florian Zeitz wrote: > >> Am 03.10.2012 01:42, schrieb Timo Sirainen: >>> On 3.10.2012, at 0.05, Florian Zeitz wrote: >>> >>>> attached is an hg export on top of the current dovecot-2.2 branch, which >>>> adds support for a SCRAM-SHA-1 password scheme. >>> >>> Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. >>> >> Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the >> RFC, and SHA-1 is the hash name registered with IANA [1]. >> I did call the password scheme SCRAM-SHA1 to be consistent with other >> current password schemes. I'm not 100% sure which one to use, or whether >> a mix might even be the way to go ("correct" messages, but minimum user >> confusion for password schemes). > > Hmm. Probably not worth it to have both SCRAM-SHA1 and SCRAM-SHA-1. And now I see that the user-visible strings are about SCRAM-SHA-1 mechanism, not the hash. So yeah, I guess the best way to avoid confusion is to call it SCRAM-SHA-1 everywhere. > Seems sensible. Attached is a new export incorporating your feedback. The iteration count is now limited to [4096, INT_MAX]. The lower bound is a recommendation of the RFC. -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1348017219 -7200 # Node ID a0b0eece12335905500631477ec1d6ab31014469 # Parent 99843f74422ac68bfde86e9cee6920164eae4d5d auth: Add and use SCRAM-SHA-1 password scheme diff --git a/src/auth/Makefile.am b/src/auth/Makefile.am --- a/src/auth/Makefile.am +++ b/src/auth/Makefile.am @@ -44,6 +44,7 @@ password-scheme.c \ password-scheme-crypt.c \ password-scheme-md5crypt.c \ + password-scheme-scram.c \ password-scheme-otp.c \ password-scheme-rpa.c diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -1,11 +1,14 @@ /* * SCRAM-SHA-1 SASL authentication, see RFC-5802 * - * Copyright (c) 2011 Florian Zeitz + * Copyright (c) 2011-2012 Florian Zeitz * * This software is released under the MIT license. */ +#include +#include + #include "auth-common.h" #include "base64.h" #include "buffer.h" @@ -15,6 +18,7 @@ #include "safe-memset.h" #include "str.h" #include "strfuncs.h" +#include "strnum.h" #include "mech.h" /* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ @@ -29,45 +33,22 @@ /* sent: */ const char *server_first_message; - unsigned char salt[16]; - unsigned char salted_password[SHA1_RESULTLEN]; + const char *snonce; /* received: */ const char *gs2_cbind_flag; const char *cnonce; - const char *snonce; const char *client_first_message_bare; const char *client_final_message_without_proof; buffer_t *proof; + + /* stored */ + buffer_t *stored_key; + buffer_t *server_key; }; -static void Hi(const unsigned char *str, size_t str_size, - const unsigned char *salt, size_t salt_size, unsigned int i, - unsigned char result[SHA1_RESULTLEN]) -{ - struct hmac_context ctx; - unsigned char U[SHA1_RESULTLEN]; - unsigned int j, k; - - /* Calculate U1 */ - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, salt, salt_size); - hmac_update(&ctx, "\0\0\0\1", 4); - hmac_final(&ctx, U); - - memcpy(result, U, SHA1_RESULTLEN); - - /* Calculate U2 to Ui and Hi */ - for (j = 2; j <= i; j++) { - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, U, sizeof(U)); - hmac_final(&ctx, U); - for (k = 0; k < SHA1_RESULTLEN; k++) - result[k] ^= U[k]; - } -} - -static const char *get_scram_server_first(struct scram_auth_request *request) +static const char *get_scram_server_first(struct scram_auth_request *request, + int iter, const char *salt) { unsigned char snonce[SCRAM_SERVER_NONCE_LEN+1]; string_t *str; @@ -84,12 +65,9 @@ snonce[sizeof(snonce)-1] = '\0'; request->snonce = p_strndup(request->pool, snonce, sizeof(snonce)); - random_fill(request->salt, sizeof(request->salt)); - - str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(request->salt))); - str_printfa(str, "r=%s%s,s=", request->cnonce, request->snonce); - base64_encode(request->salt, sizeof(request->salt), str); - str_printfa(str, ",i=%d", SCRAM_ITERATE_COUNT); + str = t_str_new(sizeof(snonce)); + str_printfa(str, "r=%s%s,s=%s,i=%d", request->cnonce, request->snonce, + salt, iter); return str_c(str); } @@ -105,15 +83,8 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Server Key", 10); - hmac_final(&ctx, server_key); - - safe_memset(request->salted_password, 0, - sizeof(request->salted_password)); - - hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_init(&ctx, request->server_key->data, request->server_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, server_signature); @@ -211,8 +182,7 @@ return TRUE; } -static bool verify_credentials(struct scram_auth_request *request, - const unsigned char *credentials, size_t size) +static bool verify_credentials(struct scram_auth_request *request) { struct hmac_context ctx; const char *auth_message; @@ -221,54 +191,90 @@ unsigned char stored_key[SHA1_RESULTLEN]; size_t i; - /* FIXME: credentials should be SASLprepped UTF8 data here */ - Hi(credentials, size, request->salt, sizeof(request->salt), - SCRAM_ITERATE_COUNT, request->salted_password); - - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Client Key", 10); - hmac_final(&ctx, client_key); - - sha1_get_digest(client_key, sizeof(client_key), stored_key); - auth_message = t_strconcat(request->client_first_message_bare, ",", request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_init(&ctx, request->stored_key->data, request->stored_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) - client_signature[i] ^= client_key[i]; + client_key[i] = + ((char*)request->proof->data)[i] ^ client_signature[i]; + + sha1_get_digest(client_key, sizeof(client_key), stored_key); safe_memset(client_key, 0, sizeof(client_key)); - safe_memset(stored_key, 0, sizeof(stored_key)); + safe_memset(client_signature, 0, sizeof(client_signature)); - return memcmp(client_signature, request->proof->data, - request->proof->used) == 0; + return memcmp(stored_key, request->stored_key->data, + request->stored_key->used) == 0; } static void credentials_callback(enum passdb_result result, const unsigned char *credentials, size_t size, struct auth_request *auth_request) { + const char *const *fields; + size_t len; + unsigned int iter; + const char *salt; struct scram_auth_request *request = (struct scram_auth_request *)auth_request; - const char *server_final_message; switch (result) { case PASSDB_RESULT_OK: - if (!verify_credentials(request, credentials, size)) { + fields = t_strsplit(t_strndup(credentials, size), ","); + + if (str_array_length(fields) != 4) { auth_request_log_info(auth_request, "scram-sha-1", - "password mismatch"); + "Invalid passdb entry"); auth_request_fail(auth_request); - } else { - server_final_message = get_scram_server_final(request); - auth_request_success(auth_request, server_final_message, - strlen(server_final_message)); + break; } + + if (str_to_uint(fields[0], &iter) || (iter < 4096) || + (iter > INT_MAX)) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid iteration count"); + auth_request_fail(auth_request); + break; + } + + salt = fields[1]; + + len = strlen(fields[2]); + request->stored_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[2], len, NULL, + request->stored_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of StoredKey in passdb"); + auth_request_fail(auth_request); + break; + } + + len = strlen(fields[3]); + request->server_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[3], len, NULL, + request->server_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of ServerKey in passdb"); + auth_request_fail(auth_request); + break; + } + + request->server_first_message = p_strdup(request->pool, + get_scram_server_first(request, iter, salt)); + + auth_request_handler_reply_continue(auth_request, + request->server_first_message, + strlen(request->server_first_message)); break; case PASSDB_RESULT_INTERNAL_FAILURE: auth_request_internal_failure(auth_request); @@ -333,8 +339,6 @@ request->client_final_message_without_proof = p_strdup(request->pool, t_strarray_join(fields, ",")); - auth_request_lookup_credentials(&request->auth_request, "PLAIN", - credentials_callback); return TRUE; } @@ -345,22 +349,35 @@ struct scram_auth_request *request = (struct scram_auth_request *)auth_request; const char *error = NULL; + const char *server_final_message; + int len; if (!request->client_first_message_bare) { /* Received client-first-message */ if (parse_scram_client_first(request, data, data_size, &error)) { - request->server_first_message = p_strdup(request->pool, - get_scram_server_first(request)); - auth_request_handler_reply_continue(auth_request, - request->server_first_message, - strlen(request->server_first_message)); + auth_request_lookup_credentials(&request->auth_request, + "SCRAM-SHA-1", + credentials_callback); return; } } else { /* Received client-final-message */ - if (parse_scram_client_final(request, data, data_size, &error)) - return; + if (parse_scram_client_final(request, data, data_size, + &error)) { + if (!verify_credentials(request)) { + auth_request_log_info(auth_request, + "scram-sha-1", + "password mismatch"); + } else { + server_final_message = + get_scram_server_final(request); + len = strlen(server_final_message); + auth_request_success(auth_request, + server_final_message, len); + return; + } + } } if (error != NULL) diff --git a/src/auth/password-scheme-scram.c b/src/auth/password-scheme-scram.c new file mode 100644 --- /dev/null +++ b/src/auth/password-scheme-scram.c @@ -0,0 +1,139 @@ +/* + * SCRAM-SHA-1 SASL authentication, see RFC-5802 + * + * Copyright (c) 2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include + +#include "lib.h" +#include "safe-memset.h" +#include "base64.h" +#include "buffer.h" +#include "hmac.h" +#include "randgen.h" +#include "sha1.h" +#include "str.h" +#include "password-scheme.h" + +/* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ +#define SCRAM_ITERATE_COUNT 4096 + +static void Hi(const unsigned char *str, size_t str_size, + const unsigned char *salt, size_t salt_size, unsigned int i, + unsigned char result[SHA1_RESULTLEN]) +{ + struct hmac_context ctx; + unsigned char U[SHA1_RESULTLEN]; + unsigned int j, k; + + /* Calculate U1 */ + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); + + memcpy(result, U, SHA1_RESULTLEN); + + /* Calculate U2 to Ui and Hi */ + for (j = 2; j <= i; j++) { + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); + for (k = 0; k < SHA1_RESULTLEN; k++) + result[k] ^= U[k]; + } +} + +/* password string format: iter,salt,stored_key,server_key */ + +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED) +{ + struct hmac_context ctx; + string_t *str; + const char *const *fields; + int iter; + const unsigned char *salt; + size_t salt_len; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + fields = t_strsplit(t_strndup(raw_password, size), ","); + iter = atoi(fields[0]); + salt = buffer_get_data(t_base64_decode_str(fields[1]), &salt_len); + str = t_str_new(strlen(fields[2])); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, salt_len, + iter, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + base64_encode(stored_key, sizeof(stored_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + return strcmp(fields[2], str_c(str)) == 0 ? 1 : 0; +} + +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r) +{ + string_t *str; + struct hmac_context ctx; + unsigned char salt[16]; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char server_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + random_fill(salt, sizeof(salt)); + + str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(salt))); + str_printfa(str, "%i,", SCRAM_ITERATE_COUNT); + base64_encode(salt, sizeof(salt), str); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, + sizeof(salt), SCRAM_ITERATE_COUNT, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + str_append_c(str, ','); + base64_encode(stored_key, sizeof(stored_key), str); + + /* Calculate ServerKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); + str_append_c(str, ','); + base64_encode(server_key, sizeof(server_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(server_key, 0, sizeof(server_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + *raw_password_r = (const unsigned char *)str_c(str); + *size_r = str_len(str); +} diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -822,6 +822,8 @@ { "PLAIN-TRUNC", PW_ENCODING_NONE, 0, plain_trunc_verify, plain_generate }, { "CRAM-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, + { "SCRAM-SHA-1", PW_ENCODING_NONE, 0, scram_sha1_verify, + scram_sha1_generate}, { "HMAC-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, { "DIGEST-MD5", PW_ENCODING_HEX, MD5_RESULTLEN, diff --git a/src/auth/password-scheme.h b/src/auth/password-scheme.h --- a/src/auth/password-scheme.h +++ b/src/auth/password-scheme.h @@ -85,6 +85,12 @@ const unsigned char *raw_password, size_t size, const char **error_r); +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED); +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r); + /* check wich of the algorithms Blowfisch, SHA-256 and SHA-512 are supported by the used libc's/glibc's crypt() */ void password_scheme_register_crypt(void); From tss at iki.fi Wed Oct 3 03:50:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 03 Oct 2012 03:50:00 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B8281.9060703@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> <506B8281.9060703@babelmonkeys.de> Message-ID: <1349225400.18782.54.camel@innu> On Wed, 2012-10-03 at 02:10 +0200, Florian Zeitz wrote: > Attached is a new export incorporating your feedback. Committed. Also what do you think about the attached patch? (Compiles, untested.) -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 9633 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 3 05:23:57 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 02 Oct 2012 21:23:57 -0500 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <506BA1BD.1070903@brightok.net> On 10/2/2012 4:39 PM, Cor Bosman wrote: > > Anyone else with NFS mailspools seeing this? > > Cor > > I haven't seen them yet, however, to help troubleshoot, see this link and follow it's links for more details on .nfs files http://wordpress.org/support/topic/how-can-i-prevent-unwanted-nfs-files-from-being-created Jack From amateo at um.es Wed Oct 3 09:20:00 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 03 Oct 2012 08:20:00 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: References: <5062DF3C.3050601@um.es> Message-ID: <506BD910.90200@um.es> El 02/10/12 21:38, Timo Sirainen escribi?: > On 26.9.2012, at 13.55, Angel L. Mateo wrote: > >> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >> done > > -S auth-userdb? You've named it completely wrong if that works :) > auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? >> The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: >> >> e62e0d3834ed094e5c7900007efb8a67 66 >> e62e0d3834ed094e5c7900007efb8a67 71 >> e62e0d3834ed094e5c7900007efb8a67 74 >> e62e0d3834ed094e5c7900007efb8a67 >> 75 e62e0d3834ed094e5c7900007efb8a67 >> 77 e62e0d3834ed094e5c7900007efb8a67 >> 78 e62e0d3834ed094e5c7900007efb8a67 > > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/94c7e875f9b9 > Thanks, I'll check as soon as I can. From amateo at um.es Wed Oct 3 09:25:38 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 03 Oct 2012 08:25:38 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> Message-ID: <506BDA62.2010604@um.es> El 02/10/12 22:18, Timo Sirainen escribi?: > On 2.10.2012, at 11.41, Angel L. Mateo wrote: > >> I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). > > You could flush the whole cache also. > Oh... I was so obfuscated trying to expire just the user that I forgot I could flush the whole cache :-( >> I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: > > Flush both the user and user at domain entries? > Yes, I could do this, but why there are entries with user and user at domain?, because I have three user databases: * master password: it is not normally used * pam: I have the cache_key=%n on it * ldap: I don't know to configure cache_key (I tried args = cache_key=%n /etc/dovecot/dovecot-ldap.conf.ext but it didn't work) From tss at iki.fi Wed Oct 3 16:26:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 16:26:34 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <506BDA62.2010604@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> <506BDA62.2010604@um.es> Message-ID: <5B43FF27-C875-48D7-91DA-FA86848B02B9@iki.fi> On 3.10.2012, at 9.25, Angel L. Mateo wrote: >>> I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: >> >> Flush both the user and user at domain entries? >> > Yes, I could do this, but why there are entries with user and user at domain?, because I have three user databases: > > * master password: it is not normally used > * pam: I have the cache_key=%n on it > * ldap: I don't know to configure cache_key (I tried args = cache_key=%n /etc/dovecot/dovecot-ldap.conf.ext but it didn't work) For LDAP the cache_key is figured out automatically based on the used %variables. You can't override the cache key. The only way to make it work would be to change the LDAP query to use only %n and no %u/%d (which I guess would be possible by checking for %n@* ?) From tss at iki.fi Wed Oct 3 16:27:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 16:27:32 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <506BD910.90200@um.es> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> Message-ID: <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> On 3.10.2012, at 9.20, Angel L. Mateo wrote: > El 02/10/12 21:38, Timo Sirainen escribi?: >> On 26.9.2012, at 13.55, Angel L. Mateo wrote: >> >>> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >>> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >>> done >> >> -S auth-userdb? You've named it completely wrong if that works :) >> > auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? From florob at babelmonkeys.de Wed Oct 3 16:33:32 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 15:33:32 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <1349225400.18782.54.camel@innu> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> <506B8281.9060703@babelmonkeys.de> <1349225400.18782.54.camel@innu> Message-ID: <506C3EAC.6000504@babelmonkeys.de> Am 03.10.2012 02:50, schrieb Timo Sirainen: > On Wed, 2012-10-03 at 02:10 +0200, Florian Zeitz wrote: >> Attached is a new export incorporating your feedback. > > Committed. Also what do you think about the attached patch? (Compiles, > untested.) > Moving the passdb parsing into a separate function seems like a nice idea to me. Style changes and removing an unused variable is obviously fine (I'm a bit surprised I got no compiler warning about the latter, but oh well). I did a quick test. Login and error checking seem to still work fine with this patch in place. Wouldn't have seen anything in the code to suggest otherwise either. From list at airstreamcomm.net Wed Oct 3 17:34:51 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 03 Oct 2012 09:34:51 -0500 Subject: [Dovecot] LDA vs LMTP index files Message-ID: <506C4D0B.8030709@airstreamcomm.net> In the docs it states that LDA " ...takes mail from anMTAand delivers it to a user's mailbox, while keeping Dovecot index files up to date." I am wondering if LMTP also interacts with the Dovecot index files and keeps them up to date? From jbates at brightok.net Wed Oct 3 18:06:08 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 03 Oct 2012 10:06:08 -0500 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C4D0B.8030709@airstreamcomm.net> References: <506C4D0B.8030709@airstreamcomm.net> Message-ID: <506C5460.205@brightok.net> On 10/3/2012 9:34 AM, list at airstreamcomm.net wrote: > In the docs it states that LDA " ...takes mail from anMTAand delivers > it to a user's mailbox, while keeping Dovecot index files up to > date." I am wondering if LMTP also interacts with the Dovecot index > files and keeps them up to date? > Brand new account created with LMTP. So I'd say yes. -rw-------. 1 vmail vmail 16384 Oct 2 20:21 dovecot.index.cache -rw-------. 1 vmail vmail 560 Oct 2 20:21 dovecot.index.log -rw-------. 1 vmail vmail 152 Oct 2 20:21 dovecot-uidlist -rw-------. 1 vmail vmail 8 Oct 2 18:58 dovecot-uidvalidity -r--r--r--. 1 vmail vmail 0 Oct 2 18:58 dovecot-uidvalidity.506b393c -rw-------. 1 vmail vmail 28 Oct 2 20:21 maildirsize Jack From CMarcus at Media-Brokers.com Wed Oct 3 18:03:18 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 03 Oct 2012 11:03:18 -0400 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C4D0B.8030709@airstreamcomm.net> References: <506C4D0B.8030709@airstreamcomm.net> Message-ID: <506C53B6.6070706@Media-Brokers.com> On 2012-10-03 10:34 AM, list at airstreamcomm.net wrote: > In the docs it states that LDA " ...takes mail from anMTAand delivers > it to a user's mailbox, while keeping Dovecot index files up to > date." I am wondering if LMTP also interacts with the Dovecot index > files and keeps them up to date? Yes... although it doesn't seem to state that explicitly in the wiki... http://wiki2.dovecot.org/LMTP -- Best regards, Charles From jbates at brightok.net Wed Oct 3 18:13:40 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 03 Oct 2012 10:13:40 -0500 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C53B6.6070706@Media-Brokers.com> References: <506C4D0B.8030709@airstreamcomm.net> <506C53B6.6070706@Media-Brokers.com> Message-ID: <506C5624.8090609@brightok.net> On 10/3/2012 10:03 AM, Charles Marcus wrote: > On 2012-10-03 10:34 AM, list at airstreamcomm.net > wrote: >> In the docs it states that LDA " ...takes mail from anMTAand delivers >> it to a user's mailbox, while keeping Dovecot index files up to >> date." I am wondering if LMTP also interacts with the Dovecot index >> files and keeps them up to date? > > Yes... although it doesn't seem to state that explicitly in the wiki... > > http://wiki2.dovecot.org/LMTP > "The main difference is that the LDA is a short-running process, started as a binary from command line, while LMTP is a long-running process started by Dovecot's master process." I think they are trying to reduce duplication of information. Jack From patrickdk at patrickdk.com Wed Oct 3 19:02:20 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 03 Oct 2012 12:02:20 -0400 Subject: [Dovecot] possible nfs issue In-Reply-To: References: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> Message-ID: <20121003120220.Horde.5oXwJJLnE6FQbGGMsr6Tr4A@mail.patrickdk.com> Maybe it's a cross program issue? We used to randomly have this happen a long time ago, when using postfix and dovecot. Since switching to using the dovecot lda/lmtp instead of postfix for mailbox delievery, I haven't seen this happen at all anymore. I'm not saying that postfix is at fault for this, but could be a timing/race issue between postfix/dovecot accesses to the mailbox. Quoting Cor Bosman : > On Oct 3, 2012, at 12:35 AM, Timo Sirainen wrote: > >> On 3.10.2012, at 0.45, Timo Sirainen wrote: >> >>> On 3.10.2012, at 0.39, Cor Bosman wrote: >>> >>>> With NFS these files are created when a file gets unlinked, but >>>> another process still has it open. It disappears as soon as the >>>> other process closes it. For some reason they dont disappear. As >>>> far as I can tell we've had no server crashes that could explain >>>> this. One possible theory is that a rename happens after an >>>> unlink. In that case the file remains. This could possibly be a >>>> dovecot issue. >>> >>> How can a rename happen after unlink? The rename should fail. >>> (Unless doing rename(.nfs1234, something), but Dovecot definitely >>> isn't doing that.) >> >> You could see if this old test program leaves .nfs files lying around: >> >> http://dovecot.org/tmp/readdir.c >> >> Just comment out the line: >> >> close(fd); >> > > I meant the .nfs1234 indeed, but it seemed very unlikely. Thanks for > clarifying. The readdir program leaves no .nfs files. We'll have to > explore other possibilities. > > Cor From list at airstreamcomm.net Wed Oct 3 22:26:20 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 03 Oct 2012 14:26:20 -0500 Subject: [Dovecot] LMTP userdb lookup Message-ID: <506C915C.2070709@airstreamcomm.net> Is it possible to have separate userdb lookups for LMTP and POP/IMAP? From marc at perkel.com Wed Oct 3 22:48:21 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 12:48:21 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release Message-ID: <506C9685.8070906@perkel.com> Hi, I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first. I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives. I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend. However What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders? Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains? One thing I can do is deliver the spam to 3 different places so it's visible on all levels. I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system. From ecasarero at gmail.com Wed Oct 3 22:58:35 2012 From: ecasarero at gmail.com (Eduardo Casarero) Date: Wed, 3 Oct 2012 16:58:35 -0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: Hi Marc, i solved this using an automated report for users quarantine. In front of dovecot i have 2 mailscanner boxes that stores spam emails in quarantine and logs them to a database, periodically there is a script that sends an html report to users that recieved spam in the last interval (1h, 4h, 24hs depending on the user preferences) showing a list of time-from-subject of all new items in quarantine. There is also a link to release the email from quarantine and the users recieves it on his inbox. So our users can release emails without bothering anyone. (There is also an admin view where the admin can see all the trafic for the domain). my 2cents. regards, eduardo. 2012/10/3 Marc Perkel > Hi, > > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. (Junk > Email Filter) I want to add a system where I store a copy of spam on a > server and make it available to the customer to review and maybe resent on > false positives. > > I know I could do something simple where I deliver all spam to a domain > account and make it available to an administrator. Then if it's a false > positive they would drag the message to a "resend" folder. I'll have > something the checks the folder one a minute to pick up and resend. > > However > > What would be very cool is delivering the spam to individual accounts. So > a user who logs in individually can see their own spam. But the admin for > the domain would be able to see all users. Maybe the users would appear as > folders? > > Then a master account (me) would be able to log in and see all the domains > as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And I'll > need an authentication system. > > From campbell at cnpapers.com Wed Oct 3 22:59:09 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Wed, 03 Oct 2012 15:59:09 -0400 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506C990D.50800@cnpapers.com> If you ever figure out how to do this, I've got an excellent name for it: MailWatch http://sourceforge.net/projects/mailwatch/ steve On 10/3/2012 3:48 PM, Marc Perkel wrote: > Hi, > > I'm looking for some advice to do a really advanced trick with > Dovecot. I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of > spam on a server and make it available to the customer to review and > maybe resent on false positives. > > I know I could do something simple where I deliver all spam to a > domain account and make it available to an administrator. Then if it's > a false positive they would drag the message to a "resend" folder. > I'll have something the checks the folder one a minute to pick up and > resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the > admin for the domain would be able to see all users. Maybe the users > would appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > From tss at iki.fi Wed Oct 3 23:04:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 03 Oct 2012 23:04:39 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: Maildir, layout=fs /var/vmail/domain/user/ Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Marc Perkel wrote: Hi, I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first. I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives. I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend. However What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders? Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains? One thing I can do is deliver the spam to 3 different places so it's visible on all levels. I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system. From hakon at alstadheim.priv.no Wed Oct 3 23:14:37 2012 From: hakon at alstadheim.priv.no (=?ISO-8859-1?Q?H=E5kon_Alstadheim?=) Date: Wed, 03 Oct 2012 22:14:37 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506C9CAD.5080004@alstadheim.priv.no> On 03. okt. 2012 21:48, Marc Perkel wrote: > Hi, > > I'm looking for some advice to do a really advanced trick with > Dovecot. I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of > spam on a server and make it available to the customer to review and > maybe resent on false positives. > > I know I could do something simple where I deliver all spam to a > domain account and make it available to an administrator. Then if it's > a false positive they would drag the message to a "resend" folder. > I'll have something the checks the folder one a minute to pick up and > resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the > admin for the domain would be able to see all users. Maybe the users > would appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > > Check out the dovecot sieve plugin. I use the following default pre-filter for all users: --- require ["regex", "fileinto", "imap4flags"]; # Catch mail tagged as Spam, except Spam retrained and delivered to the mailbox if allof (header :regex "X-DSPAM-Result" "^(Spam|Virus|Bl[ao]cklisted)$", not header :contains "X-DSPAM-Reclassified" "Innocent", not header :contains "Received-SPF" "pass .securityfocus.com") { # Mark as read #setflag "\\Seen"; addflag "$junk"; # Move into the Junk folder fileinto "INBOX.Junk"; # Stop processing here stop; } ----- Together with the dovecot antispam plugin this makes the beginnings of a very intuitive system. I just click to remove the junk flag on any false positive, and it gets re-delivered to me. The dovecot lda also supports a switch to deliver to a specific folder I believe. This would be an alternative if you get the spam delivered through a separate channel anyway. ------- The other part of your requirements could be met by using dovecot public folders, which I have never used myself. Maybe set up so admins can subscribe to the junk-folder of any user they want ? Refiling false positives might get messy for an admin though. Regards, H?kon. From tss at iki.fi Wed Oct 3 23:58:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 23:58:21 +0300 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <506C915C.2070709@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> Message-ID: On 3.10.2012, at 22.26, list at airstreamcomm.net wrote: > Is it possible to have separate userdb lookups for LMTP and POP/IMAP? protocol lmtp { userdb { .. } } protocol !lmtp { userdb { .. } } From marc at perkel.com Thu Oct 4 02:42:52 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 16:42:52 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> Message-ID: <506CCD7C.6070507@perkel.com> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? On 10/3/2012 1:04 PM, Timo Sirainen wrote: > Maildir, layout=fs > /var/vmail/domain/user/ > > Spams get delivered there. User has access there. Domain admin has > mail_location=/var/vmail/domain, you have mail_location=/var/vmail > -- > Sent from my Android phone with K-9 Mail. Please excuse my brevity. > > Marc Perkel wrote: > > Hi, > > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of spam > on a server and make it available to the customer to review and maybe > resent on false positives. > > I know I could do something simple where I deliver all spam to a domain > account and make it available to an administrator. Then if it's a false > positive they would drag the message to a "resend" folder. I'll have > something the checks the folder one a minute to pick up and resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the admin > > for the domain would be able to see all users. Maybe the users would > appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > From tss at iki.fi Thu Oct 4 02:46:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 02:46:34 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506CCD7C.6070507@perkel.com> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: On 4.10.2012, at 2.42, Marc Perkel wrote: > On 10/3/2012 1:04 PM, Timo Sirainen wrote: >> Maildir, layout=fs >> /var/vmail/domain/user/ >> >> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail > I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? users: mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs domain admins: mail_location = maildir:/var/vmail/%d:LAYOUT=fs full admins: mail_location = maildir:/var/vmail:LAYOUT=fs The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. From marc at perkel.com Thu Oct 4 03:10:23 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 17:10:23 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: <506CD3EF.6080704@perkel.com> On 10/3/2012 4:46 PM, Timo Sirainen wrote: > On 4.10.2012, at 2.42, Marc Perkel wrote: > >> On 10/3/2012 1:04 PM, Timo Sirainen wrote: >>> Maildir, layout=fs >>> /var/vmail/domain/user/ >>> >>> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail >> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? > users: > mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs > > domain admins: > mail_location = maildir:/var/vmail/%d:LAYOUT=fs > > full admins: > mail_location = maildir:/var/vmail:LAYOUT=fs > > The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. > > I'm testing it now and the user level works. But the other levels I don't see anything. I am a little brain dead today though. I'll test more From kgc at corp.sonic.net Thu Oct 4 04:55:42 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 03 Oct 2012 18:55:42 -0700 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <506CEC9E.9060105@corp.sonic.net> On 10/2/2012 2:39 PM, Cor Bosman wrote: > Anyone else with NFS mailspools seeing this? Yes, it is like 1999 all over again. I haven't had a chance to track them down or setup a cron job to rm them all. All of the ones I'm seeing are ex dovecot.index files but it looks like yours are ex messages? I figured this was a probably a regression in the RHEL6.3 (Sl6.3) (2.6.32-279.9.1.el6.x86_64) kernel. What are you running Cor? -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From marc at perkel.com Thu Oct 4 05:28:19 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 19:28:19 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: <506CF443.5080904@perkel.com> On 10/3/2012 4:46 PM, Timo Sirainen wrote: > On 4.10.2012, at 2.42, Marc Perkel wrote: > >> On 10/3/2012 1:04 PM, Timo Sirainen wrote: >>> Maildir, layout=fs >>> /var/vmail/domain/user/ >>> >>> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail >> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? > users: > mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs > > domain admins: > mail_location = maildir:/var/vmail/%d:LAYOUT=fs > > full admins: > mail_location = maildir:/var/vmail:LAYOUT=fs > > The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. > > Hi Timo, Thanks for your help. I think I'm close. This works: mail_location = maildir:/email/%d/%n:LAYOUT=fs This doesn't: mail_location = maildir:/email/%d:LAYOUT=fs The email client doesn't see the directories as folders and nothing is visible. I must be missing something. From marc at perkel.com Thu Oct 4 07:40:37 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 21:40:37 -0700 Subject: [Dovecot] Getting rid of the subscription file Message-ID: <506D1345.9070406@perkel.com> I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? Thanks in advance. From daniel.parthey at informatik.tu-chemnitz.de Thu Oct 4 07:58:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 4 Oct 2012 06:58:18 +0200 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1345.9070406@perkel.com> References: <506D1345.9070406@perkel.com> Message-ID: <20121004045818.GA15696@daniel.localdomain> Marc Perkel wrote: > I'd like to eliminate the subscription file and have it just list > the folders that are there. How do I do that? Dovecot allows to automatically subscribe folders when mails are delivered: lda_mailbox_autosubscribe = yes RFC 3501 tells that the server side MUST NOT unilaterally remove an existing mailbox name from the subscription list even if a mailbox by that name no longer exists, see SUBSCRIBE in section 6.3.6: http://tools.ietf.org/html/rfc3501#section-6.3.6 Regards Daniel -- https://plus.google.com/103021802792276734820 From marc at perkel.com Thu Oct 4 08:06:28 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 22:06:28 -0700 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <20121004045818.GA15696@daniel.localdomain> References: <506D1345.9070406@perkel.com> <20121004045818.GA15696@daniel.localdomain> Message-ID: <506D1954.1000204@perkel.com> On 10/3/2012 9:58 PM, Daniel Parthey wrote: > Marc Perkel wrote: >> I'd like to eliminate the subscription file and have it just list >> the folders that are there. How do I do that? > Dovecot allows to automatically subscribe folders when mails are delivered: > lda_mailbox_autosubscribe = yes > > RFC 3501 tells that the server side MUST NOT unilaterally remove an > existing mailbox name from the subscription list even if a mailbox > by that name no longer exists, see SUBSCRIBE in section 6.3.6: > http://tools.ietf.org/html/rfc3501#section-6.3.6 > > Regards > Daniel In my case I don't care what the standard is. I want to get rid of the subscription concept completely. From amateo at um.es Thu Oct 4 09:14:29 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 04 Oct 2012 08:14:29 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> Message-ID: <506D2945.9040208@um.es> El 03/10/12 15:27, Timo Sirainen escribi?: > On 3.10.2012, at 9.20, Angel L. Mateo wrote: > >> El 02/10/12 21:38, Timo Sirainen escribi?: >>> On 26.9.2012, at 13.55, Angel L. Mateo wrote: >>> >>>> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >>>> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >>>> done >>> >>> -S auth-userdb? You've named it completely wrong if that works :) >>> >> auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? > > -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? > Yes, I have tried and it works without using -S. So, what is the reason for this option? Because I'm sure I'm using because I've read it in examples (not with auth-userdb obviously, this is my mistake) From robert at schetterer.org Thu Oct 4 09:36:39 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 04 Oct 2012 08:36:39 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506D2E77.5030303@schetterer.org> Am 03.10.2012 21:48, schrieb Marc Perkel: > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of spam > on a server and make it available to the customer to review and maybe > resent on false positives. this is the job of your filter comapny first, anyway , dont use them anymore and use i.e amavis with quarantaine i dont think other cases make sense in real by getting very complicated -- Best Regards MfG Robert Schetterer From cor at xs4all.nl Thu Oct 4 11:11:56 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 4 Oct 2012 10:11:56 +0200 Subject: [Dovecot] possible nfs issue In-Reply-To: <506CEC9E.9060105@corp.sonic.net> References: <506CEC9E.9060105@corp.sonic.net> Message-ID: <5A995EF4-6EAA-41FF-926B-912FFD59EC07@xs4all.nl> On Oct 4, 2012, at 3:55 AM, Kelsey Cummings wrote: > On 10/2/2012 2:39 PM, Cor Bosman wrote: >> Anyone else with NFS mailspools seeing this? > > Yes, it is like 1999 all over again. I haven't had a chance to track them down or setup a cron job to rm them all. All of the ones I'm seeing are ex dovecot.index files but it looks like yours are ex messages? > > I figured this was a probably a regression in the RHEL6.3 (Sl6.3) (2.6.32-279.9.1.el6.x86_64) kernel. What are you running Cor? We're running debian with a 3.2.2 kernel. Just yesterday one of my colleagues had a few new ones in his mailspool. Definitely no server crash or anything. Something is creating these outside the 'normal' parameters for .nfs files. My colleague said these were emails he deleted that day. We've set up a cleaning run, and are probably going to ignore it for now. These things are near impossible to track down without a lot of debugging. Cor From marc at perkel.com Thu Oct 4 15:54:35 2012 From: marc at perkel.com (Marc Perkel) Date: Thu, 04 Oct 2012 05:54:35 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506D2E77.5030303@schetterer.org> References: <506C9685.8070906@perkel.com> <506D2E77.5030303@schetterer.org> Message-ID: <506D870B.5020001@perkel.com> On 10/3/2012 11:36 PM, Robert Schetterer wrote: > Am 03.10.2012 21:48, schrieb Marc Perkel: >> I'm looking for some advice to do a really advanced trick with Dovecot. >> I'm not sure if this can be done. I need to describe first. >> >> I have a spam filtering company that does front end spam filtering. >> (Junk Email Filter) I want to add a system where I store a copy of spam >> on a server and make it available to the customer to review and maybe >> resent on false positives. > this is the job of your filter comapny first, > anyway , dont use them anymore and use i.e amavis with quarantaine > i dont think other cases make sense in real by getting very complicated I am the spam filtering company. :) From benedetto.vassallo at unipa.it Thu Oct 4 16:00:03 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Thu, 04 Oct 2012 15:00:03 +0200 Subject: [Dovecot] Maildir hardlinks Message-ID: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Hello list, Excuse me for my poor english. I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. All works fine, but with the new version it seems that dovecot don't do hardlinks when deliver a message to multiple users. I have checked my config and the only rule I can see aboout that is maildir_copy_with_hardlinks = yes in /etc/dovecot/conf.d/10-mail.conf My mail location config is mail_location = maildir:~/MailDir:LAYOUT=fs I tryed using lmtp directly issuing 'telnet localhost 24' and sending a test message to 3 recipients. Then issuing a 'ls -il' in the "new" directory of that users, I saw the inode was not the same. I rechecked my config and take a look in the wiki and in the list for someone who had the same problems, but found nothing. Maybe I did something wrong, but I can't understand what. Can you help me? Thank you -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From tss at iki.fi Thu Oct 4 17:20:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:20:03 +0300 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1345.9070406@perkel.com> References: <506D1345.9070406@perkel.com> Message-ID: On 4.10.2012, at 7.40, Marc Perkel wrote: > I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? Write a plugin. From tss at iki.fi Thu Oct 4 17:21:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:21:02 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <506D2945.9040208@um.es> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> <506D2945.9040208@um.es> Message-ID: <2E23C2EC-6324-4182-9828-1F063F63C86C@iki.fi> On 4.10.2012, at 9.14, Angel L. Mateo wrote: >> -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? >> > Yes, I have tried and it works without using -S. So, what is the reason for this option? Because I'm sure I'm using because I've read it in examples (not with auth-userdb obviously, this is my mistake) I'm guessing that it's used only when the user lookup isn't returning proxy=y From tss at iki.fi Thu Oct 4 17:29:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:29:02 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506CF443.5080904@perkel.com> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> Message-ID: <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> On 4.10.2012, at 5.28, Marc Perkel wrote: > Thanks for your help. I think I'm close. > > This works: > mail_location = maildir:/email/%d/%n:LAYOUT=fs > > This doesn't: > mail_location = maildir:/email/%d:LAYOUT=fs > > The email client doesn't see the directories as folders and nothing is visible. I must be missing something. Dunno. At least this method of testing works: create test mail: doveadm -O -o mail=maildir:/tmp/vmail/domain/user mailbox create INBOX touch /tmp/vmail/domain/user/cur/newmail test that user at domain works: ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs a select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS test that domain works: ./imap -O -o mail=maildir:/tmp/vmail/domain:LAYOUT=fs b list "" * * LIST (\HasNoChildren) "/" "user" * LIST (\HasNoChildren) "/" "INBOX" b OK List completed. c select user * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 0 RECENT test that superuser works: ./imap -O -o mail=maildir:/tmp/vmail:LAYOUT=fs d list "" * * LIST (\Noselect \HasChildren) "/" "domain" * LIST (\HasNoChildren) "/" "domain/user" * LIST (\HasNoChildren) "/" "INBOX" d OK List completed. e select domain/user * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS From list at airstreamcomm.net Thu Oct 4 17:41:39 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 04 Oct 2012 09:41:39 -0500 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: References: <506C915C.2070709@airstreamcomm.net> Message-ID: <506DA023.5030609@airstreamcomm.net> On 10/3/12 3:58 PM, Timo Sirainen wrote: > On 3.10.2012, at 22.26, list at airstreamcomm.net wrote: > >> Is it possible to have separate userdb lookups for LMTP and POP/IMAP? > protocol lmtp { > userdb { > .. > } > } > protocol !lmtp { > userdb { > .. > } > } > > Forgot to mention I am running 2.0.17. I separated all the userdb passdb lookups into their own protocol configuration like so: protocol imap { userdb { .. } passdb { .. } } protocol pop3 { userdb { .. } passdb { .. } } protocol lmtp { userdb { .. } } And I am getting the following error: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one From a previous post it appears that Dovecot cannot run without a global lookups specified: http://www.dovecot.org/list/dovecot/2012-March/064407.html Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? From tss at iki.fi Thu Oct 4 17:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:58:53 +0300 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <506DA023.5030609@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> Message-ID: On 4.10.2012, at 17.41, list at airstreamcomm.net wrote: >> protocol lmtp { >> userdb { >> .. >> } >> } >> protocol !lmtp { >> userdb { >> .. >> } >> } >> >> > Forgot to mention I am running 2.0.17. The above works in v2.1. > And I am getting the following error: > > auth: Fatal: No passdbs specified in configuration file. PLAIN > mechanism needs one > > > From a previous post it appears that Dovecot cannot run without a global lookups specified: > > http://www.dovecot.org/list/dovecot/2012-March/064407.html > > Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? The !lmtp version avoids that fatal problem. So the solution is: upgrade. From CMarcus at Media-Brokers.com Thu Oct 4 18:03:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 04 Oct 2012 11:03:02 -0400 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Message-ID: <506DA526.4020606@Media-Brokers.com> On 2012-10-04 9:00 AM, Benedetto Vassallo wrote: > Hello list, > Excuse me for my poor english. > I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. If you are going to update, why ohy why update to an outdated version? Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... From benedetto.vassallo at unipa.it Thu Oct 4 19:13:41 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Thu, 04 Oct 2012 18:13:41 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <506DA526.4020606@Media-Brokers.com> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <506DA526.4020606@Media-Brokers.com> Message-ID: <20121004181341.14266g7w0m2ie75h@webmail.unipa.it> Def. Quota Charles Marcus : > On 2012-10-04 9:00 AM, Benedetto Vassallo > wrote: >> Hello list, >> Excuse me for my poor english. >> I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. > > If you are going to update, why ohy why update to an outdated version? > > Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... > I did it, but still don't work :-( -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From slusarz at curecanti.org Thu Oct 4 21:57:45 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Thu, 04 Oct 2012 12:57:45 -0600 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1954.1000204@perkel.com> References: <506D1345.9070406@perkel.com> <20121004045818.GA15696@daniel.localdomain> <506D1954.1000204@perkel.com> Message-ID: <20121004125745.Horde.Wz49XoF5lbhQbdwp53YwxRA@bigworm.curecanti.org> Quoting Marc Perkel : > On 10/3/2012 9:58 PM, Daniel Parthey wrote: >> Marc Perkel wrote: >>> I'd like to eliminate the subscription file and have it just list >>> the folders that are there. How do I do that? [snip] >> RFC 3501 tells that the server side MUST NOT unilaterally remove an >> existing mailbox name from the subscription list even if a mailbox >> by that name no longer exists, see SUBSCRIBE in section 6.3.6: >> http://tools.ietf.org/html/rfc3501#section-6.3.6 >> >> Regards >> Daniel > > In my case I don't care what the standard is. I want to get rid of > the subscription concept completely. Use an MUA that allows configuration to explicitly ignore the subscription concept. michael From micha at krausam.de Fri Oct 5 10:45:07 2012 From: micha at krausam.de (Micha Krause) Date: Fri, 05 Oct 2012 09:45:07 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> Message-ID: <506E9003.7030201@krausam.de> Hi, > ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs Wow, thats a really cool way to debug/test mailboxes, is this documented somewhere? What does -O do, any other interesting options? Micha Krause From stsiol at yahoo.co.uk Fri Oct 5 15:00:20 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Fri, 5 Oct 2012 13:00:20 +0100 (BST) Subject: [Dovecot] horde sync status ? Message-ID: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Hello all, I had a quick look at the horde site and noticed that horde is being advertised as, let's say, "smartphone friendly". Does anyone know if the newest horde version can "talk" to? smart phones in regards to e-mails ? In other words, can a user owning a smartphone get his/her e-mails on it apart from the webpage ? Thank you, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From h.reindl at thelounge.net Fri Oct 5 15:05:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 05 Oct 2012 14:05:03 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506ECCEF.1020904@thelounge.net> Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > Hello all, > > I had a quick look at the horde site and noticed that > horde is being advertised as, let's say, "smartphone friendly". > > Does anyone know if the newest horde version can "talk" to > smart phones in regards to e-mails ? > > In other words, can a user owning a smartphone get his/her > e-mails on it apart from the webpage? a little off-topic at all, but however horde/imp is a webmail and accessing the mailserver via IMAP smart-phone freindly means it can be used on smartphones without scrolling to dead horde is not for having a layer between mail-client on the smartphone and the server - this makes pretty no sense each smartphone these days has a mail-client like K9 on android and can access imap/exchange directly - why should horde be involved here als additional layer? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From patrickdk at patrickdk.com Fri Oct 5 17:17:44 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Fri, 05 Oct 2012 10:17:44 -0400 Subject: [Dovecot] horde sync status ? In-Reply-To: <506ECCEF.1020904@thelounge.net> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506ECCEF.1020904@thelounge.net> Message-ID: <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> Quoting Reindl Harald : > Am 05.10.2012 14:00, schrieb Spyros Tsiolis: >> Hello all, >> >> I had a quick look at the horde site and noticed that >> horde is being advertised as, let's say, "smartphone friendly". >> >> Does anyone know if the newest horde version can "talk" to >> smart phones in regards to e-mails ? >> >> In other words, can a user owning a smartphone get his/her >> e-mails on it apart from the webpage? > > a little off-topic at all, but however > > horde/imp is a webmail and accessing the mailserver via IMAP > smart-phone freindly means it can be used on smartphones > without scrolling to dead > > horde is not for having a layer between mail-client on the > smartphone and the server - this makes pretty no sense > > each smartphone these days has a mail-client like K9 on > android and can access imap/exchange directly - why should > horde be involved here als additional layer? Many reasons for this, I personally use it for contact and calender sync, and the new version of horde that is still in beta, can also be used for email sync too. This will simplify setup for many people, using autodiscovery feature of activesync. From tss at iki.fi Fri Oct 5 17:31:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 5 Oct 2012 17:31:18 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506E9003.7030201@krausam.de> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> Message-ID: <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> On 5.10.2012, at 10.45, Micha Krause wrote: >> ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs > > Wow, thats a really cool way to debug/test mailboxes, is this documented somewhere? No. The -O, -o, -k and some other options should be put into some new global.inc where it gets included to all doveadm/dovecot/doveconf man pages.. > What does -O do, any other interesting options? All the global settings are: -O ignores dovecot.conf and just uses the default settings. -o = can be used multiple times to override any setting -k preserves environment variables (which can also be used to override settings, e.g. MAIL=foo) -c changes dovecot.conf path -i changes to dovecot.conf used by the given instance name -L logs directly to destination specified by log_path/info_log_path/debug_log_path, bypassing log process (allowing logging to different location than normally, log process always logs only to one location) From h.reindl at thelounge.net Fri Oct 5 17:38:50 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 05 Oct 2012 16:38:50 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506ECCEF.1020904@thelounge.net> <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> Message-ID: <506EF0FA.1010307@thelounge.net> Am 05.10.2012 16:17, schrieb Patrick Domack: >> each smartphone these days has a mail-client like K9 on >> android and can access imap/exchange directly - why should >> horde be involved here als additional layer? > > Many reasons for this, I personally use it for contact and calender sync, and the new version of horde that is > still in beta, can also be used for email sync too. This will simplify setup for many people, using autodiscovery > feature of activesync. why does one need this with IMAP as mail-backend? sounds more like "having solution and searching fro problem" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From robert at schetterer.org Fri Oct 5 17:56:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 05 Oct 2012 16:56:36 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506EF524.7060604@schetterer.org> Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > Hello all, > > I had a quick look at the horde site and noticed that > horde is being advertised as, let's say, "smartphone friendly". > > Does anyone know if the newest horde version can "talk" to > smart phones in regards to e-mails ? yes since version 5 > > In other words, can a user owning a smartphone get his/her > e-mails on it apart from the webpage ? horde 5 acts as active-sync server for mail , calendar, adressbook ,tasks ,notes syncml with funambol app on the smartphone side for calendar, adressbook ,tasks ,notes roadmap 5.1 is planned as card/caldav server http://wiki.horde.org/ActiveSync > > Thank you, > > spyros > > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > this is off topic with the dovecot list -- Best Regards MfG Robert Schetterer From mikydevel at yahoo.fr Fri Oct 5 20:30:45 2012 From: mikydevel at yahoo.fr (Mik J) Date: Fri, 5 Oct 2012 18:30:45 +0100 (BST) Subject: [Dovecot] Dovecot configuration and question about IP trusted Message-ID: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> Hello list, I've just finished to install Dovecot and things seems to work so far. After some little efforts though. My version is 2.0.20 Question 1: I'm trying to tighten the security a little bit and added in dovecot.conf login_trusted_networks = 192.168.1.0/30 Then restarted Dovecot My client has the IP 192.168.1.20 and it's still able to retrieve emails. I expected it to be forbidden. Am I missing something ? # dovecot -n | grep trust login_trusted_networks = 192.168.1.0/30 Question 2: I feel that Dovecot is slow. I'm doing my test with my iphone as an imap client. Test 1: I retrieve a mail on a remote server provided by a hosting company, it takes 2 seconds Test 2: I retrieve a mail on my server which is on my LAN, the mail includes a few letters in the subject and a few letters in the body. The action takes about 8 seconds. It's quite subtule to measure so first I would like to know if Dovecot tries to do a dns reverse lookup or something like that. And it would explain the overhead. Thank you From bob at computerisms.ca Fri Oct 5 21:01:13 2012 From: bob at computerisms.ca (Bob Miller) Date: Fri, 05 Oct 2012 11:01:13 -0700 Subject: [Dovecot] Dovecot configuration and question about IP trusted In-Reply-To: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> Message-ID: <1349460073.4213.59.camel@worklian> Hi, > I > 'm trying to tighten the security a little bit and added in dovecot.conf > login_trusted_networks = 192.168.1.0/30 > Then restarted Dovecot > > > My client has the IP 192.168.1.20 and it's still able to retrieve emails. I expected it to be forbidden. Am I missing something ? My interpretation of the documentation indicates that the trusted network setting causes certain authentication and security checks to be bypassed if a computer is in the trusted network, and to not bypass those authentication and security checks if the computer is not in the trusted range. I see nothing indicating this setting will "forbid" anything... > I feel that Dovecot is slow. I'm doing my test with my iphone as an imap client. > Test 1: I retrieve a mail on a remote server provided by a hosting company, it takes 2 seconds > Test 2: I retrieve a mail on my server which is on my LAN, the mail includes a few letters in the subject and a few letters in the body. The action takes about 8 seconds. > It's quite subtule to measure so first I would like to know if Dovecot tries to do a dns reverse lookup or something like that. And it would explain the overhead. I don't know about the reverse lookup, but this sounds like a caching issue to me. http://wiki2.dovecot.org/IndexFiles > > Thank you -- Computerisms Bob Miller 867-334-7117 / 867 633 3760 http://computerisms.ca From lists at luigirosa.com Fri Oct 5 21:14:53 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Fri, 05 Oct 2012 20:14:53 +0200 Subject: [Dovecot] IPv6 & SSL Message-ID: <506F239D.6090007@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I have a dual stack server with Dovecot 2.1.10 listening on v4 and v6 Dovecot has a Comodo SSL certificate issued via NameCheap that works as expected with IPv4 in 10-ssl.conf I have enabled these configuraction directives: ssl = yes ssl_cert = < /path/to/file.crt ssl_key = < /path/to/file.key ssl_parameters_regenerate = 202 hours If I connect to Dovecot using the IPv6 address of the server with Thunderbird 15.0.1 uising CRAM-MD5 averything is ok. If I enable SSL _and_ IPv6 on Thunderbird I get this error: Oct 5 20:05:04 mail dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=2001:470:1f09:203:fdbf:508e:4a29:56c5, lip=2001:470:1f09:203::badd:ecaf, TLS: SSL_read() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, session= Ciao, luigi - -- / +--[Luigi Rosa]-- \ I will tell you a great secret, Captain. Perhaps the greatest of all time. The molecules of your body are the same molecules that make up this station and the nebula outside, that burn inside the stars themselves. We are star stuff, we are the universe made manifest, trying to figure itself out. As we have both learned, sometimes the universe requires a change of perspective." --Delenn, "Distant Star", Babylon 5 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBvI50ACgkQ3kWu7Tfl6ZRBSACfRkp4FYpWaEZUQhIh0t6Vfs/I JbcAoKGZ769yogYS7faCXKvPTuhQiHA8 =jxCB -----END PGP SIGNATURE----- From lists at wildgooses.com Fri Oct 5 22:37:37 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 05 Oct 2012 20:37:37 +0100 Subject: [Dovecot] horde sync status ? In-Reply-To: <506EF524.7060604@schetterer.org> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506EF524.7060604@schetterer.org> Message-ID: <506F3701.5050805@wildgooses.com> On 05/10/2012 15:56, Robert Schetterer wrote: > Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > >> In other words, can a user owning a smartphone get his/her >> e-mails on it apart from the webpage ? > horde 5 acts as active-sync server > for mail , calendar, adressbook ,tasks ,notes > > syncml with funambol app on the smartphone side > for calendar, adressbook ,tasks ,notes > > roadmap > 5.1 is planned as card/caldav server > > http://wiki.horde.org/ActiveSync > Also see Sogo (and owncloud). Plus the Sogosync connector This is a developing area (at last) Ed W From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 5 22:48:47 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 5 Oct 2012 21:48:47 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> Message-ID: <20121005194847.GA15222@daniel.localdomain> Timo Sirainen wrote: > -i changes to dovecot.conf used by the given instance name This does not seem to work, at least not with version 2.1.10: mail01:~# doveadm instance list path name last used running /var/run/dovecot dovecot-mailbox 2012-10-05 19:19:33 yes /var/run/dovecot-director dovecot-director 2012-10-05 19:20:13 yes mail01:~# doveadm -c /etc/dovecot-director/dovecot-director.conf director status dparthey at example.org Current: 10.129.3.192 (expires 2012-10-07 20:10:25) Hashed: 10.129.3.192 Initial config: 10.129.3.192 mail01:~# doveadm -i dovecot-director director status dparthey at example.org doveadm(root): Fatal: read(/var/run/dovecot/director-admin) failed: Connection reset by peer Regards Daniel -- https://plus.google.com/103021802792276734820 From nick+dovecot at bunbun.be Fri Oct 5 23:47:53 2012 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Fri, 05 Oct 2012 22:47:53 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F239D.6090007@luigirosa.com> References: <506F239D.6090007@luigirosa.com> Message-ID: <506F4779.4040109@bunbun.be> Luigi Rosa wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > I have a dual stack server with Dovecot 2.1.10 listening on v4 and v6 > > Dovecot has a Comodo SSL certificate issued via NameCheap that works as > expected with IPv4 > > in 10-ssl.conf I have enabled these configuraction directives: > > ssl = yes > ssl_cert =< /path/to/file.crt > ssl_key =< /path/to/file.key > ssl_parameters_regenerate = 202 hours > > > If I connect to Dovecot using the IPv6 address of the server with Thunderbird > 15.0.1 uising CRAM-MD5 averything is ok. > If I enable SSL _and_ IPv6 on Thunderbird I get this error: How do you enable this in Thunderbird? If by "enabling IPv6" you mean you put in the IPv6 address in stead of the hostname, that's probably where you're wrong. The certificate contains your hostname, not the IP-address so the hostname verification check fails if you insert the IPv6 address (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your connection fails). I've verified this by changing the hostname to IPv6 in Thunderbird and got the same error as you do. You would get the same error if you configure the IPv4 address in TB. > Oct 5 20:05:04 mail dovecot: imap-login: Disconnected (no auth attempts in 1 > secs): user=<>, rip=2001:470:1f09:203:fdbf:508e:4a29:56c5, > lip=2001:470:1f09:203::badd:ecaf, TLS: SSL_read() failed: error:14094418:SSL > routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, > session= This is a valid connection when I use the hostname: 2012-10-04T18:07:51.614187+02:00 mail dovecot: imap-login: Login: user=, method=CRAM-MD5, rip=yyyy:yyyy:::yyyy, lip=xxxx:xxxx:::xxxx, mpid=58179, TLS, TLSv1 with cipher RC4-MD5 (128/128 bits) Configure your DNS so your hostname points to both the IPv6 and IPv4 address. Your client will take take whichever protocol is preferred (IPv4 or IPv6). Rgds, N. > > Ciao, > luigi > > - -- > / > +--[Luigi Rosa]-- > \ > > I will tell you a great secret, Captain. Perhaps the greatest of all > time. The molecules of your body are the same molecules that make up > this station and the nebula outside, that burn inside the stars > themselves. We are star stuff, we are the universe made manifest, > trying to figure itself out. As we have both learned, sometimes > the universe requires a change of perspective." > --Delenn, "Distant Star", Babylon 5 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Mozilla -http://www.enigmail.net/ > > iEYEARECAAYFAlBvI50ACgkQ3kWu7Tfl6ZRBSACfRkp4FYpWaEZUQhIh0t6Vfs/I > JbcAoKGZ769yogYS7faCXKvPTuhQiHA8 > =jxCB > -----END PGP SIGNATURE----- From lists at luigirosa.com Sat Oct 6 08:20:20 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 07:20:20 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F4779.4040109@bunbun.be> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> Message-ID: <506FBF94.30607@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nick Rosier said the following on 05/10/12 22:47: >> How do you enable this in Thunderbird? If by "enabling IPv6" you mean you >> put in the IPv6 address in stead of the hostname, that's probably where >> you're wrong. The certificate contains your hostname, not the IP-address >> so the hostname verification check fails if you insert the IPv6 address >> (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your >> connection fails). Good point. But does not explain why it works if I put the IPv4 address of the server (the local LAN IPv4, not the public IPv4). >> I've verified this by changing the hostname to IPv6 in Thunderbird and >> got the same error as you do. You would get the same error if you >> configure the IPv4 address in TB. The server I am referring to has 2 NICs one with a public IP and the other with a local IP address (10.0.0.254) If I put 10.0.0.254 instead of the IPv6 address I can successfully connect using TLS: Oct 6 07:13:44 mail dovecot: imap-login: Login: user=, method=CRAM-MD5, rip=10.0.0.155, lip=10.0.0.254, mpid=17812, TLS, session= >> Configure your DNS so your hostname points to both the IPv6 and IPv4 >> address. Your client will take take whichever protocol is preferred (IPv4 >> or IPv6). Thunderbird uses IPv4 as mail protocol, I wanted to test IPv6... Thank you for your help Ciao, luigi - -- / +--[Luigi Rosa]-- \ Success is 99% failure. --Soichiro Honda -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBvv4kACgkQ3kWu7Tfl6ZQp2wCgvXPgRGANlAIaVkMvXZHIThYE OiwAoIOqIMD+3mT1znMl6lCCbHanwBta =B/r2 -----END PGP SIGNATURE----- From kamath at moltingpenguin.com Sat Oct 6 08:44:56 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Fri, 5 Oct 2012 22:44:56 -0700 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506FBF94.30607@luigirosa.com> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> <506FBF94.30607@luigirosa.com> Message-ID: <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> On Oct 5, 2012, at 10:20 PM, Luigi Rosa wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Nick Rosier said the following on 05/10/12 22:47: > >>> How do you enable this in Thunderbird? If by "enabling IPv6" you mean you >>> put in the IPv6 address in stead of the hostname, that's probably where >>> you're wrong. The certificate contains your hostname, not the IP-address >>> so the hostname verification check fails if you insert the IPv6 address >>> (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your >>> connection fails). > > Good point. But does not explain why it works if I put the IPv4 address of the > server (the local LAN IPv4, not the public IPv4). > >>> I've verified this by changing the hostname to IPv6 in Thunderbird and >>> got the same error as you do. You would get the same error if you >>> configure the IPv4 address in TB. > > The server I am referring to has 2 NICs one with a public IP and the other > with a local IP address (10.0.0.254) > > If I put 10.0.0.254 instead of the IPv6 address I can successfully connect > using TLS: > > Oct 6 07:13:44 mail dovecot: imap-login: Login: user=, > method=CRAM-MD5, rip=10.0.0.155, lip=10.0.0.254, mpid=17812, TLS, > session= And do you have a PTR record for 10.0.0.254? Sean From lists at luigirosa.com Sat Oct 6 09:33:31 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 08:33:31 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> <506FBF94.30607@luigirosa.com> <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> Message-ID: <506FD0BB.2020000@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sean Kamath said the following on 06/10/12 07:44: >> Oct 6 07:13:44 mail dovecot: imap-login: Login: >> user=, method=CRAM-MD5, rip=10.0.0.155, >> lip=10.0.0.254, mpid=17812, TLS, session= > > And do you have a PTR record for 10.0.0.254? No, no PTR o other DNS entry for that address. No entry of that address in /etc/hosts on the Linux with Thunderbird or on the Linux with Dovecot. Ciao, luigi - -- / +--[Luigi Rosa]-- \ The past was erased, the erasure was forgotten, the lie became truth. --George Orwell, "1984" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBv0LsACgkQ3kWu7Tfl6ZRTUgCgh1epu40NUiZ6CPlBrcFZezt/ nMYAnjUS5IxodwJfW7o9pJHfKoVCc9xK =8O4T -----END PGP SIGNATURE----- From pw at wk-serv.de Sat Oct 6 10:29:05 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 06 Oct 2012 09:29:05 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F239D.6090007@luigirosa.com> References: <506F239D.6090007@luigirosa.com> Message-ID: Can you provide the output of doveconf -n? Regards Patrick From lists at luigirosa.com Sat Oct 6 11:10:40 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 10:10:40 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: References: <506F239D.6090007@luigirosa.com> Message-ID: <506FE780.9000900@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Patrick Westenberg said the following on 06/10/12 09:29: > Can you provide the output of doveconf -n? Sure, here it is: # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.1.1.el5.centos.plus x86_64 CentOS release 5.8 (Final) auth_cache_negative_ttl = 0 auth_cache_size = 100 k auth_cache_ttl = 8 hours auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes base_dir = /var/run/dovecot/ login_greeting = Ready. login_trusted_networks = 10.0.0.0/24 mail_plugins = " stats" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 10s stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service stats { fifo_listener stats-mail { mode = 0666 } } ssl_cert = References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> Message-ID: <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> > De?: Bob Miller > > Hi, >> I'm trying to tighten the security a little bit and added in dovecot.conf > >> login_trusted_networks = 192.168.1.0/30 >> Then restarted Dovecot >> >> >> My client has the IP 192.168.1.20 and it's still able to retrieve > emails. I expected it to be forbidden. Am I missing something ? > > My interpretation of the documentation indicates that the trusted > network setting causes certain authentication and security checks to be > bypassed if a computer is in the trusted network, and to not bypass > those authentication and security checks if the computer is not in the > trusted range.? I see nothing indicating this setting will "forbid" > anything... > >> I feel that Dovecot is slow. I'm doing my test with my iphone as an > imap client. >> Test 1: I retrieve a mail on a remote server provided by a hosting company, > it takes 2 seconds >> Test 2: I retrieve a mail on my server which is on my LAN, the mail > includes a few letters in the subject and a few letters in the body. The action > takes about 8 seconds. >> It's quite subtule to measure so first I would like to know if Dovecot > tries to do a dns reverse lookup or something like that. And it would explain > the overhead. > > I don't know about the reverse lookup, but this sounds like a caching > issue to me.? http://wiki2.dovecot.org/IndexFiles Hello Bob, Thank you for this clarification about the parameter login_trusted_networks. Regarding the indexfiles, I've read the page but I don't see at any moment, how to enable or disable the indexes. Also how do you understand this sentence "# Don't use mmap() at all. This is required if you store indexes to shared # filesystems (NFS or clustered filesystem) or for some operating systems # which use a separate cache for mmap, such as OpenBSD. mmap_disable = yes" I've read it 10 times, and I don't know if this should be set to yes or no (probably because my english is not perfect). My operating system is OpenBSD and I don't share NFS or cluster filesystems. Thank you From pw at wk-serv.de Sat Oct 6 13:02:30 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 06 Oct 2012 12:02:30 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506FE780.9000900@luigirosa.com> References: <506F239D.6090007@luigirosa.com> <506FE780.9000900@luigirosa.com> Message-ID: <507001B6.2000704@wk-serv.de> Hi Luigi, with regard to SSL my configuration is much more simple and it works fine with IPv4 and IPv6. But you have of course to use a hostname matching the certificates common name. # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain login director_mail_servers = 172.17.1.1 172.17.1.2 director_servers = 172.17.1.3 172.17.1.4 lmtp_proxy = yes log_path = /var/log/dovecot.log managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacati on subaddress comparator-i;ascii-numeric relational regex imap4flags copy includ e variables body enotify environment mailbox date ihave protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 172.17.1.3 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 172.17.1.3 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = References: <506F239D.6090007@luigirosa.com> <506FE780.9000900@luigirosa.com> <507001B6.2000704@wk-serv.de> Message-ID: <507044D3.3030309@puzzled.xs4all.nl> On 10/06/2012 12:02 PM, Patrick Westenberg wrote: > Hi Luigi, > > with regard to SSL my configuration is much more simple and it works > fine with IPv4 and IPv6. But you have of course to use a hostname > matching the certificates common name. You could add additional hostnames in the certificate by specifying them in SubjectAltName. I use that so my certificate works with both the public FQDN going over the Internet as well as the internal hostname when using a VPN or on the local LAN. Regards, Patrick From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 6 18:53:53 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 6 Oct 2012 17:53:53 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <98D34C84-B1F4-47B3-9145-06E262FC11D7@iki.fi> References: <53B237A0-3A44-47DC-B41A-82CB5D174254@iki.fi> <20120710224947.GA10641@daniel.localdomain> <20120801193209.GA9069@daniel.localdomain> <20120801202502.GA9951@daniel.localdomain> <98D34C84-B1F4-47B3-9145-06E262FC11D7@iki.fi> Message-ID: <20121006155353.GA11391@daniel.localdomain> Hi Timo and list members, Timo Sirainen wrote: > On 1.8.2012, at 23.25, Daniel Parthey wrote: > > >> The error is still the same "config permission denied" shown above? I found that also from my server and added a debug patch, but it hasn't crashed yet. Could you try the attached patch and getting a gdb backtrace from the resulting core file? (Or at least the raw backtrace - getting a core file might be tricky.) > > > > Running command on a four host setup with mailbox+director instance each: > > /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A > > > > dovecot: doveadm(username at example.org): Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(vmail) egid=123(vmail) missing +r perm: /var/run/dovecot/auth-userdb, we're not in group 122(dovecot), dir owned by 0:0 mode=0755) > > Ah, so the original patch helped! This is a different error. > > > srw-rw---- 1 dovecot dovecot 0 2012-07-11 18:35 auth-userdb > > Simplest solution now would be to make this world-rw, see the auth-userdb socket configuration in http://wiki2.dovecot.org/LDA#Virtual_users > > But I guess this should also be fixed by doveadm-server. Although I don't think this should be happening by default anyway. Maybe this is also solved by the http://hg.dovecot.org/dovecot-2.1/rev/476381017ec7 patch? I finally found time to update from 2.1.8 to 2.1.10 and change service auth-user db socket to default mode of 0666. Unfortunately, the issue is still not solved and I did not manage to get a gdb backtrace, since it does not crash or assert. Current configuration of both mailbox and director is attached. The error "Permission denied" from the mailbox logs is gone, but the director doveadm command: /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A still throws the error message: doveadm(username at example.org): Error: doveadm server failure doveadm: Error: Failed to iterate through some users Any idea what I could do in addition to making /var/run/dovecot/auth-userdb world-rw? Regards Daniel -- https://plus.google.com/103021802792276734820 -------------- next part -------------- # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-43-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = imapc_features = rfc822.size imapc_host = local-mailbox imapc_port = 18143 instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota stats mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota stats imap_quota imap_stats } protocol lmtp { mail_plugins = quota stats sieve } -------------- next part -------------- # 2.1.10: /etc/dovecot-director/dovecot-director.conf # OS: Linux 2.6.32-43-server x86_64 Ubuntu 10.04.4 LTS auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot-director deliver_log_format = director: deliver: msgid=%m from=%f: %$ director_doveadm_port = 20000 director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_user_expire = 2 days disable_plaintext_auth = no doveadm_password = doveadm_proxy_port = 19000 instance_name = dovecot-director lmtp_proxy = yes login_greeting = Mail Balancer login_log_format = director: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "director: mail: %s(%u): " mail_max_userip_connections = 20 mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /etc/dovecot-director/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { executable = doveadm-server director inet_listener doveadm-server { port = 20000 } } service imap-login { executable = imap-login director inet_listener imap { port = 20143 } inet_listener imaps { port = 20993 ssl = yes } } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { address = * port = 20024 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 20200 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 20110 } inet_listener pop3s { port = 20995 ssl = yes } } ssl_cert = References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> Message-ID: <1349543117.2086.10.camel@worklian> Hi Mik, > > Also how do you understand this sentence > "# Don't use mmap() at all. This is required if you store indexes to shared > # filesystems (NFS or clustered filesystem) or for some operating systems > # which use a separate cache for mmap, such as OpenBSD. > mmap_disable = yes" > I've read it 10 times, and I don't know if this should be set to yes or no (probably because my english is not perfect). > My operating system is OpenBSD and I don't share NFS or cluster filesystems. Well, your english is monumentally better than my second language (if you could even say I have one), so good on you... I interpret this sentence as an if statement: if [[ (using NFS||Cluster) == true || (using OS w separate cache for mmap, such as OpenBSD) == true ]]; then setting is required (set to yes/true) fi > > Thank you -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca From marc at perkel.com Sat Oct 6 22:08:12 2012 From: marc at perkel.com (Marc Perkel) Date: Sat, 06 Oct 2012 12:08:12 -0700 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: References: <506D1345.9070406@perkel.com> Message-ID: <5070819C.40104@perkel.com> On 10/4/2012 7:20 AM, Timo Sirainen wrote: > On 4.10.2012, at 7.40, Marc Perkel wrote: > >> I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? > Write a plugin. > I have had some luck using an external script to generate the subscriptions files. From p.heinlein at heinlein-support.de Sun Oct 7 00:32:56 2012 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Sat, 06 Oct 2012 23:32:56 +0200 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files Message-ID: <5070A388.8070205@heinlein-support.de> Several times we already had the problems, that accounts with more the 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if vsize_limit of 750 MB is set. In this case, the lmtpd-process haven't been able to allocate more memory to read/write/update the index-files and crashed (and the index-files become corrupted at the end.) [Please -- don't discuss about the need of INBOXes with 1.7 million (unread) e-mails (don't discuss that with ME. Personally, I agree, that there's NO need for that...).] But: We also noticed accounts with ~ 300.000 e-Mails running out of memory in the same situations. This happends, if the subject is very large (subject or some other header attributes). And: We've been able to reproduce out-of-memory-Problems with just 13.000 e-mails with VERY long subjects (e.g.: network monitoring status informations), even with a vsize_limit of 750 MB (which is already very much). 13.000 e-mails isn't very much. And it's easy to inject several thousands of prepared e-mails. Having many mails for accounts with huge (and broken) index-files slows down the delivery rate VERY much and increases the need for memory and cpu resources and I/O very much. So: This could be used for a very easy to do denial-of-service attac against Dovecot-based mailservers. I don't have a clear solution for that, Dovecot needs the subject information in its index files. But it looks like, it isn't a good idea to put the whole subject into the index. Maybe it's better/necessary to use just the first 50-70 characters for that and to keep the rest away from the index? I think I would prefer that even if that means, that accessing those folders with "special" e-mails will become slower because Dovecot has to get those informations directly from the e-mail. This performance issue is just a problem for the user. But crashing lmtpd-processes and lowering the delivery rate is a *real* problem for the whole IMAP-cluster. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From slitt at troubleshooters.com Sun Oct 7 02:44:24 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Sat, 6 Oct 2012 19:44:24 -0400 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <5070A388.8070205@heinlein-support.de> References: <5070A388.8070205@heinlein-support.de> Message-ID: <20121006194424.47f7f80b@mydesk> On Sat, 06 Oct 2012 23:32:56 +0200, Peer Heinlein said: > > Several times we already had the problems, that accounts with more the > 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if > vsize_limit of 750 MB is set. > > In this case, the lmtpd-process haven't been able to allocate more > memory to read/write/update the index-files and crashed (and the > index-files become corrupted at the end.) > > [Please -- don't discuss about the need of INBOXes with 1.7 million > (unread) e-mails (don't discuss that with ME. Personally, I agree, > that there's NO need for that...).] > > But: We also noticed accounts with ~ 300.000 e-Mails running out of > memory in the same situations. This happends, if the subject is very > large (subject or some other header attributes). > > And: We've been able to reproduce out-of-memory-Problems with just > 13.000 e-mails with VERY long subjects (e.g.: network monitoring > status informations), even with a vsize_limit of 750 MB (which is > already very much). > > 13.000 e-mails isn't very much. And it's easy to inject several > thousands of prepared e-mails. > > Having many mails for accounts with huge (and broken) index-files > slows down the delivery rate VERY much and increases the need for > memory and cpu resources and I/O very much. > > So: This could be used for a very easy to do denial-of-service attac > against Dovecot-based mailservers. > > I don't have a clear solution for that, Dovecot needs the subject > information in its index files. But it looks like, it isn't a good > idea to put the whole subject into the index. Maybe it's > better/necessary to use just the first 50-70 characters for that and > to keep the rest away from the index? > > I think I would prefer that even if that means, that accessing those > folders with "special" e-mails will become slower because Dovecot has > to get those informations directly from the e-mail. > > This performance issue is just a problem for the user. > > But crashing lmtpd-processes and lowering the delivery rate is a > *real* problem for the whole IMAP-cluster. > > Peer While the real solution is being decided, can I avoid this possible DOS attack by using procmail to /dev/null anything with more than a 256 byte subject, before it ever gets to Dovecot IMAP? Thanks SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From mikydevel at yahoo.fr Sun Oct 7 12:36:59 2012 From: mikydevel at yahoo.fr (Mik J) Date: Sun, 7 Oct 2012 10:36:59 +0100 (BST) Subject: [Dovecot] Dovecot configuration and question about IP trusted In-Reply-To: <1349543117.2086.10.camel@worklian> References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349543117.2086.10.camel@worklian> Message-ID: <1349602619.92555.YahooMailNeo@web28801.mail.ir2.yahoo.com> > De?: Bob Miller > Hi Mik, > >> >> Also how do you understand this sentence >> "# Don't use mmap() at all. This is required if you store indexes > to shared >> # filesystems (NFS or clustered filesystem) or for some operating systems >> # which use a separate cache for mmap, such as OpenBSD. >> mmap_disable = yes" >> I've read it 10 times, and I don't know if this should be set to > yes or no (probably because my english is not perfect). >> My operating system is OpenBSD and I don't share NFS or cluster > filesystems. > > Well, your english is monumentally better than my second language (if > you could even say I have one), so good on you... > > I interpret this sentence as an if statement: > > if [[ (using NFS||Cluster) == true || (using OS w separate cache for > mmap, such as OpenBSD) == true ]]; then > ??? setting is required (set to yes/true) > fi Thank you for your answers. Have a nice week end From marc at perkel.com Sun Oct 7 22:47:44 2012 From: marc at perkel.com (Marc Perkel) Date: Sun, 07 Oct 2012 12:47:44 -0700 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? Message-ID: <5071DC60.8090108@perkel.com> Here's what I'm trying to do. I have a spam filtering operation as a front end for other servers. I've created a virtual server for spam storage where the user will be able to log in using squirrelmail/dovecot to review and release their spam. The email is stored in maildir format. Piecing it together I can use squirrelmail to pipe the email into something so that if a use finds a false positive they can hit the "release" button and the message is sent on to the recipient. Squirrelmail sends the message as you would receive it as stored in maildir format. I need to take this format and translate it to send it on to the user. Wondering what is the best way to do that. Something that translated it into bsmtp format would be great. I'm sure thare must be something out there. I just haven't found it yet. Thanks in advance. From robert at schetterer.org Sun Oct 7 23:48:10 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 07 Oct 2012 22:48:10 +0200 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? In-Reply-To: <5071DC60.8090108@perkel.com> References: <5071DC60.8090108@perkel.com> Message-ID: <5071EA8A.3060605@schetterer.org> Am 07.10.2012 21:47, schrieb Marc Perkel: > Here's what I'm trying to do. I have a spam filtering operation as a > front end for other servers. I've created a virtual server for spam > storage where the user will be able to log in using squirrelmail/dovecot > to review and release their spam. The email is stored in maildir format. > > Piecing it together I can use squirrelmail to pipe the email into > something so that if a use finds a false positive they can hit the > "release" button and the message is sent on to the recipient. > Squirrelmail sends the message as you would receive it as stored in > maildir format. > > I need to take this format and translate it to send it on to the user. > > Wondering what is the best way to do that. Something that translated it > into bsmtp format would be great. I'm sure thare must be something out > there. I just haven't found it yet. > > Thanks in advance. > sorry my hard words ,thats enorm complicated the whole idea sounds broken somehow why not reject spam in smtp income level i.e with clamav-milter and sanesecurity antispam signatures and spamass-milter reject all mail tagged over i.e level 15 for the rest ( which will be quite low ) use i.e some quarantaine feature amavis etc if users should manage it ie http://www.maiamailguard.com/maia/wiki/AboutMaia or equal may a good idea and i am quite sure there are some other well done projects out there which doing equal stuff -- Best Regards MfG Robert Schetterer From tss at iki.fi Mon Oct 8 03:11:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 8 Oct 2012 03:11:15 +0300 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <5070A388.8070205@heinlein-support.de> References: <5070A388.8070205@heinlein-support.de> Message-ID: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> On 7.10.2012, at 0.32, Peer Heinlein wrote: > Several times we already had the problems, that accounts with more the > 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if > vsize_limit of 750 MB is set. > > In this case, the lmtpd-process haven't been able to allocate more > memory to read/write/update the index-files and crashed (and the > index-files become corrupted at the end.) I don't think dovecot.index file is much of a problem. With 1M mails it usually only takes something like 8-32 MB of memory depending on what mailbox format is used. dovecot.index.log file doesn't depend on the mailbox size at all. The main problem is dovecot.index.cache file. I've thought about the cache file problems earlier also, but it's a bit difficult to figure out the best solution for it. And since nobody had actually complained about it, I hadn't really done anything about it. Also I hadn't previously thought of LMTP/LDA processes crashing because of it, that's a bigger problem than IMAP process crashing. Although I think you're getting a lot more of "mmap(dovecot.index.cache) failed: Out of memory" errors than crashes for large mailboxes? So, subproblems related to this: 1. Filling out dovecot.index.cache too easily. A rather simple possibility that would catch all the possible ways would be to limit the max. size of a single message's cache entry to X kilobytes (64?). If it becomes larger, it's simply not written to the cache file. 2. Filling out memory too easily. If a long header is wanted to be cached or used for other purposes (e.g. Message-ID), it's still fully read into memory. Add some reasonable limit to max. length of a single header. Can't be too small, because some headers are legitimately pretty long (DKIM and such). Maybe something like 10kB would be safe enough for everyone? 3. If existing dovecot.index.cache is larger than X MB, shrink it first below X. Shrinking could begin with trying to do it the nice way of removing only unneeded data, but if that fails it could forcibly just remove some old messages. The X would have to be related to the process's VSZ limit. 4. Dovecot currently doesn't close index files immediately when mailbox is closed, because it's thinking that IMAP clients might reopen the index soon anyway. Max 3 indexes can be kept open, so 3x already different very large indexes can be too much. I'm not sure if this is actually useful at all. Maybe I should disable it for LMTP, or maybe just remove it completely. The 3. part is what I like changing the least. An alternative solution would be to just not map the entire cache file into memory all at once. The code was actually originally designed to do just that, but munmap()ing + mmap()ing again wasn't very efficient. But for LMTP there's really no need to map the whole file. All it really wants is to read a couple of header records and then append to the file. Maybe it could use an alternative code path that would simply do that instead of mmap()ing anything. It wouldn't solve it for IMAP though. > I don't have a clear solution for that, Dovecot needs the subject > information in its index files. But it looks like, it isn't a good idea > to put the whole subject into the index. Maybe it's better/necessary to > use just the first 50-70 characters for that and to keep the rest away > from the index? 50-70 is way too little. The cached subject gets sent to the IMAP client. I think 200 bytes would be minimum and 1000 would be something I could probably even hardcode. But anyway, subject isn't the only way to trigger this and 1000 bytes is too low for some headers. From tss at iki.fi Mon Oct 8 04:45:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 08 Oct 2012 04:45:14 +0300 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> References: <5070A388.8070205@heinlein-support.de> <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> Message-ID: <1349660714.13571.75.camel@hurina> On Mon, 2012-10-08 at 03:11 +0300, Timo Sirainen wrote: > The 3. part is what I like changing the least. An alternative solution > would be to just not map the entire cache file into memory all at > once. The code was actually originally designed to do just that, but > munmap()ing + mmap()ing again wasn't very efficient. But for LMTP > there's really no need to map the whole file. All it really wants is > to read a couple of header records and then append to the file. Maybe > it could use an alternative code path that would simply do that > instead of mmap()ing anything. It wouldn't solve it for IMAP though. Attached patch changes LMTP/LDA to not mmap() the target mailbox's cache file. I did a few quick tests and it seems to work. I'll probably commit it to 2.1 hg after some more tests. -------------- next part -------------- A non-text attachment was scrubbed... Name: index-saveonly.diff Type: text/x-patch Size: 17995 bytes Desc: not available URL: From stan at hardwarefreak.com Mon Oct 8 09:11:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 08 Oct 2012 01:11:50 -0500 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? In-Reply-To: <5071DC60.8090108@perkel.com> References: <5071DC60.8090108@perkel.com> Message-ID: <50726EA6.4030702@hardwarefreak.com> This request for assistance is a train wreck, with cars strewn everywhere, chaos ensuing, the carnage preventing everyone from being able to see what's actually going on... On 10/7/2012 2:47 PM, Marc Perkel wrote: > Here's what I'm trying to do. I have a spam filtering operation as a > front end for other servers. I've created a virtual server for spam > storage where the user will be able to log in using squirrelmail/dovecot > to review and release their spam. The email is stored in maildir format. So you're trying to somewhat duplicate the functionality of a Barracuda or other AS gateway appliance, with vanilla SM and Dovecot with very little modification. Good luck with this. You're going to need to write a pretty complex shell or perl script to do most of the work, and call it from the SM "release button" routine. > Piecing it together I can use squirrelmail to pipe the email into > something so that if a use finds a false positive they can hit the > "release" button and the message is sent on to the recipient. > Squirrelmail sends the message as you would receive it as stored in > maildir format. SM only sends messages via SMTP submitted to the SMTP relay host specified in the config file, or via dropping to the local MTA via stdin/out. Maildir is a mail storage directory and file format protocol, not a transmission protocol. It's physically impossible to "send" a msg in maildir format, or mbox, or dbox, etc. The format of the SMTP headers and message body is plain text, possibly with MIME encoding. So what you really mean is you need a way to read a maildir message file, scrape the recipient address, strip all of the AS headers you've inserted, drop this 'new' message to the MTA, which sends the message to the recipient. When it arrives it appears never to have been molested by your AS software, with only the proper headers and original body. Then your script needs to check the log for successful delivery (250), then send the proper commands to dovecot to log into the account as the user and delete the message. I can't begin to estimate the amount of coding and testing required here, but it will be high. > I need to take this format and translate it to send it on to the user. Translate it? I've never used maildir, but I can't imagine the on disk message file contents need "translating". See above. > Wondering what is the best way to do that. Something that translated it > into bsmtp format would be great. I'm sure thare must be something out > there. I just haven't found it yet. BSMTP isn't a file format. It's a simple Mail User Agent with some unique capabilities. Given you already have an MTA on the host, why would you need BSMTP to submit or deliver the msg? You simply need to learn the proper commands to submit mail to your local MTA. With Postfix you'd use postdrop or the sendmail compatibility command which use stdin to read the message. My no BS assessment and recommendation are that you're likely in way over your head here (and don't realize it yet), and you need to focus your efforts on locating an integrated FOSS solution, or even a commercial solution, to fit your needs. I seem to understand better than you what is required to pull this off, and I wouldn't touch this project with a 50 ft pole. I simply don't have the coding skills for this. -- Stan From stan at hardwarefreak.com Mon Oct 8 10:44:57 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 08 Oct 2012 02:44:57 -0500 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> References: <5070A388.8070205@heinlein-support.de> <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> Message-ID: <50728479.6070003@hardwarefreak.com> On 10/7/2012 7:11 PM, Timo Sirainen wrote: > I don't think dovecot.index file is much of a problem. With 1M mails it usually only takes something like 8-32 MB of memory depending on what mailbox format is used. dovecot.index.log file doesn't depend on the mailbox size at all. The main problem is dovecot.index.cache file. > > I've thought about the cache file problems earlier also, but it's a bit difficult to figure out the best solution for it. And since nobody had actually complained about it, I hadn't really done anything about it. Also I hadn't previously thought of LMTP/LDA processes crashing because of it, that's a bigger problem than IMAP process crashing. Although I think you're getting a lot more of "mmap(dovecot.index.cache) failed: Out of memory" errors than crashes for large mailboxes? > > So, subproblems related to this: > > 1. Filling out dovecot.index.cache too easily. A rather simple possibility that would catch all the possible ways would be to limit the max. size of a single message's cache entry to X kilobytes (64?). If it becomes larger, it's simply not written to the cache file. > > 2. Filling out memory too easily. If a long header is wanted to be cached or used for other purposes (e.g. Message-ID), it's still fully read into memory. Add some reasonable limit to max. length of a single header. Can't be too small, because some headers are legitimately pretty long (DKIM and such). Maybe something like 10kB would be safe enough for everyone? > > 3. If existing dovecot.index.cache is larger than X MB, shrink it first below X. Shrinking could begin with trying to do it the nice way of removing only unneeded data, but if that fails it could forcibly just remove some old messages. The X would have to be related to the process's VSZ limit. > > 4. Dovecot currently doesn't close index files immediately when mailbox is closed, because it's thinking that IMAP clients might reopen the index soon anyway. Max 3 indexes can be kept open, so 3x already different very large indexes can be too much. I'm not sure if this is actually useful at all. Maybe I should disable it for LMTP, or maybe just remove it completely. > > The 3. part is what I like changing the least. An alternative solution would be to just not map the entire cache file into memory all at once. The code was actually originally designed to do just that, but munmap()ing + mmap()ing again wasn't very efficient. But for LMTP there's really no need to map the whole file. All it really wants is to read a couple of header records and then append to the file. Maybe it could use an alternative code path that would simply do that instead of mmap()ing anything. It wouldn't solve it for IMAP though. > 50-70 is way too little. The cached subject gets sent to the IMAP client. I think 200 bytes would be minimum and 1000 would be something I could probably even hardcode. But anyway, subject isn't the only way to trigger this and 1000 bytes is too low for some headers. Nearly all mail servers have two resources of interest here in great excess: CPU cycles, and cache/RAM b/w, due to multicore CPUs and 2-4 memory channels per socket. The two bottlenecks are IO bandwidth/latency, and, for many, RAM capacity. So let's take advantage of both the strengths and weaknesses of our hardware to possibly address the above issue. What happens if we insert a subroutine to compress/decompress each field in the cache array files individually, in real time? You should still be able to mmap the files. The individual array fields and total cache file sizes would be much smaller on disk and in memory. Any cache file contents mapped to memory, that aren't currently being used, are stored compressed in memory, directly addressing the problem in this thread. When a field is needed we decompress it on the fly after reading it from memory. This should be very fast as the fields are relatively small. When it's written out we compress on the fly. With each field stored compressed on disk, not only is file size decreased, but more importantly, each read/write moves more data per physical IO. So not only are increasing storage capacity, we're also decreasing IOPS. It would be preferable to do this de/compression in kernel rather than user space, but I don't think that's a real option. However, libz and libbz2 are pretty fast and small, and the code easily fits in CPU cache. Combined with the massive L1/L2/L3 and RAM b/w of modern systems, execution in user space should still be very fast, and not noticeably degrade performance. I'm not a programmer, so I have no idea if this is even plausible, or possible. But if it is, it seems worth exploring, as it would seem to benefit Dovecot performance in multiples areas, and possibly solve this, and other current/future memory capacity and/or performance related problems. -- Stan From benedetto.vassallo at unipa.it Mon Oct 8 15:44:05 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Mon, 08 Oct 2012 14:44:05 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <506DA526.4020606@Media-Brokers.com> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <506DA526.4020606@Media-Brokers.com> Message-ID: <20121008144405.21350fs5aq2vwl91@webmail.unipa.it> Def. Quota Charles Marcus : > On 2012-10-04 9:00 AM, Benedetto Vassallo > wrote: >> Hello list, >> Excuse me for my poor english. >> I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. > > If you are going to update, why ohy why update to an outdated version? > > Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... > Hello, I upgraded to 2.1.10 but still dont't have hardlinks working. May I have something else to do to make them working? Thanks. -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From tibby at tibby.hu Mon Oct 8 16:49:33 2012 From: tibby at tibby.hu (Tibby) Date: Mon, 8 Oct 2012 15:49:33 +0200 (CEST) Subject: [Dovecot] another mysql quota problem Message-ID: <1182860249.2592.1349704173895.JavaMail.root@tibby.hu> Hello! I am using dovecot 1.2.15 on Debian 6 I have Postfix configured with mysql, and also dovecot is auth-ing form mysql users table. The users table has username password quota stored. When I set qouta to whatever number, it doesnt even show in dovecot. telnet localhost 143 a login username at domain.tld password a getquotaroot inbox * QUOTAROOT "INBOX" a OK Getquotaroot completed. Basicly it doesn't show any kind of quota. in the /etc/dovecot/dovecot-sql.conf: driver = mysql connect = host=127.0.0.1 dbname=mail user=USERNAME password=PASSWORD default_pass_scheme = CRYPT password_query = SELECT email as user, password FROM users WHERE email='%u'; user_query = SELECT CONCAT(('/home/vmail/'), SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1),'/') AS home, 5000 AS uid, 5000 AS gid, CONCAT('maildir:storage=', floor(quota/1024)) AS quota FROM users WHERE email='%u'; Why it doesn't get the quota out from the mail DB's users table? From roundcube222 at alaadin.org Mon Oct 8 18:12:58 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Mon, 08 Oct 2012 18:12:58 +0300 Subject: [Dovecot] Dovecot Hangs while mutile users download mail for same account using pop3 Message-ID: <793760c2702e89acc526a66c0b543293@Coptics.org> Hello, I have a weird problem in dovecot, Dovecot Hangs while multiple users download mail for same account using pop3 Three persons use 1 same email , and three of them use outlook express to check That specific mail .. Also some times one of the three users check the mail for this Account using imap (squirrel mail) And here comes the problem. suddenly all the three users are unable to check the mail And when the outlook express connect to the pop3 server, they are unable to disconnect And the outlook express keep asking "the server is not responding for 60 mins would you like to wait ? i checked /var/log/maillog, i can see the following errors dovecot: pop3(sales): Disconnected: Storage error during logout. to=, orig_to=, relay=local, delay=357, delays=338/0.01/0/19, dsn=4.2.0, status=deferred (cannot update mailbox /var/mail/sales for user sales. unable to lock for exclusive access: Resource temporarily unavailable) When i used lsof | grep sales pop3 4278 sales 10u REG 8,3 22897673 12615705 /var/spool/mail/sales i went to /home/sales/mail/.imap/INBOX , but i didnot find any lock files -rw------- 1 sales sales 944 Oct 7 13:35 dovecot.index -rw------- 1 sales sales 49152 Oct 7 16:57 dovecot.index.cache -rw------- 1 sales sales 14044 Oct 7 16:57 dovecot.index.log 1- So what is exactly the problem? is the problem that three users are checking the same exact mail ? 2- is the problem that 2 users checking mail using pop3 and the third one checking it using imap insame time ? 3- is dovecot locking the inbox file while one of the users getting the mail ? 4- What is the cause of the problem ? Please help Regards Robert JR From marc at perkel.com Mon Oct 8 19:23:27 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 09:23:27 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? Message-ID: <5072FDFF.8050402@perkel.com> I'm trying to grasp the namespace and prefix stuff and looking for a good tutorial page that explains what it is and how to use it. Thanks in advance From robert at schetterer.org Mon Oct 8 20:59:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 19:59:34 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5072FDFF.8050402@perkel.com> References: <5072FDFF.8050402@perkel.com> Message-ID: <50731486.4080003@schetterer.org> Am 08.10.2012 18:23, schrieb Marc Perkel: > I'm trying to grasp the namespace and prefix stuff and looking for a > good tutorial page that explains what it is and how to use it. > > Thanks in advance > look http://wiki2.dovecot.org/Namespaces -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:09:00 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:09:00 -0700 Subject: [Dovecot] Master User Question Message-ID: <507316BC.1050008@perkel.com> OK - I'm trying to do weird stuff so rather not answer why I'm doing this. Trying to log in using a master user: user at example.com*master at master.com When debugging the master authenticates - but then it tries to authenticate user at example.com and it's not found. And - it is true that the user doesn't actually exist. What I want to do is allow it to log in without checking if the user exists, just on the credentials of the master. I'm not quite understanding what the login attempt for user at example.com is trying to do. Is it looking for more necessary information? Hope I asked this clearly enough. Thanks in advance. From marc at perkel.com Mon Oct 8 21:10:21 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:10:21 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <50731486.4080003@schetterer.org> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> Message-ID: <5073170D.2040508@perkel.com> On 10/8/2012 10:59 AM, Robert Schetterer wrote: > Am 08.10.2012 18:23, schrieb Marc Perkel: >> I'm trying to grasp the namespace and prefix stuff and looking for a >> good tutorial page that explains what it is and how to use it. >> >> Thanks in advance >> > look > > http://wiki2.dovecot.org/Namespaces > I've read that but it doesn't tell me what a prefix is or what a namespace is. I'm having a hard time grasping the overall concept. From robert at schetterer.org Mon Oct 8 21:26:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:26:14 +0200 Subject: [Dovecot] Master User Question In-Reply-To: <507316BC.1050008@perkel.com> References: <507316BC.1050008@perkel.com> Message-ID: <50731AC6.4060707@schetterer.org> Am 08.10.2012 20:09, schrieb Marc Perkel: > OK - I'm trying to do weird stuff so rather not answer why I'm doing this. > > Trying to log in using a master user: > > user at example.com*master at master.com > > When debugging the master authenticates - but then it tries to > authenticate user at example.com and it's not found. And - it is true that > the user doesn't actually exist. > > What I want to do is allow it to log in without checking if the user > exists, just on the credentials of the master. i dont understand why login should work with masteruser to user , when user does not exist, would you like to get the user created on the fly by masteruser login etc ? > > I'm not quite understanding what the login attempt for user at example.com > is trying to do. Is it looking for more necessary information? > > Hope I asked this clearly enough. Thanks in advance. > > you may manipulate your master query in some magic way http://wiki2.dovecot.org/Authentication/MasterUsers -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Oct 8 21:31:52 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:31:52 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5073170D.2040508@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> Message-ID: <50731C18.5060009@schetterer.org> Am 08.10.2012 20:10, schrieb Marc Perkel: > > On 10/8/2012 10:59 AM, Robert Schetterer wrote: >> Am 08.10.2012 18:23, schrieb Marc Perkel: >>> I'm trying to grasp the namespace and prefix stuff and looking for a >>> good tutorial page that explains what it is and how to use it. >>> >>> Thanks in advance >>> >> look >> >> http://wiki2.dovecot.org/Namespaces >> > > I've read that but it doesn't tell me what a prefix is or what a > namespace is. I'm having a hard time grasping the overall concept. > lookink to this http://www.ietf.org/rfc/rfc2342.txt ? -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:32:51 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:32:51 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731AC6.4060707@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> Message-ID: <50731C53.2050309@perkel.com> On 10/8/2012 11:26 AM, Robert Schetterer wrote: > Am 08.10.2012 20:09, schrieb Marc Perkel: >> OK - I'm trying to do weird stuff so rather not answer why I'm doing this. >> >> Trying to log in using a master user: >> >> user at example.com*master at master.com >> >> When debugging the master authenticates - but then it tries to >> authenticate user at example.com and it's not found. And - it is true that >> the user doesn't actually exist. >> >> What I want to do is allow it to log in without checking if the user >> exists, just on the credentials of the master. > i dont understand why login should work with masteruser to user , when > user does not exist, would you like to get the user created on the fly > by masteruser login etc ? The directory structure for user at example.com exists. Just not in the regular mysql database. If I create a fake passdb system it works: passdb { driver = static args = nopassword=y } However - it leaves the system wide open for anyone. Basically - only master users are going to log in and manage stored spam for regular users. The master user will be able to spoof being the regular user to review stored spam. From slusarz at curecanti.org Mon Oct 8 21:28:18 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 08 Oct 2012 12:28:18 -0600 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5073170D.2040508@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> Message-ID: <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> Quoting Marc Perkel : > On 10/8/2012 10:59 AM, Robert Schetterer wrote: >> Am 08.10.2012 18:23, schrieb Marc Perkel: >>> I'm trying to grasp the namespace and prefix stuff and looking for a >>> good tutorial page that explains what it is and how to use it. >>> >>> Thanks in advance >>> >> look >> >> http://wiki2.dovecot.org/Namespaces >> > > I've read that but it doesn't tell me what a prefix is or what a > namespace is. I'm having a hard time grasping the overall concept. Try reading the defining RFC document itself: http://tools.ietf.org/html/rfc2342 michael From marc at perkel.com Mon Oct 8 21:42:02 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:42:02 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> Message-ID: <50731E7A.9020001@perkel.com> On 10/8/2012 11:28 AM, Michael M Slusarz wrote: > Quoting Marc Perkel : > >> On 10/8/2012 10:59 AM, Robert Schetterer wrote: >>> Am 08.10.2012 18:23, schrieb Marc Perkel: >>>> I'm trying to grasp the namespace and prefix stuff and looking for a >>>> good tutorial page that explains what it is and how to use it. >>>> >>>> Thanks in advance >>>> >>> look >>> >>> http://wiki2.dovecot.org/Namespaces >>> >> >> I've read that but it doesn't tell me what a prefix is or what a >> namespace is. I'm having a hard time grasping the overall concept. > > Try reading the defining RFC document itself: > > http://tools.ietf.org/html/rfc2342 > > michael > I went there and maybe I'm just stupid. I don't understand what a prefix is. I'm trying to grasp the whole process. From robert at schetterer.org Mon Oct 8 21:43:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:43:20 +0200 Subject: [Dovecot] Master User Question In-Reply-To: <50731C53.2050309@perkel.com> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> Message-ID: <50731EC8.6010101@schetterer.org> Am 08.10.2012 20:32, schrieb Marc Perkel: > > On 10/8/2012 11:26 AM, Robert Schetterer wrote: >> Am 08.10.2012 20:09, schrieb Marc Perkel: >>> OK - I'm trying to do weird stuff so rather not answer why I'm doing >>> this. >>> >>> Trying to log in using a master user: >>> >>> user at example.com*master at master.com >>> >>> When debugging the master authenticates - but then it tries to >>> authenticate user at example.com and it's not found. And - it is true that >>> the user doesn't actually exist. >>> >>> What I want to do is allow it to log in without checking if the user >>> exists, just on the credentials of the master. >> i dont understand why login should work with masteruser to user , when >> user does not exist, would you like to get the user created on the fly >> by masteruser login etc ? > > The directory structure for user at example.com exists. Just not in the > regular mysql database. If I create a fake passdb system it works: > > passdb { > driver = static > args = nopassword=y > } > > However - it leaves the system wide open for anyone. Basically - only > master users are going to log in and manage stored spam for regular > users. The master user will be able to spoof being the regular user to > review stored spam. > > whats the problem in simply create the user mailbox? i.e lda etc may do it on the fly be recieving a mail http://wiki2.dovecot.org/LDA -m : Destination mailbox (default is INBOX). If the mailbox doesn't exist, it will not be created (unless the lda_mailbox_autocreate setting is set to yes). If message couldn't be saved to the mailbox for any reason, it's delivered to INBOX instead. -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Oct 8 21:49:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:49:33 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <50731E7A.9020001@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> <50731E7A.9020001@perkel.com> Message-ID: <5073203D.7060801@schetterer.org> Am 08.10.2012 20:42, schrieb Marc Perkel: > > On 10/8/2012 11:28 AM, Michael M Slusarz wrote: >> Quoting Marc Perkel : >> >>> On 10/8/2012 10:59 AM, Robert Schetterer wrote: >>>> Am 08.10.2012 18:23, schrieb Marc Perkel: >>>>> I'm trying to grasp the namespace and prefix stuff and looking for a >>>>> good tutorial page that explains what it is and how to use it. >>>>> >>>>> Thanks in advance >>>>> >>>> look >>>> >>>> http://wiki2.dovecot.org/Namespaces >>>> >>> >>> I've read that but it doesn't tell me what a prefix is or what a >>> namespace is. I'm having a hard time grasping the overall concept. >> >> Try reading the defining RFC document itself: >> >> http://tools.ietf.org/html/rfc2342 >> >> michael >> > > I went there and maybe I'm just stupid. I don't understand what a prefix > is. I'm trying to grasp the whole process. > > try http://wiki.dovecot.org/Design/Storage/MailNamespace -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:52:45 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:52:45 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731EC8.6010101@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> <50731EC8.6010101@schetterer.org> Message-ID: <507320FD.2000201@perkel.com> On 10/8/2012 11:43 AM, Robert Schetterer wrote: > Am 08.10.2012 20:32, schrieb Marc Perkel: >> On 10/8/2012 11:26 AM, Robert Schetterer wrote: >>> Am 08.10.2012 20:09, schrieb Marc Perkel: >>>> OK - I'm trying to do weird stuff so rather not answer why I'm doing >>>> this. >>>> >>>> Trying to log in using a master user: >>>> >>>> user at example.com*master at master.com >>>> >>>> When debugging the master authenticates - but then it tries to >>>> authenticate user at example.com and it's not found. And - it is true that >>>> the user doesn't actually exist. >>>> >>>> What I want to do is allow it to log in without checking if the user >>>> exists, just on the credentials of the master. >>> i dont understand why login should work with masteruser to user , when >>> user does not exist, would you like to get the user created on the fly >>> by masteruser login etc ? >> The directory structure for user at example.com exists. Just not in the >> regular mysql database. If I create a fake passdb system it works: >> >> passdb { >> driver = static >> args = nopassword=y >> } >> >> However - it leaves the system wide open for anyone. Basically - only >> master users are going to log in and manage stored spam for regular >> users. The master user will be able to spoof being the regular user to >> review stored spam. >> >> > whats the problem in simply create the user mailbox? > i.e lda etc may do it on the fly be recieving a mail > > http://wiki2.dovecot.org/LDA > -m : Destination mailbox (default is INBOX). If the mailbox > doesn't exist, it will not be created (unless the lda_mailbox_autocreate > setting is set to yes). If message couldn't be saved to the mailbox for > any reason, it's delivered to INBOX instead. Maybe there's some way to have more than one mail_location? I have this - and it's confusing: maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs the /fakedir directory is externally built and it has symlinks pointing onto the /email directory where the email is stored. If I could set up a passdb and specify a different mail_location that might solve my problem, at least one of them. So close - yet so far away. From marc at perkel.com Mon Oct 8 22:07:28 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 12:07:28 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731EC8.6010101@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> <50731EC8.6010101@schetterer.org> Message-ID: <50732470.300@perkel.com> Making some progress = this seems to work. passdb { driver = static args = mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } From calestyo at scientia.net Tue Oct 9 00:37:19 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 08 Oct 2012 23:37:19 +0200 Subject: [Dovecot] some questions on dovecot or rather a mail system setup Message-ID: <1349732239.3344.62.camel@fermat.scientia.net> Hi folks. Perhaps you find some time to look into this,... if you think I should better direct this do some dovecot mailing list,.. just tell :) I'm trying to plan my mail system and would have some questions. The overall idea is about the following: - There is a (internet) server, which is the MTA (which will be postfix) and imap and/or pop3 server (which shall be, guess, dovecot!). - Any spam filtering, virus-scanning, mail filtering happens on that server. - I have not yet decided on whether to use maildrop for this or Sieve. Maildrop is nice, but has one major deficiency which I don't know how to work around. - A few clients (that means I won't serve 100 of users) connect to that server via imap and should see all mails, etc. already in some fancy sorted hierarchy (that means filtered into different directories). So far nothing complicated. But now... - Apart from spam, I never delete mail; and because I'm subscribed to many lists, I get a lot of mail. - Storage on my server is limited and it's located somewhere at my ISP, so I generally do not trust it with respect to safety... For both reasons, I want the canonical archive of all mail to be at home at some local server. - The local server should also be an imap server, so that I can access the archive from may computers at home. - The local server won't be available from the internet. - The local archive should have the same folder hierarchy as the internet server (I'd prefer not to filter twice). Mail Flow: - I generally want to have _all_ mail (which is not sorted out because of being spam) to be archived at the local server. - But(!) I want to selectively keep (in addition) mail at the internet server. For example I may want to select the folder that contains all mail form some friend to be kept online completely. But I may want to decide that mailinglists keep only the last 10 days and/or 1000 messages of mail. - The idea is, that the local server regularly (when it is online/running) catches new mail from the internet server... and stores it in the archive. - So apart from new mail that has not yet been read, that local archive contains always all mails that are also on the internet server... the later may contain (for specific directories) the same, or just parts of. - The MUAs will then have two imap accounts, one to the internet server and one to the local archive,... each one being usable, depending on where I am. 1) This is where my first problem arises: How can I implement that mail flow, especially: - How do I secure that all mail is read from the internet server (i.e. that nothing is "forgotten"? - How do I make sure that no mails are retrieved twice (or more)? A problem which I often had with pop, when the mail client crashed during sync? - Further it must be secured, that when I delete something on the internet server, it is NOT deleted on the local server (on the next mail-fetching).... this is why I don't use the word "sync". a) One stupid solution would be, that I duplicate all mail on the online server,... one part is for staying online, one part is for being fetched to the local archive. As soon as it was fetched... that copy gets removed (always). That solution would give a clean and secured separation of both? b) I don't think offlineimap or any other caching-like solution is the right thing... especially as one must always fear that such a cache may be accidentally wiped. Are there better solutions than (a)? 2) Problem would be already a refinement of a working solution for (1) (but obviously not when using (1).(a) ). When e.g. reply to or forward a mail using the online server,... and that mail had already been fetched,... can I make the flag synced? 3) Is dovecot suitable for the local server? - Ideally of course, I would use dovecot there, too... because that would mean one piece of software less to understand. - I couldn't sue maildir locally, because I loose just to much space to the block fragmentation. - I'd prefer not to use dbox (the thing that the indices are crucial scares me a bit off). a) When using mbox... is dovecot able to manage a really big folder hierarchy that basically ever keeps growing... with easily several 100k mails per folder... and that is in total already over 100GB? - I would prefer to have fast full text search. Does dovecot provide this? I was looking into database backed mail systems (again,... just for the local archive)... namely dbmail and archiveopteryx (are there other open source solutions?)... Not sure which of the two... or whether it's a good idea at all. I remember some dovecot wiki page that showed a comparison which said that both do not perfectly implement imap. Any suggestions with respect to that? Or is there even some SQL backend planned for dovecot? Thanks a lot, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From robert at schetterer.org Tue Oct 9 09:51:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 09 Oct 2012 08:51:18 +0200 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <1349732239.3344.62.camel@fermat.scientia.net> References: <1349732239.3344.62.camel@fermat.scientia.net> Message-ID: <5073C966.6030609@schetterer.org> Am 08.10.2012 23:37, schrieb Christoph Anton Mitterer: > > > - Apart from spam, I never delete mail; and because I'm subscribed to > many lists, I get a lot of mail. > > - Storage on my server is limited and it's located somewhere at my ISP, > so I generally do not trust it with respect to safety... > For both reasons, I want the canonical archive of all mail to be at home > at some local server. sorry you questioned very complex, try to ask more simple there are many tools which may help you bcc_copy with postfix imapsync rsync dsync getmail you may use filters too like sieve, maildrop , procmail etc at the end that should solve nearly all what you might goal its not that much a dovecot question, it more depends if you find that general layout which fits best to your ideas however there is no magical imap/pop3 server more flexibel to configure then dovecot, if your ideas dont work with it, your ideas are broken -- Best Regards MfG Robert Schetterer From stan at hardwarefreak.com Tue Oct 9 10:57:43 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 09 Oct 2012 02:57:43 -0500 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <1349732239.3344.62.camel@fermat.scientia.net> References: <1349732239.3344.62.camel@fermat.scientia.net> Message-ID: <5073D8F7.1060802@hardwarefreak.com> On 10/8/2012 4:37 PM, Christoph Anton Mitterer wrote: The proper way to accomplish your goals, or at least the big ones. > - I generally want to have _all_ mail (which is not sorted out because > of being spam) to be archived at the local server. http://www.postfix.org/postconf.5.html#always_bcc > - But(!) I want to selectively keep (in addition) mail at the internet > server. > For example I may want to select the folder that contains all mail form > some friend to be kept online completely. See above. > But I may want to decide that mailinglists keep only the last 10 days > and/or 1000 messages of mail. http://wiki2.dovecot.org/Plugins/Expire Does age based deletion, but not folder message count based. You must use your MUA, TBird, for the latter. It's far easier to configure this in TBird than in Dovecot config files. You seem like the type who wants flexibility so you can change things often, so use TBird to be happy here. > - The idea is, that the local server regularly (when it is > online/running) catches new mail from the internet server... and stores > it in the archive. This is not an option. The system must be up and connected to the internet 24x7x365. It must have an MX record associated and a valid domain, or a VPN tunnel and entries in both systems hosts files, along with a Postfix transport table, and other tweaks. http://www.postfix.org/transport.5.html If you refuse to run this "local server" 24x7x365 then you will have to use a fetchmail based solution, which will not work well, and whose configuration will prompt you to kill yourself. I cannot help you with any of that. > - So apart from new mail that has not yet been read, that local archive > contains always all mails that are also on the internet server... the > later may contain (for specific directories) the same, or just parts of. No. Mail arriving at the colo/VPS host is immediately sent to the always_bcc address, an address and mailbox on your home server. You will create a duplicate IMAP folder structure on the home server by hand in your MUA. Once this is completed you will write individual user sieve scripts that sort the mail into folder just as it is sorted on the colo/VPS server. Basically, home server Dovecot IMAP config is identical in structure to colo/VPS setup, only the mailbox account names differ. Folder tree, folders, sieve scripts identical, retention policy different. > - The MUAs will then have two imap accounts, one to the internet server > and one to the local archive,... each one being usable, depending on > where I am. Yep. > > > 1) This is where my first problem arises: > How can I implement that mail flow, especially: > - How do I secure that all mail is read from the internet server (i.e. > that nothing is "forgotten"? Done: always_bcc > - How do I make sure that no mails are retrieved twice (or more)? A > problem which I often had with pop, when the mail client crashed during > sync? Done: always_bcc > - Further it must be secured, that when I delete something on the > internet server, it is NOT deleted on the local server (on the next > mail-fetching).... this is why I don't use the word "sync". Done: always_bcc > a) One stupid solution would be, that I duplicate all mail on the online > server,... one part is for staying online, one part is for being fetched > to the local archive. Done: always_bcc And yes that is stupid. > As soon as it was fetched... that copy gets removed (always). > That solution would give a clean and secured separation of both? > b) I don't think offlineimap or any other caching-like solution is the > right thing... especially as one must always fear that such a cache may > be accidentally wiped. > > Are there better solutions than (a)? Yes. Already done: always_bcc > 2) Problem would be already a refinement of a working solution for (1) > (but obviously not when using (1).(a) ). > When e.g. reply to or forward a mail using the online server,... and > that mail had already been fetched,... can I make the flag synced? No. Your stated goal is that the local server is a mail archive put into service due to limited space on your colo/VPS server. An archive is an archive, not a secondary online server. It should only be accessed, read only, when you want to search and read an old message. And in fact, since this is an archive, you should implement the zlib plugin with dbox so all this archived mail is compressed in real time. Make up your mind. You can't have it both ways. I hear the iPhone5 can do anything automatically, no setup. Get one of those, problem solved. ;) > 3) Is dovecot suitable for the local server? Yes. Probably more than any other IMAP server. > - I couldn't sue maildir locally, because I loose just to much space to > the block fragmentation. Maildir causes the least filesystem fragmentation. You must be thinking of mbox, which causes heavy fragmentation due to constant appends past EOF. As I said you need dbox. One email per file, similar to maildir, but better integration and performance with Dovecot. > - I'd prefer not to use dbox (the thing that the indices are crucial > scares me a bit off). Are you designing/building this home server to be unreliable? Does it crash often? If so fix that problem and dbox is fine. If can't make it reliable use maildir which has expendable indexes. > a) When using mbox... is dovecot able to manage a really big folder > hierarchy that basically ever keeps growing... with easily several 100k > mails per folder... and that is in total already over 100GB? You have 100K emails in a single Dovecot mbox file? Or are you talking about an IMAP folder in TB that has no email in it, but many more IMAP folders whose combined email total is 100K? If you're worried about dbox index corruption, then you should be far more worried about mbox file corruption. With mbox files that large I'm surprised you've not hit it already. This would suggest that system is pretty stable. > - I would prefer to have fast full text search. Does dovecot provide > this? Yes. The problem with speed is two fold: 1. You must FTS often to keep the search indexes up to date. Wait a week between searches, after many new emails have been added to the IMAP folder, and your search crawls, as the file contents must be reindexed before the search starts. So you need to have a cron'd script that searches daily to keep the indexes up to date. 2. The mailbox file formats that best avoid fragmentation also have the slowest FTS times as the OS much open every file, 100K of them. If you use mbox or mdbox, you have far fewer files to open. mbox has the fastest FTS times of any format when indexes aren't fully up to date. It's also the fastest when updating the indexes. Your home server probably has a single SATA disk. mbox wins hands down for FTS due to very low IOPS load on the disk. The downside here is lack of good compression support--once you compress an mbox file you can't add new mail to it. This is where mdbox with compression comes in handy. With you 100K emails declaration, I think you're best served by mdbox with zlib compression. > I was looking into database backed mail systems (again,... just for the > local archive)... namely dbmail and archiveopteryx (are there other open > source solutions?)... > Not sure which of the two... or whether it's a good idea at all. > I remember some dovecot wiki page that showed a comparison which said > that both do not perfectly implement imap. > > Any suggestions with respect to that? If you're worried about fragmentation, or performance, I'd steer clear of a database driven mail store. Please, please, do not reply to each of my points here, and do not make this thread 100 replies. I'm not here to hold your hand. I don't have the time (nor patience) to engage in these lengthy emails. I gave you the architectural overview to build the correct solution to your problem. It's up to you to choose to use it or not, and if so, to do your own homework and self education, asking here only if something is unclear to you. In closing, you need real time bcc delivery which solves a ton of your mentioned problems. I'm not open to debating the merits of this. If you're not willing to meet the requirements for always_bcc, and you're determined to power the home server down most of the time, then you need assistance from someone else, as I simply have never used fetchmail, period, and have no idea if it can meet your needs. My guess is no, simply because, AFAIK, it doesn't work with LDA, which means you can't use sieve scripts and Dovecot's automatic sorting and indexing. Good luck. -- Stan From stan at hardwarefreak.com Tue Oct 9 11:02:46 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 09 Oct 2012 03:02:46 -0500 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <5073D8F7.1060802@hardwarefreak.com> References: <1349732239.3344.62.camel@fermat.scientia.net> <5073D8F7.1060802@hardwarefreak.com> Message-ID: <5073DA26.5060809@hardwarefreak.com> On 10/9/2012 2:57 AM, Stan Hoeppner wrote: > http://www.postfix.org/postconf.5.html#always_bcc Correction. In your case you'll need to use: http://www.postfix.org/postconf.5.html#recipient_bcc_maps Because you said you only want to archive email for some users, not simply all mail received by the colo/VPS server. -- Stan From wamp at promax.media.pl Tue Oct 9 14:36:00 2012 From: wamp at promax.media.pl (wamp at promax.media.pl) Date: Tue, 9 Oct 2012 13:36:00 +0200 Subject: [Dovecot] Quota - usage counting. Message-ID: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> Hello, I use dovecot 1.2 version with postfix virtual users and mysql. All information about quota for every user is in mysql table. How dovecot compare if quota in database is over quota in /var/vmail/exampleuser directory ? It uses something like du command ? regards, Wamp From tibby at tibby.hu Tue Oct 9 14:57:45 2012 From: tibby at tibby.hu (Tibby) Date: Tue, 9 Oct 2012 13:57:45 +0200 (CEST) Subject: [Dovecot] Quota - usage counting. In-Reply-To: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> References: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> Message-ID: <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> Do you actually have a Guide? How did you set up quota form MySql? I'm having an issu getting it working. Can you share your dovecot.conf and dovecot-sql.conf? What's the user_query in your dovecot-sql.conf ? Thank you! ----- Original Message ----- From: wamp at promax.media.pl To: dovecot at dovecot.org Subject: [Dovecot] Quota - usage counting. Hello, I use dovecot 1.2 version with postfix virtual users and mysql. All information about quota for every user is in mysql table. How dovecot compare if quota in database is over quota in /var/vmail/exampleuser directory ? It uses something like du command ? regards, Wamp From wamp at promax.media.pl Tue Oct 9 16:12:41 2012 From: wamp at promax.media.pl (wamp at promax.media.pl) Date: Tue, 9 Oct 2012 15:12:41 +0200 Subject: [Dovecot] Quota - usage counting. In-Reply-To: <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> References: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> Message-ID: <7066b6528ee587362d87aa76ba75e7f4.squirrel@poczta.promax.media.pl> Hi > Do you actually have a Guide? No, I read some docs like http://www.serverubuntu.it/postfix-dovecot-guide > How did you set up quota form MySql? > I'm having an issu getting it working. > Can you share your dovecot.conf and dovecot-sql.conf? > > What's the user_query in your dovecot-sql.conf ? I can't make it working - so need information about general idea how this values should be compared. Where is info about actual size of maildir. regards, > > Thank you! > > ----- Original Message ----- > From: wamp at promax.media.pl > To: dovecot at dovecot.org > Subject: [Dovecot] Quota - usage counting. > > Hello, > > I use dovecot 1.2 version with postfix virtual users and mysql. All > information about quota for every user is in mysql table. > How dovecot compare if quota in database is over quota in > /var/vmail/exampleuser directory ? > > It uses something like du command ? > > > regards, > Wamp > > > > From fxmulder at gmail.com Tue Oct 9 21:41:08 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 12:41:08 -0600 Subject: [Dovecot] LDAP encryption Message-ID: We have an LDAP server that contains AES encrypted passwords. So far I've been able to use this by adding a passdb module that encrypts the user's password prior to ldap comparison. Now I am looking at supporting client-side encrypted passwords. To do this I need to decrypt the password returned by LDAP. Is there a way to insert a module to do this decryption between ldap returning and the auth mechanism? From fxmulder at gmail.com Tue Oct 9 22:16:15 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 13:16:15 -0600 Subject: [Dovecot] LDAP encryption In-Reply-To: <5074750D.4070302@bitrate.net> References: <5074750D.4070302@bitrate.net> Message-ID: I don't think I understand. Right now the problem is the password retrieved from LDAP cannot be hashed to compare against what the user sent because it is encrypted. I have to perform my AES decryption before it can be hashed and compared. On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: > On 2012.10.09 14.41, James Devine wrote: > >> We have an LDAP server that contains AES encrypted passwords. So far I've >> been able to use this by adding a passdb module that encrypts the user's >> password prior to ldap comparison. Now I am looking at supporting >> client-side encrypted passwords. To do this I need to decrypt the >> password >> returned by LDAP. Is there a way to insert a module to do this decryption >> between ldap returning and the auth mechanism? >> >> that would be unwise, generally speaking. as a rule of thumb, in terms > of security fundamentals, only the rootdn [or equiv] should be able to read > the values in an ldap entry's password attribute. certainly the service > account used by dovecot should not. > > in the context of ldap, authentication should be accomplished by binding > as the user, not by retrieving attribute values and performing string > comparisons. among other things, this decouples the two components and > allows applications [e.g. dovecot] to be unconcerned with whatever password > hashing scheme the directory server might be using. > > -ben > From fxmulder at gmail.com Tue Oct 9 22:24:21 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 13:24:21 -0600 Subject: [Dovecot] LDAP encryption In-Reply-To: References: <5074750D.4070302@bitrate.net> Message-ID: Here is an example of the problem: Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: password(user at domain.tld,192.168.160.49): Generating NTLM from user 'user at domain.tld@', password 'IfBG6G3jykirE5r5vienC4w==' Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: password(user at domain.tld,192.168.160.49): Credentials: f124dc24328ed3d90db035f0d5284636 The listed password is a base64 representation of its encrypted form which I need to somehow decrypt between the time LDAP returns it and these credentials are generated. On Tue, Oct 9, 2012 at 1:16 PM, James Devine wrote: > I don't think I understand. Right now the problem is the password > retrieved from LDAP cannot be hashed to compare against what the user sent > because it is encrypted. I have to perform my AES decryption before it can > be hashed and compared. > > > On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: > >> On 2012.10.09 14.41, James Devine wrote: >> >>> We have an LDAP server that contains AES encrypted passwords. So far >>> I've >>> been able to use this by adding a passdb module that encrypts the user's >>> password prior to ldap comparison. Now I am looking at supporting >>> client-side encrypted passwords. To do this I need to decrypt the >>> password >>> returned by LDAP. Is there a way to insert a module to do this >>> decryption >>> between ldap returning and the auth mechanism? >>> >>> that would be unwise, generally speaking. as a rule of thumb, in terms >> of security fundamentals, only the rootdn [or equiv] should be able to read >> the values in an ldap entry's password attribute. certainly the service >> account used by dovecot should not. >> >> in the context of ldap, authentication should be accomplished by binding >> as the user, not by retrieving attribute values and performing string >> comparisons. among other things, this decouples the two components and >> allows applications [e.g. dovecot] to be unconcerned with whatever password >> hashing scheme the directory server might be using. >> >> -ben >> > > From tss at iki.fi Tue Oct 9 22:30:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 9 Oct 2012 22:30:19 +0300 Subject: [Dovecot] LDAP encryption In-Reply-To: References: <5074750D.4070302@bitrate.net> Message-ID: <97988640-DCC0-4068-A88F-D0201DBBE8EF@iki.fi> I don't think you can do that with a plugin without core Dovecot modifications. Unless you replace the whole passdb ldap. For example you could use passdb checkpassword if performance isn't a big issue. On 9.10.2012, at 22.24, James Devine wrote: > Here is an example of the problem: > > Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: > password(user at domain.tld,192.168.160.49): Generating NTLM from user > 'user at domain.tld@', password 'IfBG6G3jykirE5r5vienC4w==' > Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: > password(user at domain.tld,192.168.160.49): Credentials: > f124dc24328ed3d90db035f0d5284636 > > The listed password is a base64 representation of its encrypted form which > I need to somehow decrypt between the time LDAP returns it and these > credentials are generated. > > > > On Tue, Oct 9, 2012 at 1:16 PM, James Devine wrote: > >> I don't think I understand. Right now the problem is the password >> retrieved from LDAP cannot be hashed to compare against what the user sent >> because it is encrypted. I have to perform my AES decryption before it can >> be hashed and compared. >> >> >> On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: >> >>> On 2012.10.09 14.41, James Devine wrote: >>> >>>> We have an LDAP server that contains AES encrypted passwords. So far >>>> I've >>>> been able to use this by adding a passdb module that encrypts the user's >>>> password prior to ldap comparison. Now I am looking at supporting >>>> client-side encrypted passwords. To do this I need to decrypt the >>>> password >>>> returned by LDAP. Is there a way to insert a module to do this >>>> decryption >>>> between ldap returning and the auth mechanism? >>>> >>>> that would be unwise, generally speaking. as a rule of thumb, in terms >>> of security fundamentals, only the rootdn [or equiv] should be able to read >>> the values in an ldap entry's password attribute. certainly the service >>> account used by dovecot should not. >>> >>> in the context of ldap, authentication should be accomplished by binding >>> as the user, not by retrieving attribute values and performing string >>> comparisons. among other things, this decouples the two components and >>> allows applications [e.g. dovecot] to be unconcerned with whatever password >>> hashing scheme the directory server might be using. >>> >>> -ben >>> >> >> From marc at perkel.com Wed Oct 10 01:16:39 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 15:16:39 -0700 Subject: [Dovecot] Multiple Maildir? Message-ID: <5074A247.6080307@perkel.com> OK - Getting close to everything working in my weird configuration. Here's a problem I'm still having. I'm authenticating with this: userdb { driver = sql args = /etc/dovecot/master-combined-sql.conf } Default mail location: mail_location = maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs However - what I'd like to do is if the mail location doesn't exist then I want to try a second mail location: mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs How do I do that? Thanks in advance From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 10 01:42:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 10 Oct 2012 00:42:18 +0200 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074A247.6080307@perkel.com> References: <5074A247.6080307@perkel.com> Message-ID: <20121009224218.GA11401@daniel.localdomain> Marc Perkel wrote: > if the mail location doesn't exist > then I want to try a second mail location: > mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs You might do this with a script which exports the MAIL environment variable and then executes the service binary: http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection Regards Daniel -- https://plus.google.com/103021802792276734820 From kgc at corp.sonic.net Wed Oct 10 03:12:47 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 09 Oct 2012 17:12:47 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <20121009224218.GA11401@daniel.localdomain> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> Message-ID: <5074BD7F.50206@corp.sonic.net> On 10/09/12 15:42, Daniel Parthey wrote: > Marc Perkel wrote: >> if the mail location doesn't exist >> then I want to try a second mail location: >> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > > You might do this with a script which exports the MAIL environment > variable and then executes the service binary: It will work, we do this to set the maildir location to a custom hashed directory and muck around with the namespaces a bit. The script is in perl - the relevant parts look like this. #set user's maildir location for dovecot $ENV{'MAIL'} = 'maildir:' . getmaildir($ENV{'USER'}); $ENV{'USERDB_KEYS'} .= 'MAIL'; #pass along to dovecot's next process exec { $ARGV[0] } @ARGV; -K From marc at perkel.com Wed Oct 10 04:09:22 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:09:22 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <20121009224218.GA11401@daniel.localdomain> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> Message-ID: <5074CAC2.2030507@perkel.com> On 10/9/2012 3:42 PM, Daniel Parthey wrote: > Marc Perkel wrote: >> if the mail location doesn't exist >> then I want to try a second mail location: >> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > You might do this with a script which exports the MAIL environment > variable and then executes the service binary: > > http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection > > Regards > Daniel I see the idea. But how do I pick up the name and domain parameters to test the directory? From marc at perkel.com Wed Oct 10 04:12:11 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:12:11 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074BD7F.50206@corp.sonic.net> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> <5074BD7F.50206@corp.sonic.net> Message-ID: <5074CB6B.4090202@perkel.com> On 10/9/2012 5:12 PM, Kelsey Cummings wrote: > On 10/09/12 15:42, Daniel Parthey wrote: >> Marc Perkel wrote: >>> if the mail location doesn't exist >>> then I want to try a second mail location: >>> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> >> You might do this with a script which exports the MAIL environment >> variable and then executes the service binary: > > It will work, we do this to set the maildir location to a custom > hashed directory and muck around with the namespaces a bit. > > The script is in perl - the relevant parts look like this. > > #set user's maildir location for dovecot > $ENV{'MAIL'} = 'maildir:' . getmaildir($ENV{'USER'}); > $ENV{'USERDB_KEYS'} .= 'MAIL'; > > #pass along to dovecot's next process > exec { $ARGV[0] } @ARGV; > > -K > > Namespaces is something I don't understand. Still wondering what environment variables I can pick up in this script. What I want to do is first try /fakedir/%d/%n and if that doesn't exist I want to go to /email/%d/%n From marc at perkel.com Wed Oct 10 04:34:08 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:34:08 -0700 Subject: [Dovecot] Feature Request Message-ID: <5074D090.3010909@perkel.com> It would be handy (for me) if there were a userdb where a directory structure defined the db. userdb stat { mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } userdb stat { mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } The idea being that if the first directory doesn't exist then it will try the second one. From tss at iki.fi Wed Oct 10 05:29:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 05:29:05 +0300 Subject: [Dovecot] Feature Request In-Reply-To: <5074D090.3010909@perkel.com> References: <5074D090.3010909@perkel.com> Message-ID: <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> On 10.10.2012, at 4.34, Marc Perkel wrote: > It would be handy (for me) if there were a userdb where a directory structure defined the db. > > userdb stat { > mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > } > > userdb stat { > mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > } > > The idea being that if the first directory doesn't exist then it will try the second one. You could already implement this as userdb checkpassword script. From marc at perkel.com Wed Oct 10 05:40:27 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 19:40:27 -0700 Subject: [Dovecot] Feature Request In-Reply-To: <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> Message-ID: <5074E01B.8030001@perkel.com> On 10/9/2012 7:29 PM, Timo Sirainen wrote: > On 10.10.2012, at 4.34, Marc Perkel wrote: > >> It would be handy (for me) if there were a userdb where a directory structure defined the db. >> >> userdb stat { >> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> } >> >> userdb stat { >> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> } >> >> The idea being that if the first directory doesn't exist then it will try the second one. > You could already implement this as userdb checkpassword script. Can you give me an example? From tss at iki.fi Wed Oct 10 05:50:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 05:50:15 +0300 Subject: [Dovecot] Feature Request In-Reply-To: <5074E01B.8030001@perkel.com> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> <5074E01B.8030001@perkel.com> Message-ID: <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> On 10.10.2012, at 5.40, Marc Perkel wrote: >>> It would be handy (for me) if there were a userdb where a directory structure defined the db. >>> >>> userdb stat { >>> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >>> } >>> >>> userdb stat { >>> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >>> } >>> >>> The idea being that if the first directory doesn't exist then it will try the second one. >> You could already implement this as userdb checkpassword script. > > Can you give me an example? Something like this: userdb { driver = checkpasword args = /usr/local/bin/userdb.sh } /usr/local/bin/userdb.sh: #!/bin/sh path=/fakedir/$AUTH_DOMAIN/$AUTH_USERNAME if [ -d $path ]; then MAIL=maildir:$path:INBOX=/email/$AUTH_DOMAIN/$AUTH_USERNAME:LAYOUT=fs EXTRA=mail AUTHORIZED=2 exec "$@" fi exit 1 (I'm not sure if the MAIL/mail should be USERDB_MAIL/userdb_mail instead. Probably not.) From raabe at froglogic.com Wed Oct 10 10:29:00 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 09:29:00 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes Message-ID: <507523BC.9050004@froglogic.com> Hi, I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for providing access to a mailing list archive. The archive is implemented as a public read-only mailbox with per-user index files (i.e. the \Seen flags are per-user). I recently enbled the Squat plugin to accelerate searches in the message bodies and noticed that every user (I'm using a virtual user setup) gets his own dovecot.index.search and dovecot.index.search.uids copies. Is it possible to share those files among all users of the system? The squat plugin appears to store the search indices among the other index files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no matter what; I considered storing a central copy of the index files somewhere and then creating symlinks for all users. It should be ok as far as file-permissions go since all mail is owned by a single vmail system user, but I wonder whether the indices are really the same (I noticed their md5 checksums differ) and whether there may be file locking issues in case two users search message bodies simultaneously. Can anybody shed some light? -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From robert at schetterer.org Wed Oct 10 10:49:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 09:49:33 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <507523BC.9050004@froglogic.com> References: <507523BC.9050004@froglogic.com> Message-ID: <5075288D.9080304@schetterer.org> Am 10.10.2012 09:29, schrieb Frerich Raabe: > Hi, > > I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for > providing access to a mailing list archive. The archive is implemented > as a public read-only mailbox with per-user index files (i.e. the \Seen > flags are per-user). i guess better upgrade to 2.1.x first > > I recently enbled the Squat plugin to accelerate searches in the message > bodies and noticed that every user (I'm using a virtual user setup) gets > his own dovecot.index.search and dovecot.index.search.uids copies. > > Is it possible to share those files among all users of the system? The > squat plugin appears to store the search indices among the other index > files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no > matter what; I considered storing a central copy of the index files > somewhere and then creating symlinks for all users. It should be ok as > far as file-permissions go since all mail is owned by a single vmail > system user, but I wonder whether the indices are really the same (I > noticed their md5 checksums differ) and whether there may be file > locking issues in case two users search message bodies simultaneously. > > Can anybody shed some light? > after upgrade http://wiki2.dovecot.org/Plugins/FTS/Lucene may be better choice this info might help http://wiki2.dovecot.org/MailLocation ---snip Index files Index files are by default stored under the same directory as mails. With maildir they are stored in the actual maildirs, with mbox they are stored under .imap/ directory. You may want to change the index file location if you're using NFS or if you're setting up shared mailboxes. You can change the index file location by adding :INDEX= to mail_location. For example: mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u --snip after upgrade come back, ask again, or meanwhile Timo gives better advice -- Best Regards MfG Robert Schetterer From raabe at froglogic.com Wed Oct 10 11:06:47 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 10:06:47 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <5075288D.9080304@schetterer.org> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> Message-ID: <50752C97.1010209@froglogic.com> Am 10.10.2012 09:49, schrieb Robert Schetterer: > Am 10.10.2012 09:29, schrieb Frerich Raabe: >> I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for >> providing access to a mailing list archive. The archive is implemented >> as a public read-only mailbox with per-user index files (i.e. the \Seen >> flags are per-user). > > i guess better upgrade to 2.1.x > first Given that Dovecot 1.2.17 works fine for me, I actually didn't see the need to upgrade yet. >> I recently enbled the Squat plugin to accelerate searches in the message >> bodies and noticed that every user (I'm using a virtual user setup) gets >> his own dovecot.index.search and dovecot.index.search.uids copies. >> >> Is it possible to share those files among all users of the system? The >> squat plugin appears to store the search indices among the other index >> files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no >> matter what; I considered storing a central copy of the index files >> somewhere and then creating symlinks for all users. It should be ok as >> far as file-permissions go since all mail is owned by a single vmail >> system user, but I wonder whether the indices are really the same (I >> noticed their md5 checksums differ) and whether there may be file >> locking issues in case two users search message bodies simultaneously. >> >> Can anybody shed some light? >> > > after upgrade > http://wiki2.dovecot.org/Plugins/FTS/Lucene > may be better choice Why? > this info might help > > http://wiki2.dovecot.org/MailLocation > > ---snip > Index files > > Index files are by default stored under the same directory as mails. > With maildir they are stored in the actual maildirs, with mbox they are > stored under .imap/ directory. You may want to change the index file > location if you're using NFS or if you're setting up shared mailboxes. > > You can change the index file location by adding :INDEX= to > mail_location. For example: > > mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u > --snip I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: namespace public { separator = / prefix = Lists/ location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists subscriptions = no } Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) > after upgrade come back, ask again, or meanwhile Timo gives better advice Does this imply that questions regarding Dovecot 1.2.17 are considered offtopic on this list? If so, I apologize - I'll look for another forum then. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From robert at schetterer.org Wed Oct 10 11:24:49 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 10:24:49 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <50752C97.1010209@froglogic.com> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> Message-ID: <507530D1.8070202@schetterer.org> Am 10.10.2012 10:06, schrieb Frerich Raabe: > I already use this; as I mentioned, the index files of the public > readonly mailbox is stored per-user so that each user has his own set of > \Seen flags. Here's my public namespace: > > namespace public { > separator = / > prefix = Lists/ > location = > maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > > subscriptions = no > } > > Alas, this means that *all* index files (including the Squat index) is > stored per-user whereas I'd just to have just *some* of them per-user. :-) > >> after upgrade come back, ask again, or meanwhile Timo gives better advice > > Does this imply that questions regarding Dovecot 1.2.17 are considered > offtopic on this list? If so, I apologize - I'll look for another forum > then. no wait till others will reply, indexing questions might be more complex to answer, then i know recent in short time perhaps meanwhile this helps -snip http://wiki2.dovecot.org/SharedMailboxes/Public With Maildir a dovecot-shared file controls if the \Seen flags are shared or private. The file must be created separately inside each Maildir, although if the file already exists in the Maildir root it's automatically copied for newly created mailboxes. If dovecot-shared file doesn't exist in Maildir, the \Seen flags are shared. If it exists, the \Seen flag state is stored only in the user's index files. By making each user have their own private index files, you can make the \Seen flag private for the users. For example: namespace { type = public separator = / prefix = Public/ location = maildir:/var/mail/public:INDEX=~/Maildir/public subscriptions = no } Now when accessing e.g. "Public/lkml" mailbox, Dovecot keeps its index files in ~/Maildir/public/lkml/ directory. If it ever gets deleted, the \Seen flags are lost. If you want to change what flags are shared when dovecot-shared file exists, currently you'll have to modify the source code: src/lib-storage/index/maildir/maildir-storage.c maildir_open() has mbox->ibox.box.private_flags_mask = MAIL_SEEN; Change the MAIL_SEEN to any flag combination you want. See src/lib-mail/mail-types.h for list of valid flags. --snip however if there is a need for some new/debugged/better features relate to what you asked you have to upgrade to latest dovecot -- Best Regards MfG Robert Schetterer From sandro.tosi at dada.eu Wed Oct 10 12:23:17 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Wed, 10 Oct 2012 11:23:17 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups Message-ID: <50753E85.5060904@dada.eu> Hello, we're scouting if it's possible to use Pigeonhole (currently v0.3.1, as this will be provided with an upcoming Debian package) with MySQL dict lookups with the mail setup we're designing. Our (main) goals are: 1. store the filters on the database 2. allow each user to enable/disable any of the filters set we provide (it's a static set of some general filters, available to all the users; we're currently not providing the possibility to users to write their own filters) For point 1) we already see[1] that's possible, but it uses the map construct that might not fit with our current database structure: we have a domain table (storing the domain info) and a mailbox table (storing the mailbox info, but the username is composed by the local part, stored in this table, and the domain part is a FK to the domain table, using an id). Do you think it's possible to run a join query on domain+mailbox to retrieve the mailbox_id needed to query the table for the filters? Or do we have to create the filter table and store the local at domain.ext info there ("relaxing" the integrity relationships between tables)? How do we specify which filters are enabled for any given user? We originally thought of an "Enabled" field on the filter table, but in the example in the doc[1] I hadn't seen a way to do that: it seems like the filter list is specified in the proxy definition - am I wrong? How can we do that? [1] http://hg.rename-it.nl/dovecot-2.1-pigeonhole/file/e9ed5d5cef4b/doc/script-location-dict.txt I think it's enough for now, maybe additional questions will arise going deeper in details :) Thanks in advance, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From raabe at froglogic.com Wed Oct 10 12:49:27 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 11:49:27 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <507530D1.8070202@schetterer.org> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <507530D1.8070202@schetterer.org> Message-ID: <507544A7.9050306@froglogic.com> Am 10.10.2012 10:24, schrieb Robert Schetterer: > Am 10.10.2012 10:06, schrieb Frerich Raabe: >> I already use this; as I mentioned, the index files of the public >> readonly mailbox is stored per-user so that each user has his own set of >> \Seen flags. Here's my public namespace: >> >> namespace public { >> separator = / >> prefix = Lists/ >> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >> subscriptions = no >> } [..] > perhaps meanwhile this helps > > -snip > > http://wiki2.dovecot.org/SharedMailboxes/Public [..] > namespace { > type = public > separator = / > prefix = Public/ > location = maildir:/var/mail/public:INDEX=~/Maildir/public > subscriptions = no > } Note how this is basically exactly the same as what I posted, except that it uses the Dovecot 2 configuration file format ('type = public') and that it calls the prefix/location "public" instead of "lists". > If you want to change what flags are shared when dovecot-shared file > exists, currently you'll have to modify the source code: > src/lib-storage/index/maildir/maildir-storage.c maildir_open() has > mbox->ibox.box.private_flags_mask = MAIL_SEEN; Change the MAIL_SEEN to > any flag combination you want. See src/lib-mail/mail-types.h for list of > valid flags. I don't think this is applicable to my case, and a check of the source code seems to confirm that: I'm not trying to change the set of flags stored for a given mail but rather the index file of the Squat plugin. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From natanael.copa at gmail.com Wed Oct 10 13:00:51 2012 From: natanael.copa at gmail.com (Natanael Copa) Date: Wed, 10 Oct 2012 12:00:51 +0200 Subject: [Dovecot] [PATCH] dovadm plugins underlinking Message-ID: Hi, Running doveadm on Alpine Linux will show various underlinking errors: /usr/bin/doveadm: symbol 'acl_user_module': can't resolve symbol in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. /usr/bin/doveadm: symbol 'acl_identifier_parse': can't resolve symbol in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. ... (complete list is found here: http://bugs.alpinelinux.org/issues/1274) I understand that those are "harmless" (at least as long as you dont use those plugins), and even if some dynamic linkers are forgiving, the doveadm plugin modules are technically underlinked. The lib10_doveadm_acl_plugin.so uses symbol acl_user_module: $ nm -D /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep acl_user_module U acl_user_module That symbol is defined in lib01_acl_plugin.so: $ nm -D /usr/lib/dovecot/lib01_acl_plugin.so | grep acl_user_module 000000000020fb80 D acl_user_module Which means that lib10_doveadm_acl_plugin.so should be directly linked to lib01_acl_plugin.so. But it is not: $ readelf -d /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep NEEDED 0x0000000000000001 (NEEDED) Shared library: [librt.so.0.9.32] 0x0000000000000001 (NEEDED) Shared library: [libc.so.0.9.32] Below is a patch that should solve it. --- diff -r 4d268e810c15 src/plugins/acl/Makefile.am --- a/src/plugins/acl/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/acl/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -10,7 +10,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_acl_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_acl_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_acl_plugin_la_LIBADD = $(module_LTLIBRARIES) lib01_acl_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/expire/Makefile.am --- a/src/plugins/expire/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/expire/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -14,7 +14,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_expire_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_expire_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_expire_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_expire_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/fts/Makefile.am --- a/src/plugins/fts/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/fts/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib20_doveadm_fts_plugin_la_LDFLAGS = -module -avoid-version +lib20_doveadm_fts_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib20_doveadm_fts_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_fts_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/quota/Makefile.am --- a/src/plugins/quota/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/quota/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_quota_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_quota_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_quota_plugin_la_LIBADD = $(module_LTLIBRARIES) lib10_quota_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/zlib/Makefile.am --- a/src/plugins/zlib/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/zlib/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_zlib_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_zlib_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_zlib_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_zlib_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ From tss at iki.fi Wed Oct 10 13:22:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 13:22:53 +0300 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: References: Message-ID: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> On 10.10.2012, at 13.00, Natanael Copa wrote: > Running doveadm on Alpine Linux will show various underlinking errors: > /usr/bin/doveadm: symbol 'acl_user_module': can't resolve symbol in > lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. > /usr/bin/doveadm: symbol 'acl_identifier_parse': can't resolve symbol > in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. .. > Which means that lib10_doveadm_acl_plugin.so should be directly linked > to lib01_acl_plugin.so. But it is not: .. > Below is a patch that should solve it. I think I once did that with imap_quota plugin for quota plugin. The problem was that then it was possible to set "mail_plugins=imap_quota", which automatically loaded the quota plugin, but didn't call its quota_plugin_init() function, which caused some strange behavior (crashed probably). With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. Maybe there are some other ways to make this work more nicely. There would still be time to change it for v2.2 if you have good ideas. :) From natanael.copa at gmail.com Wed Oct 10 15:17:58 2012 From: natanael.copa at gmail.com (Natanael Copa) Date: Wed, 10 Oct 2012 14:17:58 +0200 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> References: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> Message-ID: On Wed, Oct 10, 2012 at 12:22 PM, Timo Sirainen wrote: > On 10.10.2012, at 13.00, Natanael Copa wrote: > >> Running doveadm on Alpine Linux will show various underlinking errors: > With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. Why does it need to load all the plugins? Why not only try to load those who are enabled? > Maybe there are some other ways to make this work more nicely. There would still be time to change it for v2.2 if you have good ideas. :) Link statically? Or only link in the .o files that has the needed symbols? I'm not familiar with the code, but how about adding an char* adm_module to the module struct, and then instead of trying to dlopen all in doveadm dir, loop through all the already loaded modules and dlopen(loaded_module->adm_module)? -- Natanael Copa From listas at adminlinux.com.br Wed Oct 10 15:46:56 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Wed, 10 Oct 2012 09:46:56 -0300 Subject: [Dovecot] About ManageSieve and TLS In-Reply-To: <506B40F9.1000905@rename-it.nl> References: <506B3653.5020804@adminlinux.com.br> <506B40F9.1000905@rename-it.nl> Message-ID: <50756E40.2040408@adminlinux.com.br> It works ! Thanks Stephan! On 02-10-2012 16:31, Stephan Bosch wrote: > If you really want to, you can disable TLS for ManageSieve > specifically by putting a ssl=no inside the protocol sieve {} section. -- Thiago Henrique adminlinux.com.br From listas at adminlinux.com.br Wed Oct 10 16:39:27 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Wed, 10 Oct 2012 10:39:27 -0300 Subject: [Dovecot] Irrelevant information filling logs Message-ID: <50757A8F.4030508@adminlinux.com.br> Hi, I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. There is a lot of logs of "quota exceeded" like this: Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. Is there a way to send specific Dovecot errors on specific files or just discard them? Thanks -- Thiago Henrique adminlinux.com.br From robert at schetterer.org Wed Oct 10 17:14:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 16:14:15 +0200 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <507582B7.5040400@schetterer.org> Am 10.10.2012 15:39, schrieb 3.listas at adminlinux.com.br: > Hi, > > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My > mailbox server is shared by ~ 10k domains. It works fine with ~50k > accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, > user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: > msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded > (mailbox for user is full) > > These messages are not important to me. But these messages fill the log > files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or just > discard them? > > Thanks > -- > Thiago Henrique > adminlinux.com.br no sure if you can disable this notice without loosing other wanted ones by dovecot log settings if using rsyslog you can do it with discard action http://www.rsyslog.com/discarding-unwanted-messages/ -- Best Regards MfG Robert Schetterer From tibby at tibby.hu Wed Oct 10 17:29:08 2012 From: tibby at tibby.hu (Tibby) Date: Wed, 10 Oct 2012 16:29:08 +0200 (CEST) Subject: [Dovecot] I need a quota expert In-Reply-To: <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> References: <20121010142523.871B11AE881D@dovecot.org> <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> Message-ID: <648980957.4179.1349879348102.JavaMail.root@tibby.hu> I'm having problems with dovecot quota configuration. If I Include the driver mysql in dovecot-sql.conf I'm getting this error: dovecot: dict: Error in configuration file /etc/dovecot/dovecot-sql.conf line 1: Unknown setting: driver dovecot: dict: Failed to initialize dictionary 'quotadict' If I Don't include it: dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf Now What??? How Should I configure quota ? My SQL has a mail db with a user table. The table has email, password, quota_kb fields. my dovecot.conf: ---------------- protocols = imap imaps pop3 pop3s log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir disable_plaintext_auth = no ssl_cert_file = /etc/ssl/certs/dovecot.pem ssl_key_file = /etc/ssl/private/dovecot.pem namespace private { separator = . prefix = INBOX. inbox = yes } protocol imap { mail_plugins = quota imap_quota } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster at domain.tld mail_plugins = sieve quota global_script_path = /home/vmail/globalsieverc mail_plugin_dir = /usr/lib/dovecot/modules/lda } protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } dict { quotadict = mysql:/etc/dovecot/dovecot-sql.conf } plugin { quota = dict:user::proxy::quotadict quota_rule = *:storage=10M:messages=1000 } auth default { user = root passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb sql { args = /etc/dovecot/dovecot-sql.conf } socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } mail_uid = 5000 mail_gid = 5000 my dovecot-sql.conf: ---------------- driver = mysql connect = host=127.0.0.1 dbname=mail user=****** password=****** default_pass_scheme = CRYPT password_query = SELECT email as user, password, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u'; user_query = SELECT CONCAT('/home/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u' map { pattern = priv/quota/storage table = user username_field = user value_field = quota_kb } SOMEBODY PLEASE HELP ME!!! :) From jbates at brightok.net Wed Oct 10 17:37:16 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:37:16 -0500 Subject: [Dovecot] lmtp proxy logging Message-ID: <5075881C.4060905@brightok.net> The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. Jack From roundcube222 at alaadin.org Wed Oct 10 17:40:12 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 10 Oct 2012 17:40:12 +0300 Subject: [Dovecot] Dovecot doesnot disconnect at end of pop3 session Message-ID: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> Hello, I have 3 users checking same email account using pop3 (outlook express) suddenly after some hours , users are unable to disconnect from pop3 after checking mail So i telnet to the pop3 and found that the issue is true but i do not know the reason telnet mymail.com 110 user username pass password list quit all commands work except the quit command, Dovecot hang at this point !!! Why Dovecot hang on receiving the Quit command !!! and doesnot disconnect !! Any help is greatly appreciated. Thanks. From jbates at brightok.net Wed Oct 10 17:43:02 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:43:02 -0500 Subject: [Dovecot] I need a quota expert In-Reply-To: <648980957.4179.1349879348102.JavaMail.root@tibby.hu> References: <20121010142523.871B11AE881D@dovecot.org> <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> <648980957.4179.1349879348102.JavaMail.root@tibby.hu> Message-ID: <50758976.8060302@brightok.net> Looking at the wiki: http://wiki2.dovecot.org/Quota/Dict It appears that the conf files for dict are different than the sql files for auth. This is why you are getting error messages. The auth config needs the driver option, the dict config does not. Create a file specifically for dict and point to that. Use the above wiki's examples for format. Jack On 10/10/2012 9:29 AM, Tibby wrote: > I'm having problems with dovecot quota configuration. > If I Include the driver mysql in dovecot-sql.conf I'm getting this error: > dovecot: dict: Error in configuration file /etc/dovecot/dovecot-sql.conf line 1: Unknown setting: driver > dovecot: dict: Failed to initialize dictionary 'quotadict' > > If I Don't include it: > dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf > dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf > > Now What??? > How Should I configure quota ? > > My SQL has a mail db with a user table. > The table has email, password, quota_kb fields. > > my dovecot.conf: > ---------------- > protocols = imap imaps pop3 pop3s > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/home/vmail/%d/%n/Maildir > disable_plaintext_auth = no > ssl_cert_file = /etc/ssl/certs/dovecot.pem > ssl_key_file = /etc/ssl/private/dovecot.pem > namespace private { > separator = . > prefix = INBOX. > inbox = yes > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol lda { > log_path = /home/vmail/dovecot-deliver.log > auth_socket_path = /var/run/dovecot/auth-master > postmaster_address = postmaster at domain.tld > mail_plugins = sieve quota > global_script_path = /home/vmail/globalsieverc > mail_plugin_dir = /usr/lib/dovecot/modules/lda > } > protocol pop3 { > pop3_uidl_format = %08Xu%08Xv > mail_plugins = quota > } > dict { > quotadict = mysql:/etc/dovecot/dovecot-sql.conf > } > plugin { > quota = dict:user::proxy::quotadict > quota_rule = *:storage=10M:messages=1000 > } > auth default { > user = root > passdb sql { > args = /etc/dovecot/dovecot-sql.conf > } > userdb sql { > args = /etc/dovecot/dovecot-sql.conf > } > socket listen { > master { > path = /var/run/dovecot/auth-master > mode = 0600 > user = vmail > } > client { > path = /var/spool/postfix/private/auth > mode = 0660 > user = postfix > group = postfix > } > } > } > mail_uid = 5000 > mail_gid = 5000 > > > > > > my dovecot-sql.conf: > ---------------- > driver = mysql > connect = host=127.0.0.1 dbname=mail user=****** password=****** > default_pass_scheme = CRYPT > password_query = SELECT email as user, password, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u'; > > user_query = SELECT CONCAT('/home/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u' > > map { > pattern = priv/quota/storage > table = user > username_field = user > value_field = quota_kb > } > > > SOMEBODY PLEASE HELP ME!!! :) From jbates at brightok.net Wed Oct 10 17:47:24 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:47:24 -0500 Subject: [Dovecot] Dovecot doesnot disconnect at end of pop3 session In-Reply-To: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> References: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> Message-ID: <50758A7C.1050501@brightok.net> On 10/10/2012 9:40 AM, Robert JR wrote: > > telnet mymail.com 110 > user username > pass password > list > quit > > all commands work except the quit command, Dovecot hang at this point !!! > > Why Dovecot hang on receiving the Quit command !!! and doesnot > disconnect !! > Please include your dovecot version. Also, does the list command end with a line that contains a period? You might want to include dovecot -n as well, which will show everyone what your mail storage type is and other relevant information. Jack From roundcube222 at alaadin.org Wed Oct 10 18:07:57 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 10 Oct 2012 18:07:57 +0300 Subject: [Dovecot] Fwd: Re: Dovecot doesnot disconnect at end of pop3 session Message-ID: On 2012-10-10 17:47, Jack Bates wrote: > On 10/10/2012 9:40 AM, Robert JR wrote: > >> telnet mymail.com 110 user username pass password list quit all >> commands work except the quit command, Dovecot hang at this point >> !!! >> Why Dovecot hang on receiving the Quit command !!! and doesnot >> disconnect !! > Please include your dovecot version. Also, does the list command end > with a line that contains a period? You might want to include dovecot > -n > as well, which will show everyone what your mail storage type is and > other relevant information. Jack # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.9.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login auth_username_format = %Lu disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mal:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = References: Message-ID: <50759753.5040301@brightok.net> On 10/10/2012 10:07 AM, Robert JR wrote: > > list command return msg number and size only. > Thanks Below is the proper format you should see. The line with a period at the end is required to show that the list command has completed. If you do not get that, then the server hasn't successfully completed the list command to accept new commands. Check your logs for any mention of the connection and errors that might have generated. If you are using NFS, there are several issues with locking that can be problematic. If you are getting the '.' line, then try other commands besides quit. It is important to determine if the code is ignoring input or if it has a problem with the actual quit code. Also, I notice you mentioned several hours. It would be useful to know if this occurs on startup or if it fails over time. It is also useful to know if it effects all accounts or just some of the active accounts. It is also useful to know if there are people currently logged in on those accounts when you perform your test. telnet localhost 110 Trying ::1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user **** +OK pass **** +OK Logged in. list +OK 3 messages: 1 2821 2 5907 3 11171 . quit +OK Logging out. Connection closed by foreign host. From slitt at troubleshooters.com Wed Oct 10 20:35:08 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Wed, 10 Oct 2012 13:35:08 -0400 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <20121010133508.6f836584@mydesk> On Wed, 10 Oct 2012 10:39:27 -0300, 3.listas at adminlinux.com.br said: > Hi, > > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. > My mailbox server is shared by ~ 10k domains. It works fine with ~50k > accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, > user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: > msgid=unspecified: failed to store into mailbox 'INBOX': Quota > exceeded (mailbox for user is full) > > These messages are not important to me. But these messages fill the > log files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or > just discard them? If all else fails, you could cat the log through a grep -v to filter out the quota exceeded messages, and then pipe it to less for viewing. SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From jbates at brightok.net Wed Oct 10 21:09:25 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 13:09:25 -0500 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <5075B9D5.8030007@brightok.net> On 10/10/2012 8:39 AM, 3.listas at adminlinux.com.br wrote: > > Is there a way to send specific Dovecot errors on specific files or > just discard them? syslog/rsyslog filters are the best method, in my opinion. This would allow you to filter to another file or discard things you aren't interested in all together. I personally like quota messages in another logfile. Then I can tail the file with a script and issue changes so that the MTA servers reject emails to that user instead of trying to deliver, then periodically check quotas for users we currently block to determine when it is okay to accept emails again. It really lowers the bounces on the MTA servers. Jack From tss at iki.fi Wed Oct 10 22:37:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 22:37:25 +0300 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <5B596E8E-0721-4EE9-81E2-FC7F308B3C5C@iki.fi> On 10.10.2012, at 16.39, 3.listas at adminlinux.com.br wrote: > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) > > These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or just discard them? Upgrade to v2.1, they are logged with info level there. (I think v2.0 also logs them with info level if you don't use Sieve.) From dovecot at freakout.de Thu Oct 11 09:43:18 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Thu, 11 Oct 2012 08:43:18 +0200 (CEST) Subject: [Dovecot] dovecot cores Message-ID: <201210110643.q9B6hIF6003121@bongo.freakout.de> Hi dovecot-community, i have set up a new dovecot server - everything is self-compiled and with newest versions. dovecot daemon dies every night: Bugreport Mail - dovecot cores with sig11 my dovecot daemon cores every night - no message in log file. Please help. It runs in an OpenVZ container - therefore reiserfs is reported as filesystem. root at glen ~]# cat /etc/dovecot/dovecot.conf # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.8.2.el5.028stab101.1PAE i686 CentOS release 5.8 (Final) reiserfs auth_mechanisms = plain login cram-md5 apop default_login_user = nobody disable_plaintext_auth = no first_valid_uid = 300 listen = * mail_gid = 332 mail_home = /var/dovecot mail_location = maildir:/var/spool/mail/virtual/%d/%n mail_plugins = quota mail_uid = 332 passdb { driver = sql args = /etc/dovecot/sql.conf } plugin { quota = maildir:User quota } protocols = imap pop3 ssl_cert = This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /opt/dovecot-2.1.10-3/sbin/dovecot...(no debugging symbols found)...done. [New Thread 20049] Reading symbols from /opt/mysql/lib/libmysqlclient.so.18...done. Loaded symbols for /opt/mysql/lib/libmysqlclient.so.18 Reading symbols from /opt/dovecot/lib/libdovecot.so.0...done. Loaded symbols for /opt/dovecot/lib/libdovecot.so.0 Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /opt/mysql/lib/libstrings.so...done. Loaded symbols for /opt/mysql/lib/libstrings.so Reading symbols from /opt/zlib/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /opt/zlib/lib/libz.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /opt/ssp/lib/libssp.so.0...done. Loaded symbols for /opt/ssp/lib/libssp.so.0 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/opt/dovecot/sbin/dovecot'. Program terminated with signal 11, Segmentation fault. #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 (gdb) bt #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 #1 0xb7f131c0 in master_instance_list_add_line (list=0x9d48880, line=0x9d540c8 "1349762052\tdovecot\t/var/dovecot/run\t") at master-instance.c:85 #2 0xb7f1331f in master_instance_list_refresh (list=0x9d48880) at master-instance.c:115 #3 0xb7f13620 in master_instance_write_init (list=0x9d48880, dotlock_r=0xbff6fb60) at master-instance.c:173 #4 0xb7f1390d in master_instance_list_set_name (list=0x9d48880, base_dir=0x8055300 "/var/dovecot/run", name=0x8055323 "dovecot") at master-instance.c:252 #5 0x0804c347 in net_accept () at network.c:502 #6 0xb7f3255b in io_loop_handle_timeouts_real (ioloop=0x9d465b0) at ioloop.c:354 #7 0xb7f325de in io_loop_handle_timeouts (ioloop=0x9d465b0) at ioloop.c:367 #8 0xb7f338b6 in io_loop_handler_run (ioloop=0x9d465b0) at ioloop-poll.c:171 #9 0xb7f326d3 in io_loop_run (ioloop=0x9d465b0) at ioloop.c:398 #10 0xb7f17304 in master_service_run (service=0x9d464e0, callback=0) at master-service.c:543 #11 0x0804d375 in net_accept () at network.c:502 #12 0xb7d84e9c in __libc_start_main () from /lib/libc.so.6 #13 0x0804b501 in net_accept () at network.c:502 (gdb) q Regards Axel From calestyo at scientia.net Thu Oct 11 05:18:34 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 04:18:34 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup (ignore) In-Reply-To: <1349921426.3341.175.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> Message-ID: <1349921914.3341.183.camel@fermat.scientia.net> Oops... that was obivously not intended for dovecot but AOX mailing list,... where I ask around similar questions. Sorry for the noise =) Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From talanchor at mail.ru Thu Oct 11 13:21:09 2012 From: talanchor at mail.ru (=?UTF-8?B?LiAu?=) Date: Thu, 11 Oct 2012 14:21:09 +0400 Subject: [Dovecot] =?utf-8?q?memory_allocation_in_new_thread?= Message-ID: <1349950869.22650094@f123.mail.ru> Hi! I have some problems with memory allocation. I create new thread in cidir storage and call malloc(), and it fails to allocate even 1 byte. What can cause this problem? Dovecot vesrion is: 2.1.10 (130563b592c9+) Sample code looks like this (I also link to pthread with: -pthread): #define TEST_MALLOC() \ { \ ??? void *p; \ ??? p = malloc(1); \ ??? if (!p) { \ ??? ??? i_info("%s: malloc() failed", __FUNCTION__); \ ??? } else { \ ??? ??? i_info("%s: malloc() succeeded", __FUNCTION__); \ ??? ??? free(p); \ ??? } \ } #include static void *test_pthread_malloc_func(void *data_) { ??? TEST_MALLOC(); ??? return NULL; } static void test_pthread_malloc() { ??? int ret; ??? pthread_t tid; ??? TEST_MALLOC(); ??? ret = pthread_create(&tid, NULL, test_pthread_malloc_func, NULL); ??? if (ret) { ??????? i_info("failed to start thread"); ??? } else { ??????? pthread_join(tid, NULL); ??? } } I call test_pthread_malloc() function. It produces following output: Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc: malloc() succeeded Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc_func: malloc() failed From calestyo at scientia.net Thu Oct 11 05:10:26 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 04:10:26 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup Message-ID: <1349921426.3341.175.camel@fermat.scientia.net> Hi folks. Perhaps you find some time to look into this,... I'm trying to plan my mail system and would have some questions. The overall idea is about the following: - There is a (internet) server, which is the MTA (which will be postfix) and imap and/or pop3 server (which shall be, guess, dovecot ... AOX comes in later). - Any spam filtering, virus-scanning, mail filtering happens on that server. - I have not yet decided on whether to use maildrop for this or Sieve. Maildrop is nice, but has one major deficiency which I don't know how to work around. - A few clients (that means I won't serve 100 of users) connect to that server via imap and should see all mails, etc. already in some fancy sorted hierarchy (that means filtered into different directories). So far nothing complicated. But now... - Apart from spam, I never delete mail; and because I'm subscribed to many lists, I get a lot of mail. - Storage on my server is limited and it's located somewhere at my ISP, so I generally do not trust it with respect to safety... For both reasons, I want the canonical archive of all mail to be at home at some local server. - The local server should also be an imap server, so that I can access the archive from may computers at home. - The local server won't be available from the internet. - The local archive should have the same folder hierarchy as the internet server (I'd prefer not to filter twice). - The local archive might be AOX... Mail Flow: - I generally want to have _all_ mail (which is not sorted out because of being spam) to be archived at the local server. - But(!) I want to selectively keep (in addition) mail at the internet server. For example I may want to select the folder that contains all mail form some friend to be kept online completely. But I may want to decide that mailinglists keep only the last 10 days and/or 1000 messages of mail. - The idea is, that the local server regularly (when it is online/running) catches new mail from the internet server... and stores it in the archive. - So apart from new mail that has not yet been read, that local archive contains always all mails that are also on the internet server... the later may contain (for specific directories) the same, or just parts of. - The MUAs will then have two imap accounts, one to the internet server and one to the local archive,... each one being usable, depending on where I am. 1) This is where my first problem arises: How can I implement that mail flow, especially: - How do I secure that all mail is read from the internet server (i.e. that nothing is "forgotten"? - How do I make sure that no mails are retrieved twice (or more)? A problem which I often had with pop, when the mail client crashed during sync? - Further it must be secured, that when I delete something on the internet server, it is NOT deleted on the local server (on the next mail-fetching).... this is why I don't use the word "sync". a) One stupid solution would be, that I duplicate all mail on the online server,... one part is for staying online, one part is for being fetched to the local archive. As soon as it was fetched... that copy gets removed (always). That solution would give a clean and secured separation of both? b) I don't think offlineimap or any other caching-like solution is the right thing... especially as one must always fear that such a cache may be accidentally wiped. Are there better solutions than (a)? 2) Problem would be already a refinement of a working solution for (1) (but obviously not when using (1).(a) ). When e.g. reply to or forward a mail using the online server,... and that mail had already been fetched,... can I make the flag synced? 3) Is AOX suitable for the local server? - I couldn't use maildir locally, because I loose just to much space to the block fragmentation. But I guess I won't have this problem with the DB backend. a) Is AOX able to manage a really big folder hierarchy that basically ever keeps growing... with easily several 100k mails per folder... and that is in total already over 100GB? I read that e.g. dovecot would have more performance problems with that. b) I would prefer to have fast full text search. Does AOX provide this? I read that IMAP has limitations which make full text search not really usable via it? Further, I remember some dovecot wiki page that showed a comparison which said that both do not perfectly implement imap. Thanks a lot, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From robert at schetterer.org Thu Oct 11 15:56:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 11 Oct 2012 14:56:34 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <1349921426.3341.175.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> Message-ID: <5076C202.7080202@schetterer.org> Am 11.10.2012 04:10, schrieb Christoph Anton Mitterer: > 3) Is AOX suitable for the local server? > - I couldn't use maildir locally, because I loose just to much space to > the block fragmentation. But I guess I won't have this problem with the DB backend. > > > a) Is AOX able to manage a really big folder hierarchy that basically > ever keeps growing... with easily several 100k > mails per folder... and that is in total already over 100GB? > > I read that e.g. dovecot would have more performance problems with that. > > > b) I would prefer to have fast full text search. Does AOX provide > this? > > I read that IMAP has limitations which make full text search not really usable via it? > > > Further, I remember some dovecot wiki page that showed a comparison which said > that both do not perfectly implement imap. Christoph, sorry, what exact is AOX, and what is its relation to the dovecot list.... youre looking to an overall mail setup so split your questions up to software you wanna use and ask the related mail list, perhaps hire some mail consultant advice you -- Best Regards MfG Robert Schetterer From robert at schetterer.org Thu Oct 11 15:57:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 11 Oct 2012 14:57:11 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup (ignore) In-Reply-To: <1349921914.3341.183.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> <1349921914.3341.183.camel@fermat.scientia.net> Message-ID: <5076C227.9040303@schetterer.org> Am 11.10.2012 04:18, schrieb Christoph Anton Mitterer: > Oops... that was obivously not intended for dovecot but AOX mailing > list,... where I ask around similar questions. > > Sorry for the noise =) > > > Cheers, > Chris. > ups answered exact about this *g -- Best Regards MfG Robert Schetterer From raabe at froglogic.com Thu Oct 11 16:03:55 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 11 Oct 2012 15:03:55 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <5076C202.7080202@schetterer.org> References: <1349921426.3341.175.camel@fermat.scientia.net> <5076C202.7080202@schetterer.org> Message-ID: <5076C3BB.1090307@froglogic.com> Am 11.10.2012 14:56, schrieb Robert Schetterer: > Am 11.10.2012 04:10, schrieb Christoph Anton Mitterer: >> 3) Is AOX suitable for the local server? [..] > Christoph, sorry, what exact is AOX, and what is its relation to the > dovecot list.... I suppose he meant Archiveopteryx (another IMAP server). -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dovecot at freakout.de Thu Oct 11 16:19:27 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Thu, 11 Oct 2012 15:19:27 +0200 (CEST) Subject: [Dovecot] iterate_query does not use userdb - mail_location not found Message-ID: <201210111319.q9BDJRV3005614@bongo.freakout.de> Hi dovecot-comminity, can't get iterate_query working. doveadm cannot find mail_location which comes from userdb query. dovecot itself works fine with sql. Whats wrong? please help: [root at glen exim]# /opt/dovecot/bin/doveadm search -A mailbox Trash savedbefore 90d doveadm(uwe at mitmachnet.de): Error: user uwe at mitmachnet.de: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=(not set) doveadm(uwe at mitmachnet.de): Error: User init failed dovecot.conf: passdb { driver = sql args = /etc/dovecot/sql.conf } userdb { driver = sql args = /etc/dovecot/sql.conf } sql.conf: driver = mysql connect = host=much dbname=toarx user=exim password=xxxxxxxx default_pass_scheme = PLAIN password_query = select user as username, password, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' user_query = select user as username, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' iterate_query = select email as user from vusers Cheers Axel From jbates at brightok.net Thu Oct 11 17:28:40 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 11 Oct 2012 09:28:40 -0500 Subject: [Dovecot] iterate_query does not use userdb - mail_location not found In-Reply-To: <201210111319.q9BDJRV3005614@bongo.freakout.de> References: <201210111319.q9BDJRV3005614@bongo.freakout.de> Message-ID: <5076D798.80102@brightok.net> On 10/11/2012 8:19 AM, dovecot at freakout.de wrote: > sql.conf: > driver = mysql > connect = host=much dbname=toarx user=exim password=xxxxxxxx > default_pass_scheme = PLAIN > password_query = select user as username, password, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' > user_query = select user as username, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' > iterate_query = select email as user from vusers > Only prefetch in the password_query prefixes with userdb_. Some things such as doveadm commands and lmtp delivery don't use passdb but userdb directly. The user_query should not have the userdb_ prefix. Also, according to the wiki, iterate_query returns username, not user. http://wiki2.dovecot.org/AuthDatabase/SQL Jack From calestyo at scientia.net Thu Oct 11 19:39:49 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 18:39:49 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <5076C3BB.1090307@froglogic.com> References: <1349921426.3341.175.camel@fermat.scientia.net> <5076C202.7080202@schetterer.org> <5076C3BB.1090307@froglogic.com> Message-ID: <1349973589.3370.4.camel@fermat.scientia.net> Hi. Sorry folks for the stupid postings... At first I posted what should go to the AOX list accidentally here and then our institute's MTA hat some issues yesterday, so the mail[0] where I already tried to explain the wrong posting, came much earlier than the wrong post itself. Guess you see why I need a better mail system ;) Sorry, Chris. [0] http://dovecot.org/pipermail/dovecot/2012-October/068740.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From btb at bitrate.net Thu Oct 11 20:35:00 2012 From: btb at bitrate.net (btb) Date: Thu, 11 Oct 2012 13:35:00 -0400 Subject: [Dovecot] imap proxy setup - "killed with signal 11" Message-ID: <50770344.70905@bitrate.net> hi- i'm setting up an imap proxy in front of a novell groupwise server. it seems to so far be partially working, but dovecot is having trouble in certain cases. i expect that it's ultimately due to what i believe is a very poor implementation of imap provided by groupwise [at least based on other experiences in the past] - but that's a big part of why i'd like to have dovecot in between it and clients. below is information collected during starting of dovecot, opening/initial connection from a client [os x mail.app], closing of the client, and stopping of dovecot. os is ubuntu 12.10 development/beta, dovecot is 2.1.7 courtesy of ubuntu's packages. log entries: Oct 11 13:24:33 halo dovecot: master: Dovecot v2.1.7 starting up Oct 11 13:24:49 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14171, TLS, session= Oct 11 13:24:50 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14174, TLS, session= Oct 11 13:24:51 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14176, TLS, session= Oct 11 13:24:51 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14178, TLS, session= Oct 11 13:24:51 halo dovecot: imap(jdoe): Connection closed in=16 out=350 Oct 11 13:24:52 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14180, TLS, session= Oct 11 13:24:52 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:52 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14176 killed with signal 11 (core dumped) Oct 11 13:24:57 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:57 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14178 killed with signal 11 (core dumped) Oct 11 13:24:57 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14182, TLS, session= Oct 11 13:24:58 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:58 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14180 killed with signal 11 (core dumped) Oct 11 13:25:03 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:03 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14182 killed with signal 11 (core dumped) Oct 11 13:25:03 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14184, TLS, session= Oct 11 13:25:03 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:03 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14184 killed with signal 11 (core dumped) Oct 11 13:25:09 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:09 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14174 killed with signal 11 (core dumped) [repeats] Oct 11 13:25:27 halo dovecot: dns-client: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: dns-client: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: master: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: auth: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: config: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: ssl-params: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: anvil: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: log: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) gdb backtrace: ~ >gdb /usr/lib/dovecot/imap /var/cache/imapproxy/jdoe/core GNU gdb (GDB) 7.5-ubuntu Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug/usr/lib/dovecot/imap...done. done. [New LWP 13939] warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `dovecot/imap'. Program terminated with signal 11, Segmentation fault. #0 0x0000000000000000 in ?? () (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00007fc7f6cb611e in imap_parser_reset (parser=0x7fc7f8a0f3a0) at imap-parser.c:93 No locals. #2 0x00007fc7f6f7ada7 in imapc_connection_input_reset (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:664 No locals. #3 0x00007fc7f6f7c6f4 in imapc_connection_input_untagged (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:908 imap_args = 0x7fc7f8a0f4f8 name = 0x7fc7f8a0f5d0 "" value = parser = 0x0 reply = {name = 0x7fc7f8a0f5d0 "", num = 11, args = 0x7fc7f8a0f4f8, file_args = 0x7fc7f8a0d5d0, file_args_count = 0, resp_text_key = 0x0, resp_text_value = 0x0, untagged_box_context = 0x7fc7f8a1ad70} ret = #4 0x00007fc7f6f7d25e in imapc_connection_input_one (conn=0x7fc7f8a0d270) at imapc-connection.c:1061 tag = 0x7fc7f8a0f5c0 "" ret = -1 #5 imapc_connection_input_pending (conn=0x7fc7f8a0d270) at imapc-connection.c:1407 _data_stack_cur_id = 6 ret = #6 0x00007fc7f6f7d2c2 in imapc_connection_input (conn=0x7fc7f6c8f798) at imapc-connection.c:1100 errstr = ret = #7 0x00007fc7f6cdf006 in io_loop_call_io (io=0x7fc7f8a23800) at ioloop.c:379 ioloop = 0x7fc7f8a23630 t_id = 5 #8 0x00007fc7f6cdfcb7 in io_loop_handler_run (ioloop=ioloop at entry=0x7fc7f8a23630) at ioloop-epoll.c:213 ctx = 0x7fc7f8a23850 events = 0x7fc7f8a245e0 event = 0x7fc7f8a238c0 list = 0x7fc7f8a24320 io = tv = {tv_sec = 299, tv_usec = 999402} events_count = msecs = ret = 1 i = call = #9 0x00007fc7f6cdea18 in io_loop_run (ioloop=0x7fc7f8a23630) at ioloop.c:398 No locals. #10 0x00007fc7f6f7a0f7 in imapc_client_run_pre (client=) at imapc-client.c:142 connp = 0x7fc7f8a0cfe0 prev_ioloop = 0x7fc7f89e3670 #11 imapc_client_run (client=0x7fc7f8a0ce80) at imapc-client.c:161 No locals. #12 0x00007fc7f6f79254 in imapc_storage_run (storage=0x7fc7f8a0be60) at imapc-storage.c:118 No locals. #13 0x00007fc7f6f78311 in imapc_sync_index (ctx=0x7fc7f8a1fc70) at imapc-sync.c:351 mbox = 0x7fc7f8a1ad70 sync_rec = {uid1 = 47, uid2 = 0, type = (unknown: 4171272512), add_flags = 199 '\307', remove_flags = 127 '\177', keyword_idx = 0, guid_128 = "\000\000\000\000)\235\312\366\307\177\000\000p\255\241", } seq1 = 32767 seq2 = 0 #14 imapc_sync_begin (force=, ctx_r=, mbox=0x7fc7f8a1ad70) at imapc-sync.c:422 ctx = 0x7fc7f8a1fc70 sync_flags = ret = #15 imapc_sync (mbox=0x7fc7f8a1ad70) at imapc-sync.c:464 sync_ctx = force = #16 imapc_mailbox_sync_init (box=0x7fc7f8a1ad70, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at imapc-sync.c:498 mbox = 0x7fc7f8a1ad70 capabilities = changes = false ret = #17 0x00007fc7f6f8bd43 in mailbox_sync_init (box=0x7fc7f8a1ad70, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at mail-storage.c:1320 _data_stack_cur_id = 4 ctx = #18 0x00007fc7f6f8be67 in mailbox_sync (box=, flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at mail-storage.c:1368 ctx = status = {sync_delayed_expunges = 0} #19 0x00007fc7f74475d2 in select_open (readonly=false, mailbox=, ctx=0x7fc7f8a0da98) at cmd-select.c:296 client = 0x7fc7f8a13d30 status = {messages = 4171084000, recent = 32711, unseen = 0, uidvalidity = 0, uidnext = 13, first_unseen_seq = 0, first_recent_uid = 4137782496, last_cached_seq = 32711, highest_modseq = 13, keywords = 0x7fc7f6cd06fb , permanent_flags = 4171118192, nonpermanent_modseqs = 1, permanent_keywords = 1, allow_new_keywords = 1} flags = MAILBOX_FLAG_DROP_RECENT ret = #20 cmd_select_full (cmd=, readonly=) at cmd-select.c:419 ---Type to continue, or q to quit--- client = ctx = args = 0x7fc7f8a18598 list_args = 0x7fc7f89db0e0 mailbox = 0x7fc7f89db310 "Trash" ret = 1 __FUNCTION__ = "cmd_select_full" #21 0x00007fc7f744b29c in command_exec (cmd=cmd at entry=0x7fc7f8a0d9a0) at imap-commands.c:148 hook = 0x7fc7f89e4cd0 ret = #22 0x00007fc7f744a2ee in client_command_input (cmd=0x7fc7f6c8f798) at imap-client.c:682 client = 0x7fc7f8a13d30 command = __FUNCTION__ = "client_command_input" #23 0x00007fc7f744a39a in client_command_input (cmd=0x7fc7f8a0d9a0) at imap-client.c:733 client = 0x7fc7f8a13d30 command = __FUNCTION__ = "client_command_input" #24 0x00007fc7f744a5fd in client_handle_next_command (remove_io_r=, client=0x7fc7f8a13d30) at imap-client.c:774 size = 19 #25 client_handle_input (client=client at entry=0x7fc7f8a13d30) at imap-client.c:786 _data_stack_cur_id = 3 ret = 112 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #26 0x00007fc7f744aef5 in client_input (client=0x7fc7f8a13d30) at imap-client.c:825 cmd = output = 0x7fc7f8a0d868 bytes = 19 __FUNCTION__ = "client_input" #27 0x00007fc7f6cdf006 in io_loop_call_io (io=0x7fc7f8a11c90) at ioloop.c:379 ioloop = 0x7fc7f89e3670 t_id = 2 #28 0x00007fc7f6cdfcb7 in io_loop_handler_run (ioloop=ioloop at entry=0x7fc7f89e3670) at ioloop-epoll.c:213 ctx = 0x7fc7f89e39e0 events = 0x7fc7f8a245e0 event = 0x7fc7f89e3a50 list = 0x7fc7f8a0f380 io = tv = {tv_sec = 1739, tv_usec = 996790} events_count = msecs = ret = 1 i = call = #29 0x00007fc7f6cdea18 in io_loop_run (ioloop=0x7fc7f89e3670) at ioloop.c:398 No locals. #30 0x00007fc7f6ccb463 in master_service_run (service=0x7fc7f89e3520, callback=callback at entry=0x7fc7f7452f70 ) at master-service.c:544 No locals. #31 0x00007fc7f74428c7 in main (argc=1, argv=0x7fc7f89e3370) at main.c:389 set_roots = {0x7fc7f7658d20 , 0x0} login_set = {auth_socket_path = 0x7fc7f89db070 "/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x7fc7f7452e10 , failure_callback = 0x7fc7f7452b60 } service_flags = storage_service_flags = username = 0x0 c = (gdb) config: >doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-17-generic x86_64 Ubuntu quantal (development branch) first_valid_gid = 999 first_valid_uid = 999 imapc_host = backend.example.com last_valid_gid = 999 last_valid_uid = 999 log_timestamp = "%d.%m.%Y %H.%M.%S " login_greeting = dovecot ready mail_gid = imapproxy mail_home = /var/cache/imapproxy/%u mail_location = imapc:%h/%n/Maildir mail_uid = imapproxy passdb { args = host=backend.example.com default_fields = userdb_imapc_user=%u userdb_imapc_password=%w driver = imap } protocols = " imap" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } ssl = required ssl_cert = References: <50757A8F.4030508@adminlinux.com.br> <5B596E8E-0721-4EE9-81E2-FC7F308B3C5C@iki.fi> Message-ID: <50771D40.5000105@rename-it.nl> On 10/10/2012 9:37 PM, Timo Sirainen wrote: > On 10.10.2012, at 16.39, 3.listas at adminlinux.com.br wrote: > >> I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. >> >> There is a lot of logs of "quota exceeded" like this: >> >> Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) >> >> These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. >> >> Is there a way to send specific Dovecot errors on specific files or just discard them? > Upgrade to v2.1, they are logged with info level there. (I think v2.0 also logs them with info level if you don't use Sieve.) Nope, that is unfortunately not going to help right now. At least not until the following change I made yesterday is released: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/5c1ce25596ed Of course you can patch it if you're in a hurry. Regards, Stephan. From tss at iki.fi Thu Oct 11 22:48:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 22:48:34 +0300 Subject: [Dovecot] memory allocation in new thread In-Reply-To: <1349950869.22650094@f123.mail.ru> References: <1349950869.22650094@f123.mail.ru> Message-ID: <66C27C1C-8C0F-4835-9E73-CB5D22DFCB3F@iki.fi> On 11.10.2012, at 13.21, . . wrote: > Hi! I have some problems with memory allocation. > I create new thread in cidir storage and call malloc(), and it fails to allocate even 1 byte. > What can cause this problem? No idea. Dovecot in general isn't designed to work with threads. > Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc: malloc() succeeded > Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc_func: malloc() failed Well, or the one thing I can think of you to try: Set default_vsz_limit=0 Also straceing the process could show what exactly fails. From tss at iki.fi Thu Oct 11 22:52:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 22:52:01 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210110643.q9B6hIF6003121@bongo.freakout.de> References: <201210110643.q9B6hIF6003121@bongo.freakout.de> Message-ID: On 11.10.2012, at 9.43, dovecot at freakout.de wrote: > Core was generated by `/opt/dovecot/sbin/dovecot'. OK.. > #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 > #1 0xb7f131c0 in master_instance_list_add_line (list=0x9d48880, line=0x9d540c8 "1349762052\tdovecot\t/var/dovecot/run\t") > at master-instance.c:85 Dovecot code is calling str_to_time() from libmysqlclient, instead of from Dovecot's internal code. Not the first time mysql conflicted with Dovecot code. This could be worked around, but .. why is your dovecot binary linked with libmysqlclient? Only auth and dict binaries should be. From tss at iki.fi Thu Oct 11 23:10:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 23:10:56 +0300 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <50752C97.1010209@froglogic.com> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> Message-ID: <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> On 10.10.2012, at 11.06, Frerich Raabe wrote: > I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: > > namespace public { > separator = / > prefix = Lists/ > location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > subscriptions = no > } > > Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) You'll need v2.2 and its INDEXPVT setting. From stephan at rename-it.nl Thu Oct 11 23:35:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 11 Oct 2012 22:35:21 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50753E85.5060904@dada.eu> References: <50753E85.5060904@dada.eu> Message-ID: <50772D89.4050601@rename-it.nl> On 10/10/2012 11:23 AM, Sandro Tosi wrote: > Hello, > we're scouting if it's possible to use Pigeonhole (currently v0.3.1, > as this will be provided with an upcoming Debian package) with MySQL > dict lookups with the mail setup we're designing. > > Our (main) goals are: > > 1. store the filters on the database That is possible with some limitations. > 2. allow each user to enable/disable any of the filters set we provide > (it's a static set of some general filters, available to all the > users; we're currently not providing the possibility to users to write > their own filters) Will one or multiple scripts be active at the same time? > For point 1) we already see[1] that's possible, but it uses the map > construct that might not fit with our current database structure: we > have a domain table (storing the domain info) and a mailbox table > (storing the mailbox info, but the username is composed by the local > part, stored in this table, and the domain part is a FK to the domain > table, using an id). > > Do you think it's possible to run a join query on domain+mailbox to > retrieve the mailbox_id needed to query the table for the filters? Or > do we have to create the filter table and store the local at domain.ext > info there ("relaxing" the integrity relationships between tables)? My SQL is a bit rusty, but afaik this is possible with a JOIN or a nested query. > How do we specify which filters are enabled for any given user? We > originally thought of an "Enabled" field on the filter table, but in > the example in the doc[1] I hadn't seen a way to do that: it seems > like the filter list is specified in the proxy definition - am I > wrong? How can we do that? The above suggests that you would like to activate multiple Sieve scripts at the same time. That is currently not possible with the dict Script location. It is on my TODO list, but I am not sure when it will be ready (definitely not for coming Debian stable). For Dovecot v2.2 the new :optional tag for the Sieve include command could be used in - combination with the dict Sieve script location type - to provide some hackish solution. Unfortunately, in your case that is still not helpful, because v2.2 is not even in beta stage. :/ Regards, Stephan. From raabe at froglogic.com Thu Oct 11 23:38:51 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 11 Oct 2012 22:38:51 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> Message-ID: Am 11.10.2012 um 22:10 schrieb Timo Sirainen: > On 10.10.2012, at 11.06, Frerich Raabe wrote: >> I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: >> >> namespace public { >> separator = / >> prefix = Lists/ >> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >> subscriptions = no >> } >> >> Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) > > You'll need v2.2 and its INDEXPVT setting. Hm, you mean the feature introduced by http://hg.dovecot.org/dovecot-2.2/rev/dbd42f7198eb ? Is there some discussion of the feature somewhere? The commit log is a bit unclear to me, it says 'Per-user flags can now be stored in private index files.' however http://wiki2.dovecot.org/SharedMailboxes/Public says 'By making each user have their own private index files, you can make the \Seen flag private for the users.' (using the INDEX setting). Makes me wonder - the Wiki talks about 'private index files' when talking about 'INDEX' and the commit says 'private index files' talking about INDEXPVT - what is the difference? :-) -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dave.mehler at gmail.com Thu Oct 11 23:40:11 2012 From: dave.mehler at gmail.com (David Mehler) Date: Thu, 11 Oct 2012 16:40:11 -0400 Subject: [Dovecot] per-user quotas Message-ID: Hello, I've got quotas set up on an all-user basis on my system, it's a Postfix, Dovecot, Mysql virtual users setup. Currently I have each user getting a 1GB quota with these settings in 90-quota.conf: plugin { quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M } plugin { quota = maildir:User quota } While this works it's not what I want for all users. Say I add a virtual user called user1 to the mysql database but he's a test user and I only want that user to have 25 megabytes of storage, reading the wiki on quotas per-user items such as for example in a database overrides the global items above, is this right? If so, I'm hoping I'm not going to have to redo my entire user database, some users will have per-user quotas while I'll let others have the global quota. Thanks. Dave. From tss at iki.fi Thu Oct 11 23:46:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 23:46:01 +0300 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> Message-ID: <99F7FC09-E21F-4808-9796-E2AAC31CDED0@iki.fi> On 11.10.2012, at 23.38, Frerich Raabe wrote: > Am 11.10.2012 um 22:10 schrieb Timo Sirainen: >> On 10.10.2012, at 11.06, Frerich Raabe wrote: >>> I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: >>> >>> namespace public { >>> separator = / >>> prefix = Lists/ >>> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >>> subscriptions = no >>> } >>> >>> Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) >> >> You'll need v2.2 and its INDEXPVT setting. > > Hm, you mean the feature introduced by http://hg.dovecot.org/dovecot-2.2/rev/dbd42f7198eb ? Yes. > Is there some discussion of the feature somewhere? http://markmail.org/message/45jxf363ffrubonv has some. > The commit log is a bit unclear to me, it says 'Per-user flags can now be stored in private index files.' however http://wiki2.dovecot.org/SharedMailboxes/Public says 'By making each user have their own private index files, you can make the \Seen flag private for the users.' (using the INDEX setting). > > Makes me wonder - the Wiki talks about 'private index files' when talking about 'INDEX' and the commit says 'private index files' talking about INDEXPVT - what is the difference? :-) You can have both! Shared indexes having the shared stuff (including squat indexes), while the private indexes only have the per-user flags, nothing else. For example with sdbox/mdbox you couldn't even have set per-user INDEX location or it would have just broken. From tibby at tibby.hu Thu Oct 11 23:49:11 2012 From: tibby at tibby.hu (Tibby) Date: Thu, 11 Oct 2012 22:49:11 +0200 Subject: [Dovecot] per-user quotas In-Reply-To: References: Message-ID: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> Hello! What's the version of your dovecot? 1.2.X ? or 2.0 ? Tibby On Oct 11, 2012, at 10:40 PM, David Mehler wrote: > Hello, > > I've got quotas set up on an all-user basis on my system, it's a > Postfix, Dovecot, Mysql virtual users setup. Currently I have each > user getting a 1GB quota with these settings in 90-quota.conf: > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } > plugin { > quota = maildir:User quota > } > > While this works it's not what I want for all users. Say I add a > virtual user called user1 to the mysql database but he's a test user > and I only want that user to have 25 megabytes of storage, reading the > wiki on quotas per-user items such as for example in a database > overrides the global items above, is this right? > > If so, I'm hoping I'm not going to have to redo my entire user > database, some users will have per-user quotas while I'll let others > have the global quota. > > Thanks. > Dave. From dave.mehler at gmail.com Fri Oct 12 00:43:57 2012 From: dave.mehler at gmail.com (David Mehler) Date: Thu, 11 Oct 2012 17:43:57 -0400 Subject: [Dovecot] per-user quotas In-Reply-To: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> References: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> Message-ID: Hello, I'm running Dovecot 2.1.10. Thanks. Dave. On 10/11/12, Tibby wrote: > Hello! > > What's the version of your dovecot? 1.2.X ? or 2.0 ? > > Tibby > > On Oct 11, 2012, at 10:40 PM, David Mehler wrote: > >> Hello, >> >> I've got quotas set up on an all-user basis on my system, it's a >> Postfix, Dovecot, Mysql virtual users setup. Currently I have each >> user getting a 1GB quota with these settings in 90-quota.conf: >> >> plugin { >> quota_rule = *:storage=1G >> quota_rule2 = Trash:storage=+100M >> } >> plugin { >> quota = maildir:User quota >> } >> >> While this works it's not what I want for all users. Say I add a >> virtual user called user1 to the mysql database but he's a test user >> and I only want that user to have 25 megabytes of storage, reading the >> wiki on quotas per-user items such as for example in a database >> overrides the global items above, is this right? >> >> If so, I'm hoping I'm not going to have to redo my entire user >> database, some users will have per-user quotas while I'll let others >> have the global quota. >> >> Thanks. >> Dave. > > From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 02:01:36 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 01:01:36 +0200 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074CAC2.2030507@perkel.com> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> <5074CAC2.2030507@perkel.com> Message-ID: <20121011230136.GA9153@daniel.localdomain> Hi Marc, Marc Perkel wrote: > On 10/9/2012 3:42 PM, Daniel Parthey wrote: > >Marc Perkel wrote: > >>if the mail location doesn't exist > >>then I want to try a second mail location: > >>mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >You might do this with a script which exports the MAIL environment > >variable and then executes the service binary: > >http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection > But how do I pick up the name and domain parameters to test the directory? You can get the username and any other userdb value from the environment, have a look at the environment section: http://wiki2.dovecot.org/PostLoginScripting#Running_environment Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 02:24:08 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 01:24:08 +0200 Subject: [Dovecot] Feature Request In-Reply-To: <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> <5074E01B.8030001@perkel.com> <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> Message-ID: <20121011232408.GA9444@daniel.localdomain> Timo Sirainen wrote: > On 10.10.2012, at 5.40, Marc Perkel wrote: > > >>> It would be handy (for me) if there were a userdb where a directory structure defined the db. > >>> > >>> userdb stat { > >>> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >>> } > >>> > >>> userdb stat { > >>> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >>> } > >>> > >>> The idea being that if the first directory doesn't exist then it will try the second one. > >> You could already implement this as userdb checkpassword script. > > > > Can you give me an example? > > Something like this: > > userdb { > driver = checkpasword > args = /usr/local/bin/userdb.sh > } Here is the documentation about how to implement a checkpassword script: http://wiki2.dovecot.org/AuthDatabase/CheckPassword Dovecot sets some environment variables that the script may use. All of the AUTH_* variables are available as AUTH_ extra fields in the environment: http://wiki2.dovecot.org/Variables#line-30 Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 03:15:39 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 02:15:39 +0200 Subject: [Dovecot] per-user quotas In-Reply-To: References: Message-ID: <20121012001539.GA10473@daniel.localdomain> Hi Dave, David Mehler wrote: > I've got quotas set up on an all-user basis on my system, it's a > Postfix, Dovecot, Mysql virtual users setup. Currently I have each > user getting a 1GB quota with these settings in 90-quota.conf: > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } > plugin { > quota = maildir:User quota > } > > While this works it's not what I want for all users. Say I add a > virtual user called user1 to the mysql database but he's a test user > and I only want that user to have 25 megabytes of storage, reading the > wiki on quotas per-user items such as for example in a database > overrides the global items above, is this right? > > If so, I'm hoping I'm not going to have to redo my entire user > database, some users will have per-user quotas while I'll let others > have the global quota. You can just make your SQL query a bit more sophisticated in order to fit your needs. MySQL supports SQL CASE statement and default value with ELSE: http://dev.mysql.com/doc/refman/5.1/en/case.html This example sets quota to unlimited if mail comes in via port 20025, otherwise is uses the quota_bytes and quota_message columns: user_query = SELECT username AS user, \ home as home, \ uid as uid, \ gid as gid, \ CASE '%a' \ WHEN '20025' THEN '*:bytes=0:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota_bytes AS CHAR), ':messages=', CAST(quota_message AS CHAR)) \ END AS `quota_rule` \ FROM dovecot_users \ WHERE username='%u' Regards Daniel -- https://plus.google.com/103021802792276734820 From dovecot at freakout.de Fri Oct 12 09:50:16 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Fri, 12 Oct 2012 08:50:16 +0200 (CEST) Subject: [Dovecot] dovecot cores Message-ID: <201210120650.q9C6oGSG005182@bongo.freakout.de> According to Timo Sirainen: > On 11.10.2012, at 9.43, dovecot at freakout.de wrote: > > Dovecot code is calling str_to_time() from libmysqlclient, > instead of from Dovecot's internal code. > Not the first time mysql conflicted with Dovecot code. > This could be worked around, but .. why is your dovecot > binary linked with libmysqlclient? > Only auth and dict binaries should be. > but dovecot's configure script does not allow to specify the mysql libs and headers explictly - only by global CPPFLAGS and LDFLAGS extensions, which are used for all binaries - when i tried to specify: ./configure --prefix=/opt/dovecot --sysconfdir=/etc/dovecot --mandir=/opt/dovecot/man \ --docdir=/opt/dovecot/doc --libexecdir=/opt/dovecot/sbin --datadir=/opt/dovecot \ --with-rundir=/var/dovecot/run --with-statedir=/var/dovecot/state \ --with-mysql=/opt/mysql/bin/mysql_config checking for ... checking pkg-config is at least version 0.9.0... yes configure: error: --with-mysql=path not supported. You may want to use instead: CPPFLAGS=-I/opt/mysql/bin/mysql_config/include LDFLAGS=-L/opt/mysql/bin/mysql_config/lib ./configure --with-mysql i followed the hint from the configure script above and run into the core-dumps due to symbol clash str_to_time. How to work around with mysql in non-standard location? Thanks Axel From dovecot-list at mohtex.net Fri Oct 12 10:25:59 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 14:25:59 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120650.q9C6oGSG005182@bongo.freakout.de> References: <201210120650.q9C6oGSG005182@bongo.freakout.de> Message-ID: <5077C607.3010308@mohtex.net> dovecot at freakout.de wrote the following on 12.10.2012 13:50: > According to Timo Sirainen: >> On 11.10.2012, at 9.43, dovecot at freakout.de wrote: >> >> Dovecot code is calling str_to_time() from libmysqlclient, >> instead of from Dovecot's internal code. >> Not the first time mysql conflicted with Dovecot code. >> This could be worked around, but .. why is your dovecot >> binary linked with libmysqlclient? >> Only auth and dict binaries should be. >> > but dovecot's configure script does not allow to specify the > mysql libs and headers explictly - only by global CPPFLAGS and > LDFLAGS extensions, which are used for all binaries - when i > tried to specify: > > ./configure --prefix=/opt/dovecot --sysconfdir=/etc/dovecot --mandir=/opt/dovecot/man \ > --docdir=/opt/dovecot/doc --libexecdir=/opt/dovecot/sbin --datadir=/opt/dovecot \ > --with-rundir=/var/dovecot/run --with-statedir=/var/dovecot/state \ > --with-mysql=/opt/mysql/bin/mysql_config > checking for ... > checking pkg-config is at least version 0.9.0... yes > configure: error: --with-mysql=path not supported. You may want to use instead: > CPPFLAGS=-I/opt/mysql/bin/mysql_config/include LDFLAGS=-L/opt/mysql/bin/mysql_config/lib ./configure --with-mysql > > i followed the hint from the configure script above and run > into the core-dumps due to symbol clash str_to_time. > > How to work around with mysql in non-standard location? > > Thanks > Axel This one works for me for mysql in a non-standard location (my.cnf is in /etc): CPPFLAGS='-I/opt/mysql/include/mysql' LDFLAGS='-L/opt/mysql/lib/mysql -lmysqlclient -lz -lcrypt -lnsl -lm' ./configure --with-mysql ..... Rds Tamsy From tss at iki.fi Fri Oct 12 10:34:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:34:08 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120702.q9C72saS005717@bongo.freakout.de> References: <201210120702.q9C72saS005717@bongo.freakout.de> Message-ID: <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > According to Timo Sirainen: >> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? >> > > ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" > LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" -lmysqlclient shouldn't be in LDFLAGS. > if i omit "-lmysqlclient" (seems to be the reason for the hassle) i get: > > libtool: link: gcc4 -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include -o .libs/auth auth.o auth-cache.o auth-client-connection.o > auth-master-connection.o auth-postfix-connection.o mech-otp-skey-common.o mech-plain-common.o auth-penalty.o auth-request.o auth-request-handler.o auth-settings.o auth-stream.o auth-worker-client.o auth-worker-server.o db-checkpassword.o db-dict.o db-sql.o db-passwd-file.o main.o mech.o mech-anonymous.o mech-plain.o mech-login.o > mech-cram-md5.o mech-digest-md5.o mech-external.o mech-gssapi.o mech-ntlm.o mech-otp.o mech-scram-sha1.o mech-skey.o mech-rpa.o mech-apop.o mech-winbind.o passdb.o passdb-blocking.o passdb-bsdauth.o passdb-cache.o passdb-checkpassword.o passdb-dict.o passdb-passwd.o passdb-passwd-file.o passdb-pam.o passdb-shadow.o passdb-sia.o > passdb-vpopmail.o passdb-sql.o passdb-static.o passdb-template.o userdb.o userdb-blocking.o userdb-checkpassword.o userdb-dict.o userdb-nss.o userdb-passwd.o userdb-passwd-file.o userdb-prefetch.o userdb-static.o userdb-vpopmail.o userdb-sql.o userdb-template.o db-ldap.o passdb-ldap.o userdb-ldap.o -Wl,--export-dynamic -L/opt > /zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib libpassword.a ../lib-ntlm/libntlm.a ../lib-otp/libotp.a ../../src/lib-sql/.libs/libsql.a ../../src/lib-dovecot/.libs/libdovecot.so -lcrypt -ldl -Wl,-rpath -Wl,/opt/dovecot/lib > ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': > /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' >>>> ... tons of other undefined reference to mysqlclient > /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' > collect2: error: ld returned 1 exit status I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. From tss at iki.fi Fri Oct 12 10:39:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:39:07 +0300 Subject: [Dovecot] imap proxy setup - "killed with signal 11" In-Reply-To: <50770344.70905@bitrate.net> References: <50770344.70905@bitrate.net> Message-ID: <560FF304-5D04-44D1-AEC2-8DE8DC3F0943@iki.fi> On 11.10.2012, at 20.35, btb wrote: > i'm setting up an imap proxy in front of a novell groupwise server. it seems to so far be partially working, but dovecot is having trouble in certain cases. i expect that it's ultimately due to what i believe is a very poor implementation of imap provided by groupwise [at least based on other experiences in the past] - but that's a big part of why i'd like to have dovecot in between it and clients. > > os is ubuntu 12.10 development/beta, dovecot is 2.1.7 courtesy of ubuntu's packages. There have been a couple of imapc fixes since v2.1.7. It's possible that the crash is fixed by one of them. > Oct 11 13:24:52 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Could you get imapc rawlogs where this happens? Point imapc_rawlog_dir setting to some directory. > #0 0x0000000000000000 in ?? () > No symbol table info available. > #1 0x00007fc7f6cb611e in imap_parser_reset (parser=0x7fc7f8a0f3a0) at imap-parser.c:93 > No locals. > #2 0x00007fc7f6f7ada7 in imapc_connection_input_reset (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:664 > No locals. > #3 0x00007fc7f6f7c6f4 in imapc_connection_input_untagged (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:908 This backtrace unfortunately doesn't make it very clear what the problem is. I'd guess it's trying to use already freed memory (one such bug was already fixed). From tss at iki.fi Fri Oct 12 10:40:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:40:36 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5075881C.4060905@brightok.net> References: <5075881C.4060905@brightok.net> Message-ID: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> On 10.10.2012, at 17.37, Jack Bates wrote: > The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. > > My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. From tss at iki.fi Fri Oct 12 10:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:45:05 +0300 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: References: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> Message-ID: On 10.10.2012, at 15.17, Natanael Copa wrote: > On Wed, Oct 10, 2012 at 12:22 PM, Timo Sirainen wrote: >> On 10.10.2012, at 13.00, Natanael Copa wrote: >> >>> Running doveadm on Alpine Linux will show various underlinking errors: > >> With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. > > Why does it need to load all the plugins? Why not only try to load > those who are enabled? doveadm has two types of commands: mail commands and non-mail commands. The mail_plugins can add new mail features, and doveadm plugins can add more doveadm commands, which use the new mail features. But doveadm can also have plugins that add non-mail commands, which don't need anything in mail_plugins. So both cases would need to work.. From dovecot-list at mohtex.net Fri Oct 12 10:45:25 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 14:45:25 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> Message-ID: <5077CA95.1060506@mohtex.net> Timo Sirainen wrote the following on 12.10.2012 14:34: > On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > >> According to Timo Sirainen: >>> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? >>> >> ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" >> LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" > -lmysqlclient shouldn't be in LDFLAGS. > >> if i omit "-lmysqlclient" (seems to be the reason for the hassle) i get: >> >> libtool: link: gcc4 -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include -o .libs/auth auth.o auth-cache.o auth-client-connection.o >> auth-master-connection.o auth-postfix-connection.o mech-otp-skey-common.o mech-plain-common.o auth-penalty.o auth-request.o auth-request-handler.o auth-settings.o auth-stream.o auth-worker-client.o auth-worker-server.o db-checkpassword.o db-dict.o db-sql.o db-passwd-file.o main.o mech.o mech-anonymous.o mech-plain.o mech-login.o >> mech-cram-md5.o mech-digest-md5.o mech-external.o mech-gssapi.o mech-ntlm.o mech-otp.o mech-scram-sha1.o mech-skey.o mech-rpa.o mech-apop.o mech-winbind.o passdb.o passdb-blocking.o passdb-bsdauth.o passdb-cache.o passdb-checkpassword.o passdb-dict.o passdb-passwd.o passdb-passwd-file.o passdb-pam.o passdb-shadow.o passdb-sia.o >> passdb-vpopmail.o passdb-sql.o passdb-static.o passdb-template.o userdb.o userdb-blocking.o userdb-checkpassword.o userdb-dict.o userdb-nss.o userdb-passwd.o userdb-passwd-file.o userdb-prefetch.o userdb-static.o userdb-vpopmail.o userdb-sql.o userdb-template.o db-ldap.o passdb-ldap.o userdb-ldap.o -Wl,--export-dynamic -L/opt >> /zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib libpassword.a ../lib-ntlm/libntlm.a ../lib-otp/libotp.a ../../src/lib-sql/.libs/libsql.a ../../src/lib-dovecot/.libs/libdovecot.so -lcrypt -ldl -Wl,-rpath -Wl,/opt/dovecot/lib >> ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': >> /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' >>>>> ... tons of other undefined reference to mysqlclient >> /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' >> collect2: error: ld returned 1 exit status > I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). From tss at iki.fi Fri Oct 12 10:50:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:50:06 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <5077CA95.1060506@mohtex.net> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> <5077CA95.1060506@mohtex.net> Message-ID: <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> On 12.10.2012, at 10.45, Tamsy wrote: > Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". > > Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). If you run configure without adding the -lmysqlclient, what do you get with: egrep -i 'mysql|auth_libs|sql_libs' Makefile I guess the problem is that I shouldn't have copy&pasted the mysql detection code from php, and configure somehow passes successfully without actually setting any MYSQL_LIBS.. From dovecot-list at mohtex.net Fri Oct 12 11:04:46 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 15:04:46 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> <5077CA95.1060506@mohtex.net> <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> Message-ID: <5077CF1E.4060809@mohtex.net> Timo Sirainen wrote the following on 12.10.2012 14:50: > On 12.10.2012, at 10.45, Tamsy wrote: > >> Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". >> >> Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). > If you run configure without adding the -lmysqlclient, what do you get with: > > egrep -i 'mysql|auth_libs|sql_libs' Makefile > > I guess the problem is that I shouldn't have copy&pasted the mysql detection code from php, and configure somehow passes successfully without actually setting any MYSQL_LIBS.. Just ran configure without adding the -lmysqlclient (CPPFLAGS='-I/opt/mysql/include/mysql' LDFLAGS='-L/opt/mysql/lib/mysql -lz -lcrypt -lnsl -lm' ./configure --with-mysql.... egrep -i 'mysql|auth_libs|sql_libs' Makefile says: AUTH_LIBS = CPPFLAGS = -I/opt/mysql/include/mysql LDFLAGS = $(NOPLUGIN_LDFLAGS) -L/opt/mysql/lib/mysql -lz -lcrypt -lnsl -lm MYSQL_CFLAGS = MYSQL_CONFIG = NO MYSQL_LIBS = PGSQL_LIBS = SQL_LIBS = sql_drivers = mysql scan-build -o scan-reports ../configure --with-ldap=auto --with-pgsql=auto --with-mysql=auto --with-sqlite=auto --with-solr=auto --with-gssapi=auto --with-libwrap=auto; \ configure runs without error but make ends with: ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_get_error': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:469: undefined reference to `mysql_errno' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_get_field_value_binary': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:436: undefined reference to `mysql_fetch_lengths' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_fetch_fields': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:383: undefined reference to `mysql_num_fields' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:384: undefined reference to `mysql_fetch_fields' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_next_row': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:366: undefined reference to `mysql_fetch_row' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:370: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_free': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:351: undefined reference to `mysql_free_result' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_do_query': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:233: undefined reference to `mysql_query' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:237: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_query_s': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:315: undefined reference to `mysql_affected_rows' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:316: undefined reference to `mysql_store_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:321: undefined reference to `mysql_next_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:332: undefined reference to `mysql_free_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:327: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_exec': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:284: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_escape_string': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:274: undefined reference to `mysql_real_escape_string' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:268: undefined reference to `mysql_escape_string' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:87: undefined reference to `mysql_options' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:92: undefined reference to `mysql_ssl_set' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:110: undefined reference to `mysql_real_connect' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:123: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_deinit_v': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:226: undefined reference to `mysql_close' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_parse_connect_string': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:198: undefined reference to `mysql_init' collect2: ld returned 1 exit status make[3]: *** [auth] Error 1 make[3]: Leaving directory `/usr/local/src/dovecot-2.1.10/src/auth' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/dovecot-2.1.10/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/dovecot-2.1.10' make: *** [all] Error 2 From dovecot at freakout.de Fri Oct 12 11:10:20 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Fri, 12 Oct 2012 10:10:20 +0200 (CEST) Subject: [Dovecot] dovecot cores In-Reply-To: <5077CA95.1060506@mohtex.net> Message-ID: <201210120810.q9C8AK7V007314@bongo.freakout.de> According to Tamsy: > Timo Sirainen wrote the following on 12.10.2012 14:34: > > On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > > > >> According to Timo Sirainen: > >>> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? > >>> > >> ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" > >> LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" > > -lmysqlclient shouldn't be in LDFLAGS. > > > > I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. > Axel, please let us know whether one of these works: > "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or > MYSQL_LIBS=-lmysqlclient". > > Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and > only by putting -lmysqlclient in LDFLAGS as described before Dovecot > compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a > non-standart location). > SQL_LIBS=-lmysqlclient => not working AUTH_LIBS=-lmysqlclient => not working MYSQL_LIBS=-lmysqlclient \ LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib" \ ./configure --prefix=%{_prefix} \ --sysconfdir=%{_etcdir} --mandir=%{_mandir} --docdir=%{_docdir} --libexecdir=%{_sbindir} --datadir=%{_prefix} \ --with-rundir=/var/dovecot/run \ --with-statedir=/var/dovecot/state \ --with-mysql => WORKING BUT: [axel at joe rpm]$ ldd BUILD/dovecot-2.1.10-root/opt/dovecot-2.1.10-5/sbin/dovecot libdovecot.so.0 => /opt/dovecot/lib/libdovecot.so.0 (0x00993000) libgcc_s.so.1 => /opt/gcc4/lib/libgcc_s.so.1 (0x0092c000) libc.so.6 => /lib/libc.so.6 (0x00ebf000) >>>>! libmysqlclient.so.18 => /opt/mysql/lib/libmysqlclient.so.18 (0x001cc000) libdl.so.2 => /lib/libdl.so.2 (0x00ae3000) libssp.so.0 => /opt/ssp/lib/libssp.so.0 (0x0057b000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x002c6000) libstrings.so => /opt/mysql/lib/libstrings.so (0x0057e000) libz.so.1 => /opt/zlib/lib/libz.so.1 (0x00110000) libpthread.so.0 => /lib/libpthread.so.0 (0x00b08000) libm.so.6 => /lib/libm.so.6 (0x00135000) dovecot still seems to be linked with the mysqlclient! i have just compiled - not tried the binaries - the core dump occurs only in the night! > If you run configure without adding the -lmysqlclient, what do you get with: egrep -i 'mysql|auth_libs|sql_libs' Makefile [axel at joe dovecot-2.1.10]$ egrep -i 'mysql|auth_libs|sql_libs' Makefile AUTH_LIBS = -lcrypt -lmysqlclient CFLAGS = -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include LDFLAGS = $(NOPLUGIN_LDFLAGS) -L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib MYSQL_CFLAGS = MYSQL_CONFIG = NO MYSQL_LIBS = -lmysqlclient PGSQL_LIBS = SQL_LIBS = -lmysqlclient sql_drivers = mysql scan-build -o scan-reports ../configure --with-ldap=auto --with-pgsql=auto --with-mysql=auto --with-sqlite=auto --with-solr=auto --with-gssapi=auto --with-libwrap=auto; \ Cheers Axel From kjonca at o2.pl Fri Oct 12 11:01:49 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Fri, 12 Oct 2012 10:01:49 +0200 Subject: [Dovecot] [sieve] - counting headers Message-ID: <87wqyw6rv6.fsf@alfa.kjonca> In some of my maildrop filters I have rules with weighted scoring[1], but only to count headers (for example to count "Received:" header) ie. all these rules are of form "/pattern/:h,1" Can dovecot sieve do this? KJ [1] http://www.courier-mta.org/maildrop/maildropfilter.html -- http://blogdebart.pl/2012/06/24/hiena/ Wiesz, tryb tekstowy w Linuksie ma si? tak do DOSu jak F-117A do paralotni. (c) Dawid Kuroczko From busseniu at in.tum.de Fri Oct 12 17:10:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Fri, 12 Oct 2012 16:10:28 +0200 Subject: [Dovecot] INBOX.INBOX.Sent causes problems in 2.0 Message-ID: <507824D4.7080303@in.tum.de> Hi, we have our namespace rooted at "INBOX.". Sometimes, users have folders like "INBOX.INBOX.Sent" or "INBOX.INBOX.INBOX.Sent". I do not know why these folders are created; I suspect it is due to buggy MUAs. If a mailbox like INBOX.INBOX.Sent exists, then in some cases Dovecot counts all messages in INBOX twice. This behavior can be demonstrated as follows: 1) Create a folder named "INBOX.INBOX.Sent" (using the IMAP CREATE command) 2) Save a message to "INBOX" (in my example the message is 7 MB) 3) dovecot-quota contains: priv/quota/storage 7129025 priv/quota/messages 1 4) doveadm quota recalc -u user1 5) Now dovecot-quota contains priv/quota/storage 14258050 priv/quota/messages 2 I.e. the user will experience that his quota fills up very fast. "doveadm -f flow fetch -u user1 'mailbox guid' ALL" prints: mailbox=INBOX guid=040ce73645177850497d000040c59ffc mailbox=INBOX guid=040ce73645177850497d000040c59ffc However, "doveadm -f flow fetch -u user1 'mailbox guid' mailbox INBOX" prints the message only once: mailbox=INBOX guid=040ce73645177850497d000040c59ffc How can we prevent this kind of confusion? What I'd like most is prevent the creation of these weird folders. Most MUAs have problems listing them properly, especially if both "INBOX.Sent" and "INBOX.INBOX.Sent" exist. Dovecot 2.1 does not seem to count anything twice. Cheers, Christoph Config: # 2.0.21: /usr/local/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (6224) # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mail mail_plugins = quota mail_uid = vmail namespace default { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { quota = dict:ROOT::file:%h/dovecot-quota quota_rule = *:storage=5G } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } service imap { process_limit = 5000 } ssl_cert = Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From thefantaman at gmail.com Fri Oct 12 18:38:59 2012 From: thefantaman at gmail.com (thefantaman) Date: Fri, 12 Oct 2012 08:38:59 -0700 (PDT) Subject: [Dovecot] /var/run/dovecot/auth-userdb failed Message-ID: <1350056339814-38093.post@n4.nabble.com> Hi, I work on test server and if I send an email on log i read lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: /var/run/dovecot/auth-userdb, euid is not dir owner) This is my dovecot.conf: auth_mechanisms = plain login info_log_path = /var/log/dovecot listen = 0.0.0.0 log_path = /var/log/dovecot login_greeting = Dovecot IMAP Server ready. mail_location = maildir:/home/vmail/%d/%u mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-ldap.conf.ext.fabry driver = ldap } protocols = pop3 sieve imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = root group = root } } ssl_cert = HELP! I have installed sendmail, dovecot, and squirrel mail. The squirrel mail portion of it works just fine, but I would like to have Mozilla Thunderbird as a client. Whenever I try and connect to the server it says "Thunderbird failed to find the settings for your email account." We do have an MX record in DNS pointing to our server. We are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 We are doing this for a class project and it is due by Tuesday 10/16/2012.. Thanks, Justin From ben at indietorrent.org Fri Oct 12 20:12:57 2012 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 12 Oct 2012 13:12:57 -0400 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <50784F99.5080201@indietorrent.org> On 10/12/2012 1:09 PM, Justin Vore wrote: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email account." > We do have an MX record in DNS pointing to our server. We are using > Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 > We are doing this for a class project and it is due by Tuesday 10/16/2012.. > > > Thanks, > > > Justin > I have found Thunderbird's automatic setting detection mechanism to be rather unreliable. Try entering the settings manually. -Ben From arne at drlinux.no Fri Oct 12 20:20:46 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Fri, 12 Oct 2012 19:20:46 +0200 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <5078516E.2040505@drlinux.no> Den 12.10.2012 19:09, skrev Justin Vore: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email account." > We do have an MX record in DNS pointing to our server. We are using > Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 > We are doing this for a class project and it is due by Tuesday 10/16/2012.. This explains how to set up autoconfigure in TB. You need a DNS record and a little work on a webhost. https://developer.mozilla.org/en-US/docs/Thunderbird/Autoconfiguration Arne -- Arne K. Haaje http://www.drlinux.no/ ::: arne at drlinux.no LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From lists at kokelnet.de Fri Oct 12 22:48:03 2012 From: lists at kokelnet.de (Tobias Hachmer) Date: Fri, 12 Oct 2012 21:48:03 +0200 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: Am 12.10.2012 19:09, schrieb Justin Vore: > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email > account." > We do have an MX record in DNS pointing to our server. Enter settings manually or set up automx (http://www.automx.org/). Regards, Tobias Hachmer From tss at iki.fi Fri Oct 12 23:07:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 23:07:08 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120810.q9C8AK7V007314@bongo.freakout.de> References: <201210120810.q9C8AK7V007314@bongo.freakout.de> Message-ID: <371D7F4F-3534-4F52-B106-165A487E1828@iki.fi> On 12.10.2012, at 11.10, dovecot at freakout.de wrote: > MYSQL_LIBS=-lmysqlclient \ > LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib" \ > ./configure --prefix=%{_prefix} \ > --sysconfdir=%{_etcdir} --mandir=%{_mandir} --docdir=%{_docdir} --libexecdir=%{_sbindir} --datadir=%{_prefix} \ > --with-rundir=/var/dovecot/run \ > --with-statedir=/var/dovecot/state \ > --with-mysql > => WORKING I think this is the correct fix for this. Basically same as what I committed now: http://hg.dovecot.org/dovecot-2.1/rev/c8d55ba25f39 > BUT: > > [axel at joe rpm]$ ldd BUILD/dovecot-2.1.10-root/opt/dovecot-2.1.10-5/sbin/dovecot > libdovecot.so.0 => /opt/dovecot/lib/libdovecot.so.0 (0x00993000) > libgcc_s.so.1 => /opt/gcc4/lib/libgcc_s.so.1 (0x0092c000) > libc.so.6 => /lib/libc.so.6 (0x00ebf000) >>>>> ! libmysqlclient.so.18 => /opt/mysql/lib/libmysqlclient.so.18 (0x001cc000) > libdl.so.2 => /lib/libdl.so.2 (0x00ae3000) > libssp.so.0 => /opt/ssp/lib/libssp.so.0 (0x0057b000) > /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x002c6000) > libstrings.so => /opt/mysql/lib/libstrings.so (0x0057e000) > libz.so.1 => /opt/zlib/lib/libz.so.1 (0x00110000) > libpthread.so.0 => /lib/libpthread.so.0 (0x00b08000) > libm.so.6 => /lib/libm.so.6 (0x00135000) > > dovecot still seems to be linked with the mysqlclient! Maybe the old LDFLAGS was cached somehow? Or maybe the rpm build does something strange? I don't see how that could happen otherwise. From mark at xwax.org Sat Oct 13 00:20:03 2012 From: mark at xwax.org (Mark Hills) Date: Fri, 12 Oct 2012 22:20:03 +0100 (BST) Subject: [Dovecot] dsync ignores ssh-agent Message-ID: <1210122159360.19545@vega.localdomain> I use IMAP over SSH, in pre-auth. I wanted to use dsync to offline mail to my laptop. dsync v2.1.10 would always ask for my SSH key/passphrase, ignoring ssh-agent. The culprit is the env_clean() in the stack below. Reading the source, I saw DOVECOT_PRESERVE_ENVS. When used as follows in my script it enables dsync to find my ssh-agent: export DOVECOT_PRESERVE_ENVS="SSH_AGENT_PID SSH_AUTH_SOCK" dsync mirror ssh imap.example.com /home/mark/opt/dovecot/bin/dsync and it now works without asking for password every time. I'm posting here so that anyone else googling for the same problem will hopefully find this, as I couldn't find anything about this in the docs. Also I'm interested in why dsync so aggressively cleans the environment; I tried a naive removal of env_clean() but this breaks basic functions. With this dsync is working very well for offline mail -- combined with alpine and a local exim for the outbound queue :) Thanks -- Mark Breakpoint 2, env_clean () at env-util.c:59 59 if (clearenv() < 0) (gdb) bt #0 env_clean () at env-util.c:59 #1 0xb7df10fc in master_service_env_clean () at master-service.c:454 #2 0xb7df26d4 in master_service_exec_config (service=0x809e7d0, input=0xbffff7e4) at master-service-settings.c:103 #3 0xb7df29be in config_exec_fallback (service=0x809e7d0, input=0xbffff7e4) at master-service-settings.c:153 #4 0xb7df2b65 in master_service_open_config (service=0x809e7d0, input=0xbffff7e4, path_r=0xbffff780, error_r=0xbffff7d8) at master-service-settings.c:206 #5 0xb7df3130 in master_service_settings_read (service=0x809e7d0, input=0xbffff7e4, output_r=0xbffff7dc, error_r=0xbffff7d8) at master-service-settings.c:345 #6 0x0805c672 in doveadm_read_settings () at doveadm.c:275 #7 0x0805c7d6 in main (argc=5, argv=0x809e1c0) at doveadm.c:342 From gedalya at gedalya.net Sat Oct 13 03:45:29 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 12 Oct 2012 20:45:29 -0400 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <5078B9A9.6040707@gedalya.net> This has nothing to do with dovecot or with any server. Thunderbird tries to guess settings such as your IMAP and SMTP server addresses, ports, TLS, authentication scheme, etc. Sometimes none of the guess attempts matches your settings. That's not an actual problem. Just enter the settings manually. If your priority is to make Thunderbird quickly and automatically configure your email accounts, read here https://wiki.mozilla.org/Thunderbird:Autoconfiguration Note that this is Thunderbird-specific, Microsoft Outlook, Blackberry etc. each have their own autoconfiguration schemes. On 10/12/2012 01:09 PM, Justin Vore wrote: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email > account." We do have an MX record in DNS pointing to our server. We > are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version > 8.14.4-2ubuntu2 We are doing this for a class project and it is due by > Tuesday 10/16/2012.. > > > Thanks, > > > Justin From robert at schetterer.org Sat Oct 13 08:31:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 13 Oct 2012 07:31:54 +0200 Subject: [Dovecot] Help! In-Reply-To: <5078B9A9.6040707@gedalya.net> References: <5078B9A9.6040707@gedalya.net> Message-ID: <5078FCCA.2090606@schetterer.org> Am 13.10.2012 02:45, schrieb Gedalya: > This has nothing to do with dovecot or with any server. > Thunderbird tries to guess settings such as your IMAP and SMTP server > addresses, ports, TLS, authentication scheme, etc. Sometimes none of the > guess attempts matches your settings. That's not an actual problem. Just > enter the settings manually. > > If your priority is to make Thunderbird quickly and automatically > configure your email accounts, read here > https://wiki.mozilla.org/Thunderbird:Autoconfiguration > > Note that this is Thunderbird-specific, Microsoft Outlook, Blackberry > etc. each have their own autoconfiguration schemes. you may use http://www.automx.org/ for that > > > On 10/12/2012 01:09 PM, Justin Vore wrote: >> HELP! >> >> I have installed sendmail, dovecot, and squirrel mail. The squirrel >> mail portion of it works just fine, but I would like to have Mozilla >> Thunderbird as a client. Whenever I try and connect to the server it >> says "Thunderbird failed to find the settings for your email >> account." We do have an MX record in DNS pointing to our server. We >> are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version >> 8.14.4-2ubuntu2 We are doing this for a class project and it is due by >> Tuesday 10/16/2012.. >> >> >> Thanks, >> >> >> Justin > -- Best Regards MfG Robert Schetterer From alessio at skye.it Sat Oct 13 11:16:27 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 10:16:27 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= Message-ID: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> Hi, I'm running dovecot 2.1.10 on Debian 6. When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] If I add -D to doveadm I can see this: doveadm(myuser at mydomain.com): Debug: Added userdb setting: plugin/quota_rule=*:backend=524288000S doveadm(myuser at mydomain.com): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/2/mydomain.com/myuser doveadm(myuser at mydomain.com): Debug: Quota root: name=UserQuota backend=maildir args= doveadm(myuser at mydomain.com): Debug: Quota rule: root=UserQuota mailbox=* bytes=524288000 messages=0 doveadm(myuser at mydomain.com): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 doveadm(myuser at mydomain.com): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota doveadm(myuser at mydomain.com): Debug: dict quota: user=myuser at mydomain.com, uri=proxy::quota, noenforcing=1 doveadm(myuser at mydomain.com): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir doveadm(myuser at mydomain.com): Debug: maildir++: root=/home/vpopmail/domains/2/mydomain.com/myuser/Maildir, index=, control=, inbox=/home/vpopmail/domains/2/mydomain.com/myuser/Maildir, alt= doveadm(myuser at mydomain.com): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 doveadm(myuser at mydomain.com): Debug: acl: acl username = myuser at mydomain.com doveadm(myuser at mydomain.com): Debug: acl: owner = 1 doveadm(myuser at mydomain.com): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls doveadm(myuser at mydomain.com): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u doveadm(myuser at mydomain.com): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= doveadm(myuser at mydomain.com): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 doveadm(myuser at mydomain.com): Debug: acl: acl username = myuser at mydomain.com doveadm(myuser at mydomain.com): Debug: acl: owner = 0 doveadm(myuser at mydomain.com): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=276 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=277 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=278 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=279 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=280 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=281 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=282 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=283 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=284 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=285 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=286 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=287 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=288 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=289 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=290 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=291 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=292 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=293 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=294 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=295 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=296 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=297 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=298 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=299 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=300 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=301 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=302 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=303 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=304 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=305 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=306 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=307 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=308 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=309 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=310 doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found doveadm(myuser at mydomain.com): Debug: Namespace : Using permissions from /home/vpopmail/domains/2/mydomain.com/myuser/Maildir: mode=0700 gid=-1 doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Drafts/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Spam/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Sent/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Trash/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/dovecot-acl not found Segmentation fault If I disable "acl" plugin in dovecot all works fine. I think that this problem is connected with "Dovecot deliver Segmentation fault when arrive the first message" http://www.dovecot.org/list/dovecot/2012-September/068343.html that I'm still experiencing. How can get core dumps from "doveadm"? This is my dovecot config with acl enabled: # 2.1.10: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /usr/local/dovecot-2.1/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } ssl_cert = References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> Message-ID: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> On 13.10.2012, at 11.16, Alessio Cecchi wrote: > I'm running dovecot 2.1.10 on Debian 6. > > When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" > > [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] The most helpful way to get this fixed is to get a gdb backtrace: http://dovecot.org/bugreport.html From alessio at skye.it Sat Oct 13 13:48:46 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 12:48:46 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= In-Reply-To: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> Message-ID: <9a747f967d6b70da5a1551a82a017112@skye.it> Il 2012-10-13 10:42 Timo Sirainen ha scritto: > On 13.10.2012, at 11.16, Alessio Cecchi wrote: > >> I'm running dovecot 2.1.10 on Debian 6. >> >> When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it >> crash with "Segmentation fault" >> >> [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 >> sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] > > The most helpful way to get this fixed is to get a gdb backtrace: > http://dovecot.org/bugreport.html Hi Timo, I'm unable to get core dump from doveadm, I start dovecot after run "ulimit -c unlimited" and set echo "/tmp/%p" > /proc/sys/kernel/core_pattern, so core dumps is enable: Oct 13 12:38:02 master: Info: Dovecot v2.1.10 starting up Oct 13 12:38:18 auth-worker(5000): Info: mysql(localhost): Connected to database vpopmail Oct 13 12:38:18 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:38:26 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:38:29 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:39:51 dict: Info: mysql(109.168.113.139): Connected to database dovecot but when doveadm stops with "Segmentation fault" I'm unable to find any dump file and no information in dovecot.log. Can you help me? Thanks From c at roessner-network-solutions.com Sat Oct 13 14:22:30 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Sat, 13 Oct 2012 13:22:30 +0200 Subject: [Dovecot] Help! In-Reply-To: <50784F99.5080201@indietorrent.org> References: <50784F99.5080201@indietorrent.org> Message-ID: Hi, >> I have installed sendmail, dovecot, and squirrel mail. The squirrel >> mail portion of it works just fine, but I would like to have Mozilla >> Thunderbird as a client. Whenever I try and connect to the server it >> says "Thunderbird failed to find the settings for your email account." >> We do have an MX record in DNS pointing to our server. We are using >> Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 >> We are doing this for a class project and it is due by Tuesday 10/16/2012.. > > I have found Thunderbird's automatic setting detection mechanism to be > rather unreliable. > > Try entering the settings manually. I am one of the automx developers. Have a look at http://www.automx.org. It is open source. I also finished setting up a test server, so you can try with mail address automx at automx.org, pw: automx and see how it works. Kind regards -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From alessio at skye.it Sat Oct 13 15:38:41 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 14:38:41 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> Il 2012-10-02 21:28 Timo Sirainen ha scritto: > On 19.9.2012, at 16.07, Alessio Cecchi wrote: > >> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > > This backtrace is rather weird. Could you also do (instead of bt > full): > > fr 1 > p *ns > p *ns.user > p *auser > > It crashes because auser->dict = NULL, but it should never be NULL. Hi Timo, this is a new backtrace: root at demo-vpop ~ # /home/vpopmail/bin/vadduser test160 at qboxdns.it qweqweroot at demo-vpop ~ # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test160 at qboxdns.it Segmentation fault (core dumped) root at demo-vpop ~ # gdb /usr/local/dovecot-2.1/libexec/dovecot/deliver /tmp/10923 GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/local/dovecot-2.1/libexec/dovecot/deliver...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-lda.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-lda.so.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libbz2.so.1.0...(no debugging symbols found)...done. Loaded symbols for /lib/libbz2.so.1.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib90_sieve_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib90_sieve_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-sieve.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-sieve.so.0 Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver -d test160 at qboxdns.it'. Program terminated with signal 11, Segmentation fault. #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 221 if (dict->dict == NULL) (gdb) fr 1 #1 0x00007f9edac761b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x2496520) at acl-backend-vfile-acllist.c:297 297 (void)acl_lookup_dict_rebuild(auser->acl_lookup_dict); (gdb) p *ns $1 = {next = 0x2496860, refcount = 1, type = NAMESPACE_PRIVATE, flags = 8235, prefix = 0x24961b0 "", prefix_len = 0, alias_for = 0x0, alias_chain_next = 0x0, user = 0x249a700, owner = 0x249a700, list = 0x249ef40, storage = 0x2496210, set = 0x249b4f0, unexpanded_set = 0x249ab58, mail_set = 0x249b200, destroyed = 0} (gdb) p *ns.user $2 = {pool = 0x249a6e0, v = {deinit = 0x7f9edac7a280 }, vlast = 0x249bf38, refcount = 1, username = 0x249a7b8 "test160 at qboxdns.it", _home = 0x249bb60 "/home/vpopmail/domains/qboxdns.it/test160", uid = 89, gid = 89, service = 0x249bb90 "lda", local_ip = 0x0, remote_ip = 0x0, var_expand_table = 0x249bb98, error = 0x0, set_info = 0x2482ce8, unexpanded_set = 0x249a7d0, set = 0x249b168, namespaces = 0x2496130, storages = 0x24a1e20, hooks = {arr = {buffer = 0x249beb0, element_size = 8}, v = 0x249beb0, v_modifiable = 0x249beb0}, mountpoints = 0x0, module_contexts = {arr = {buffer = 0x249bb00, element_size = 8}, v = 0x249bb00, v_modifiable = 0x249bb00}, home_looked_up = 1, admin = 0, autocreated = 0, initialized = 1, mail_debug = 0, inbox_open_error_logged = 0, fuzzy_search = 0, dsyncing = 0} (gdb) p *auser $3 = {module_ctx = {super = {deinit = 0x7f9edaa68190 }, reg = 0x7f9edaa68190}, master_user = 0x0, acl_env = 0x249bd88 "vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300", groups = 0x0, acl_lookup_dict = 0x0} (gdb) and also but full for safety: (gdb) bt full #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 ns = ids_arr = {arr = {buffer = 0x0, element_size = 38363440}, v = 0x0, v_modifiable = 0x0} ids = 0x24787e0 i = dest = ret = -601327851 #1 0x00007f9edac761b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x2496520) at acl-backend-vfile-acllist.c:297 auser = 0x249bf10 iter = 0x0 acllist_path = 0x24787e0 "/home/vpopmail/domains/qboxdns.it/test160/Maildir/dovecot-acl-list" ret = ns = 0x2496130 output = 0x0 st = {st_dev = 2051, st_ino = 663856, st_nlink = 1, st_mode = 33152, st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1350131151, tv_nsec = 0}, st_mtim = {tv_sec = 1350131151, tv_nsec = 0}, st_ctim = {tv_sec = 1350131151, tv_nsec = 0}, __unused = {0, 0, 0}} path = 0x24783a8 ---Type to continue, or q to quit--- file_mode = 384 dir_mode = 448 gid = 4294967295 list = info = rootdir = 0x24787a0 "Sent" origin = 0x249f4c0 "/home/vpopmail/domains/qboxdns.it/test160/Maildir" fd = 8 #2 acl_backend_vfile_acllist_rebuild (backend=0x2496520) at acl-backend-vfile-acllist.c:311 acllist_path = #3 0x00007f9edac76563 in acl_backend_vfile_acllist_refresh (backend=0x2496520) at acl-backend-vfile-acllist.c:153 __FUNCTION__ = "acl_backend_vfile_acllist_refresh" #4 0x00007f9edac766d5 in acl_backend_vfile_acllist_verify (backend=0x0, name=0x2496800 "", mtime=0) at acl-backend-vfile-acllist.c:343 acllist = #5 0x00007f9edac750b8 in acl_backend_vfile_object_refresh_cache ( _aclobj=0x24967c0) at acl-backend-vfile.c:858 old_validity = validity = {global_validity = {last_check = 0, last_read_time = 1350131151, last_mtime = 0, last_size = 0}, local_validity = {last_check = 0, last_read_time = 0, ---Type to continue, or q to quit--- last_mtime = 0, last_size = 0}, mailbox_validity = { last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}} mtime = 0 ret = 38387472 #6 0x00007f9edac7325e in acl_backend_get_default_rights (backend=0x2496520, mask_r=0x28) at acl-backend.c:164 No locals. #7 0x00007f9edac795bd in acl_mailbox_try_list_fast (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:107 alist = nonowner_list_ctx = ret = backend = 0x2496520 acl_mask = 0x1 ns = 0x2496130 update_ctx = {iter_ctx = 0x7f9edc4bf2c8, tree_ctx = 0x7f9edcbdda88, glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, update_only = 0, match_parents = 0} name = #8 acl_mailbox_list_iter_init (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 _data_stack_cur_id = 2 ---Type to continue, or q to quit--- ctx = 0x2498e60 pool = i = inboxcase = #9 0x00007f9edc538d33 in mailbox_list_iter_init_multiple (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at mailbox-list-iter.c:158 ctx = ret = __FUNCTION__ = "mailbox_list_iter_init_multiple" #10 0x00007f9edc539459 in mailbox_list_iter_init (list=0x0, pattern=, flags=1350131151) at mailbox-list-iter.c:58 patterns = {0x7f9edaa696dc "*", 0x0} #11 0x00007f9edaa64370 in quota_count_namespace (root=0x2496cb0, bytes_r=, count_r=0x7fff890373d0) at quota-count.c:73 ctx = 0x7f9edc270ef3 info = #12 quota_count (root=0x2496cb0, bytes_r=, count_r=0x7fff890373d0) at quota-count.c:111 i = 0 ret = 0 #13 0x00007f9edaa657ce in dict_quota_count (root=0x0, want_bytes=true, value_r=0x7fff89037418) at quota-dict.c:113 ---Type to continue, or q to quit--- dt = bytes = 0 count = 0 #14 0x00007f9edaa6595a in dict_quota_update_callback ( ret=, context=0x249bf10) at quota-dict.c:178 value = 1 #15 0x00007f9edc244258 in client_dict_finish_transaction (dict=0x249eb30, line_r=) at dict-client.c:265 ctx = 0x24da1b0 #16 client_dict_read_one_line (dict=0x249eb30, line_r=) at dict-client.c:356 id = 1 line = ret = 0 __FUNCTION__ = "client_dict_read_one_line" #17 0x00007f9edc244565 in client_dict_wait (_dict=) at dict-client.c:520 dict = 0x249eb30 line = 0x0 ret = #18 0x00007f9edaa65ab5 in dict_quota_deinit (_root=) at quota-dict.c:90 root = 0x2496cb0 ---Type to continue, or q to quit--- #19 0x00007f9edaa61c72 in quota_root_deinit (root=0x0) at quota.c:240 pool = 0x249e900 #20 0x00007f9edaa636e1 in quota_deinit (_quota=0x249bf40) at quota.c:335 quota = 0x2496940 i = 2 #21 0x00007f9edaa681dd in quota_user_deinit (user=0x249a700) at quota-storage.c:412 quser = 0x249bf38 quota_set = 0x2499270 #22 0x00007f9edc53388e in mail_user_unref (_user=) at mail-user.c:153 user = 0x249a700 __FUNCTION__ = "mail_user_unref" #23 0x0000000000402de2 in main (argc=3, argv=0x247e370) at main.c:481 set_roots = {0x604640, 0x0} ctx = {pool = 0x247ef70, set = 0x24817e8, session = 0x247ef90, dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, src_envelope_sender = 0x0, dest_user = 0x0, dest_addr = 0x247e3c2 "test160 at qboxdns.it", final_dest_addr = 0x247e3c2 "test160 at qboxdns.it", dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = true, saved_mail = false, save_dest_mail = false, mailbox_full = false, ---Type to continue, or q to quit--- dsn = false} service_flags = user = 0x247e3c2 "test160 at qboxdns.it" errstr = 0x0 path = 0x7fff89037748 "\351\a" storage_service = 0x24803b0 service_user = 0x2480d58 service_input = {module = 0x4034d5 "lda", service = 0x4034d5 "lda", username = 0x247e3c2 "test160 at qboxdns.it", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = 0x2496210 user_source = destaddr_source = 0x403594 "user at hostname" process_euid = stderr_rejection = false ---Type to continue, or q to quit--- ret = c = error = MAIL_ERROR_NONE (gdb) and this the dovecot configuration: # dovecot -n # 2.1.9: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota zlib acl maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=109.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } ssl_cert = References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <83B37619-1CE8-4C5D-8147-A3C0E1C99CDC@iki.fi> Message-ID: <6730bf79a50779c9bd33311e50ccce9e@skye.it> Il 2012-10-02 22:15 Timo Sirainen ha scritto: > On 2.10.2012, at 22.28, Timo Sirainen wrote: > >> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >> >>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 >> >> This backtrace is rather weird. Could you also do (instead of bt >> full): > > Also, can you reproduce the crash always by running "doveadm quota > recalc -u user at domain"? If first add a news user, than run quota recalc and after deliver the first message "deliver" not crash: # vpopmail/bin/vadduser test10 at qboxdns.it # doveadm quota recalc -u test110 at qboxdns.it # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test10 at qboxdns.it When add a new user without quota recalc deliver crash: # /home/vpopmail/bin/vadduser test12 at qboxdns.it # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test12 at qboxdns.it Segmentation fault (core dumped) # Hope this will useful From simon.buongiorno at gmail.com Sun Oct 14 05:07:29 2012 From: simon.buongiorno at gmail.com (simon.buongiorno at gmail.com) Date: Sat, 13 Oct 2012 22:07:29 -0400 Subject: [Dovecot] POP UIDL Message-ID: Hi I run a small mail server with Dovecot and postfix. Several of the accounts are popped by an external email provide I use, mostly out habit and also for a bit of redundancy. I have no details on that external server except that it uses exim. Lately, it's been repopping mail from accounts (mail is left on the server so I can use IMAP) at fairly frequent, but undetermined, intervals. Before I take it to them, I want to be sure it's not an error on my side. How can I be sure Dovecot does not have a problem with the UIDL list causing this external server to repop the mail? For the record, I'm not inclined to think it's a Dovecot issue, but since that's the bit I can fix, I'd like to be sure. Cheers Simon From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 16:52:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 15:52:18 +0200 Subject: [Dovecot] [sieve] - counting headers In-Reply-To: <87wqyw6rv6.fsf@alfa.kjonca> References: <87wqyw6rv6.fsf@alfa.kjonca> Message-ID: <20121014135218.GA7602@daniel.localdomain> Kamil Jo?ca wrote: > In some of my maildrop filters I have rules with weighted scoring[1], > but only to count headers (for example to count "Received:" header) > ie. all these rules are of form "/pattern/:h,1" > > Can dovecot sieve do this? http://tools.ietf.org/rfc/rfc5231.txt To check the number of received fields in the header, the following test may be used: header :count "ge" :comparator "i;ascii-numeric" ["received"] ["3"] Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 17:24:22 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 16:24:22 +0200 Subject: [Dovecot] /var/run/dovecot/auth-userdb failed In-Reply-To: <1350056339814-38093.post@n4.nabble.com> References: <1350056339814-38093.post@n4.nabble.com> Message-ID: <20121014142422.GA8080@daniel.localdomain> thefantaman wrote: > I work on test server and if I send an email on log i read > > lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: > Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: > /var/run/dovecot/auth-userdb, euid is not dir owner) > > unix_listener auth-userdb { > mode = 0600 > user = root > group = root > } > } The problem is that LDA (local delivery agent or lmtp service) is not able to look up the destination mailbox in userdb. The socket /var/run/dovecot/auth-userdb is currently only readable or writable by user root since mode is set to 0600, not readable or writable by other groups. http://wiki2.dovecot.org/LDA#Virtual_users You'll need to set up a auth-userdb socket for dovecot-lda so it knows where to find mailboxes for the users. LDA is running under the virtual mailbox user and group "vmail", so you need to grant this user or group access to /var/run/dovecot/auth-userdb. You could do this by using group memberships and set mode = 0660 or simply make it world-readable-writable with mode = 0666: unix_listener auth-userdb { mode = 0666 user = root group = root } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 17:38:30 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 16:38:30 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: <9a747f967d6b70da5a1551a82a017112@skye.it> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> Message-ID: <20121014143830.GA8425@daniel.localdomain> Alessio Cecchi wrote: > I'm unable to get core dump from doveadm, I start dovecot after run > "ulimit -c unlimited" and set echo "/tmp/%p" > > /proc/sys/kernel/core_pattern, so core dumps is enable: > > but when doveadm stops with "Segmentation fault" I'm unable to find > any dump file and no information in dovecot.log. On Debian try to enable coredumps in /etc/default/dovecot and start dovecot as usual via init script. Also watch out for core dumps and segfaults in /var/log/kern.log Regards Daniel -- https://plus.google.com/103021802792276734820 From alessio at skye.it Sun Oct 14 18:58:40 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sun, 14 Oct 2012 17:58:40 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= In-Reply-To: <20121014143830.GA8425@daniel.localdomain> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> <20121014143830.GA8425@daniel.localdomain> Message-ID: Il 2012-10-14 16:38 Daniel Parthey ha scritto: > Alessio Cecchi wrote: >> I'm unable to get core dump from doveadm, I start dovecot after run >> "ulimit -c unlimited" and set echo "/tmp/%p" > >> /proc/sys/kernel/core_pattern, so core dumps is enable: >> >> but when doveadm stops with "Segmentation fault" I'm unable to find >> any dump file and no information in dovecot.log. > > On Debian try to enable coredumps in /etc/default/dovecot and start > dovecot as usual via init script. Also watch out for core dumps > and segfaults in /var/log/kern.log Thanks, my dovecot installation is build from source. Dovecot start fine with core dumps enabled but doveadm don't return "Core dumped" when crash. From dave at boostpro.com Sun Oct 14 21:30:10 2012 From: dave at boostpro.com (Dave Abrahams) Date: Sun, 14 Oct 2012 14:30:10 -0400 Subject: [Dovecot] Search for substring in header? Message-ID: Hi, According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), if I do a search for "TO isocpp.org" it should find all the messages whose To: field contains the string "isocpp.org", but dovecot is returning me an empty list. However, a search for "TO tm at isocpp.org" produces a long list of messages. What am I doing wrong? TIA, Dave -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From slusarz at curecanti.org Mon Oct 15 04:59:50 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Sun, 14 Oct 2012 19:59:50 -0600 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: Message-ID: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Quoting Dave Abrahams : > Hi, > > According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), > if I do a search for "TO isocpp.org" it should find all the messages > whose To: field contains the string "isocpp.org", but dovecot is > returning me an empty list. However, a search for "TO tm at isocpp.org" > produces a long list of messages. What am I doing wrong? First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. Second, your assumption is correct - TO should do a substring search. But this works fine for me (using version 2.1.10). michael From sandro.tosi at dada.eu Mon Oct 15 10:40:48 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Mon, 15 Oct 2012 09:40:48 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50772D89.4050601@rename-it.nl> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> Message-ID: <507BBE00.9010007@dada.eu> Hi Stephan, thanks a lot for your reply. On 10/11/2012 10:35 PM, Stephan Bosch wrote: > On 10/10/2012 11:23 AM, Sandro Tosi wrote: >> Hello, >> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >> as this will be provided with an upcoming Debian package) with MySQL >> dict lookups with the mail setup we're designing. >> >> Our (main) goals are: >> >> 1. store the filters on the database > That is possible with some limitations. Are the ones below the only limitatios (ie one script per user) or are there any other worth knowing? >> 2. allow each user to enable/disable any of the filters set we provide >> (it's a static set of some general filters, available to all the >> users; we're currently not providing the possibility to users to write >> their own filters) > Will one or multiple scripts be active at the same time? Yep, the idea is that any user could have multiple scripts active at the same time, and we'd like also to give them an ordering, so like managing a sort of priority (the lower the priority the sooner the script is executed, or the other way around, doesn't matter). Ideally, we have a set of several scripts and each user can select to enable only some of them, and choose the order of their executions. >> For point 1) we already see[1] that's possible, but it uses the map >> construct that might not fit with our current database structure: we >> have a domain table (storing the domain info) and a mailbox table >> (storing the mailbox info, but the username is composed by the local >> part, stored in this table, and the domain part is a FK to the domain >> table, using an id). >> >> Do you think it's possible to run a join query on domain+mailbox to >> retrieve the mailbox_id needed to query the table for the filters? Or >> do we have to create the filter table and store the local at domain.ext >> info there ("relaxing" the integrity relationships between tables)? > > My SQL is a bit rusty, but afaik this is possible with a JOIN or a > nested query. Ah no well, I mean, using map { } constructs :) The example for Sieve-MySQL only shows 2 maps, but given we've never used them, we'd want to know if a "map cascade" would work, so implementing the joins in multiple steps: selecting the ids with a map and the subsequent would use that id to exec the join and so on. >> How do we specify which filters are enabled for any given user? We >> originally thought of an "Enabled" field on the filter table, but in >> the example in the doc[1] I hadn't seen a way to do that: it seems >> like the filter list is specified in the proxy definition - am I >> wrong? How can we do that? > > The above suggests that you would like to activate multiple Sieve > scripts at the same time. That is currently not possible with the dict > Script location. It is on my TODO list, but I am not sure when it will > be ready (definitely not for coming Debian stable). I see, I think that some others would wonder the same, so you might also want to extend the doc to state that explicitly. Maybe you may want to include something in your TODO list to handle the ordering in case of multiple scripts. In our situation, what would you suggest? We're now thinking of keeping the scripts list on a separate table, and merge the "user selected ones" in a single script to write in the filters table. Is that what would you suggest? Is there a better solution? Cheers, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From stocton12 at yahoo.com Mon Oct 15 15:46:09 2012 From: stocton12 at yahoo.com (b m) Date: Mon, 15 Oct 2012 05:46:09 -0700 (PDT) Subject: [Dovecot] (no subject) Message-ID: <1350305169.43664.YahooMailNeo@web125703.mail.ne1.yahoo.com> Hi. I'm using dovecot 2.0.18 and I'm trying to authenticate through a CAS server (until now authentication was through MS Active Directory). I could not find anywhere some examples, so here is what i have done so far. -install phpcas and pam_cas -edit /etc/pam.d/dovecot ????????????????? auth??? sufficient????? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf -edit /etc.pam_cas.conf ????????????????? host mycas.mydomain.com ????????????????? port 443 ????????????????? uriValidate /cas/proxyValidate ????????????????? ssl on ????????????????? proxy ??????????????????????? ????????????????? trusted_ca /etc/cert/certificate.pem ????????????????? debug on - and finally dovecot.conf which I'm sure is complety wrong ????????????? userdb { ? ? ? ? ?? ?? args = /etc/dovecot/dovecot-ldap.conf ? ? ? ? ?? ?? driver = ldap ????????????? } ???????????? passdb { ???????????? driver = pam ? ? ? ? ? ?? args = cache_key=%u dovecot ???????????? } What I get in log is Oct 15 15:39:58 auth-worker: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): lookup service=dovecot Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): #1/1 style=1 msg=Password: Oct 15 15:39:58 auth-worker: Info: pam(user,127.0.0.1): pam_authenticate() failed: Permission denied Oct 15 15:40:00 auth: Debug: client out: FAIL??? 1??? user=user Oct 15 15:40:00 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 15 15:40:00 auth: Debug: auth client connected (pid=9019) Any ideas? Thanks. From linuxpencil at hotmail.com Mon Oct 15 16:01:04 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 09:01:04 -0400 Subject: [Dovecot] Can't Start Dovecot Message-ID: Hi; I just installed dovecot from yum on CentOS5. ps wax grep "dovecot" only brings up the grep The command "dovecot" is not recognized. # ls /usr/local/bin/dove* doveadm doveconf No dovecot. What up? TIA, John From s.lazzaris at interactive.eu Mon Oct 15 16:13:45 2012 From: s.lazzaris at interactive.eu (Simone Lazzaris) Date: Mon, 15 Oct 2012 15:13:45 +0200 Subject: [Dovecot] Plugin hooks in login process Message-ID: <1947528.35zxeZD9k1@orion> Hi all; I've setup dovecot (2.1.10) in a cluster configuration. We have two servers acting as frontend which authenticates users and proxy them to other two servers which handles the "real" work. Users credentials are on a mysql cluster; we have one master, in which read/write queries are processed, and many replicated slave, which process read-only queries. The frontend servers reads users credentials from the read-only mysql slaves. I'd like to execute a query once the client is verified to update the last login data. Right now, that query is executed on the backend servers, via a post-login service: protocols = imap service imap-postlogin { executable = script-login /usr/local/etc/dovecot/postlogin.sh unix_listener imap-postlogin { group = vchkpw mode = 0600 user = vpopmail } } service imap { executable = imap imap-postlogin process_limit = 2048 } Problem is, if I execute the update on the backend, I miss the information regarding the original IP, as I only see the IP of the proxies. I haven't been able to launch the postlogin service on the frontend, so I figured that I can try to write a plugin - that also seems to me the cleanest solution. Looking in the dovecot source code, I noticed that there aren't any hooks in the execution path used by the proxies; I am missing something ? I am the only one missing the presence of this hooks in the auth/proxy process ? I've also thought of a workaround for this problem. One way is to monitor the dovecot logs on the frontend and execute the update asyncronously. One other way is to query directly the main mysql server of the cluster and adjust the query making it call a stored procedure that updates the information in case of successful login. But I'd really prefer to create a plugin, that I'd be willing to share. I attach the configuration of the servers (front and back) generated via postfix -n. Thanks in advance for any help. -- Simone Lazzaris | Responsabile aree datacenter e VoIP Interactive Network srl | via Roggia Vignola 9, 24047 Treviglio (BG) Tel. 0363 1970352 | Fax 0363.1971971 | www.interactive.eu -------------- next part -------------- # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686-bigmem i686 Debian 6.0.2 auth_debug = yes auth_verbose = yes auth_verbose_passwords = plain base_dir = /var/run/dovecot/ default_login_user = nobody director_doveadm_port = 9091 director_mail_servers = AAA.BBB.CCC.DDD EEE.FFF.GGG.HHH director_servers = XXX.YYY.ZZZ.WWW disable_plaintext_auth = no listen = * log_path = /var/log/dovecot passdb { args = /usr/local/etc/dovecot/sql.conf driver = sql } protocols = imap service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director service_count = 0 } ssl_cert = From dave at boostpro.com Mon Oct 15 16:23:08 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 06:23:08 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: on Sun Oct 14 2012, Michael M Slusarz wrote: > Quoting Dave Abrahams : > >> Hi, >> >> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >> if I do a search for "TO isocpp.org" it should find all the messages >> whose To: field contains the string "isocpp.org", but dovecot is >> returning me an empty list. However, a search for "TO tm at isocpp.org" >> produces a long list of messages. What am I doing wrong? > > First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. Thanks for pointing me to the right one. > Second, your assumption is correct - TO should do a substring search. > But this works fine for me (using version 2.1.10). Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the same empty result with either of these two commands: UID SEARCH TO isocpp.org UID SEARCH TO "isocpp.org" Am I formatting the command wrongly? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From linuxpencil at hotmail.com Mon Oct 15 16:37:09 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 09:37:09 -0400 Subject: [Dovecot] Can't Start Dovecot In-Reply-To: References: Message-ID: Never mind. The command /etc/init.d/dovecot start would work; however, something else is using the port. Tracking it down. John > From: linuxpencil at hotmail.com > To: dovecot at dovecot.org > Date: Mon, 15 Oct 2012 09:01:04 -0400 > Subject: [Dovecot] Can't Start Dovecot > > > > Hi; > I just installed dovecot from yum on CentOS5. > ps wax grep "dovecot" only brings up the grep > The command "dovecot" is not recognized. > # ls /usr/local/bin/dove* > doveadm doveconf > No dovecot. What up? > TIA, > John > From h.reindl at thelounge.net Mon Oct 15 16:42:00 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 15 Oct 2012 15:42:00 +0200 Subject: [Dovecot] Can't Start Dovecot In-Reply-To: References: Message-ID: <507C12A8.1040702@thelounge.net> netstat --numeric-hosts --numeric-ports --programs -u -t -l will list all listening ports and as root also the exectueable /etc/init.d/dovecot star is they way to go never start a service by it's binary without a good reason without knowing exactly how it is supposed to work BTW: the dovecot binary lives in /sbin/ not /bin/ [root at srv:~]$ ps aux | grep dovecot root 1843 0.0 0.0 19548 1520 ? Ss 11:58 0:00 /usr/sbin/dovecot -F i am generally wonder about /usr/local as you said you installed with yum - typically distributions packages are using /usr/bin, /usr/sbin/ and not /usr/local Am 15.10.2012 15:37, schrieb John Reddy: > Never mind. The command > /etc/init.d/dovecot start > would work; however, something else is using the port. Tracking it down. > John > >> From: linuxpencil at hotmail.com >> To: dovecot at dovecot.org >> Date: Mon, 15 Oct 2012 09:01:04 -0400 >> Subject: [Dovecot] Can't Start Dovecot >> >> >> >> Hi; >> I just installed dovecot from yum on CentOS5. >> ps wax grep "dovecot" only brings up the grep >> The command "dovecot" is not recognized. >> # ls /usr/local/bin/dove* >> doveadm doveconf >> No dovecot. What up? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From list at airstreamcomm.net Mon Oct 15 17:36:11 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 15 Oct 2012 09:36:11 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> Message-ID: <507C1F5B.2060002@airstreamcomm.net> On 10/12/12 2:40 AM, Timo Sirainen wrote: > On 10.10.2012, at 17.37, Jack Bates wrote: > >> The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. >> >> My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. > I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. > > +1 for adding this detail to logging for LMTP. From dave at boostpro.com Mon Oct 15 18:08:59 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 08:08:59 -0700 Subject: [Dovecot] fts = squat solr Message-ID: I don't know if this was supposed to have changed with dovecot2, but http://wiki.dovecot.org/Plugins/FTS shows fts = squat solr so, since I have the lucene plugin?"fts = lucene" works by itself?I tried fts = squat lucene but: $ doveadm index '*' doveadm(dave): Error: fts: Failed to initialize backend 'squat lucene': Unknown backend So, is that syntax obsolete, is the wiki wrong, or am I doing something wrong? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Mon Oct 15 18:36:30 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 08:36:30 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: on Mon Oct 15 2012, Dave Abrahams wrote: > on Sun Oct 14 2012, Michael M Slusarz wrote: > >> Quoting Dave Abrahams : >> >>> Hi, >>> >>> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >>> if I do a search for "TO isocpp.org" it should find all the messages >>> whose To: field contains the string "isocpp.org", but dovecot is >>> returning me an empty list. However, a search for "TO tm at isocpp.org" >>> produces a long list of messages. What am I doing wrong? >> >> First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. > > Thanks for pointing me to the right one. > >> Second, your assumption is correct - TO should do a substring search. >> But this works fine for me (using version 2.1.10). > > Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the > same empty result with either of these two commands: > > UID SEARCH TO isocpp.org > > UID SEARCH TO "isocpp.org" > > Am I formatting the command wrongly? Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the same result. baffled-ly y'rs, -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From howellrepaja at gmail.com Mon Oct 15 18:43:25 2012 From: howellrepaja at gmail.com (Howell Repaja) Date: Mon, 15 Oct 2012 23:43:25 +0800 Subject: [Dovecot] Dovecot Authentication Problem Can't Make it Work Message-ID: Hi All, I am struggling for 2 weeks solving authentication problem in dovecot. logs from /etc/mail/maillog Oct 15 18:00:35 localhost dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 18:00:35 localhost dovecot: auth: Debug: auth client connected (pid=26723) Oct 15 18:00:35 localhost dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011lip=10.0.0.123#011rip=88.22.197.66#011lport=110#011rport=2358#011resp=AGhvd2VsbEB0b3VyZm9yeW91LmluZm8AanVtb25n Oct 15 18:00:35 localhost dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 18:00:35 localhost dovecot: auth: Debug: pam(howell at mydomain.info,88.22.33.66): lookup service=dovecot Oct 15 18:00:35 localhost dovecot: auth: Debug: pam(howell at mydomain.info,88.22.197.66): #1/1 style=1 msg=Password: Oct 15 18:00:37 localhost dovecot: auth: pam(howell at mydomain.info,88.22.197.66): unknown user Oct 15 18:00:39 localhost dovecot: auth: Debug: client out: FAIL#0111#011user=howell at mydomain.info#011reason=Password : Oct 15 18:00:39 localhost dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=>, method=PLAIN, rip=88.22.197.66, lip=10.0.0.123 Oct 15 18:01:05 localhost sendmail[26722]: q9FA15LB026722: [88.22.197.66] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.el6.x86_64 x86_64 CentOS release 6.3 (Final) ext4 auth_debug_passwords = yes auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb disable_plaintext_auth = no last_valid_gid = 10 last_valid_uid = 650 listen = * login_greeting = Dovecot ready for you. mail_debug = yes mail_location = mbox:/var/spool/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = setcred=yes failure_show_msg=yes cache_key=%u dovecot driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } service imap-login { inet_listener imap { port = 143 } } service pop3-login { inet_listener pop3 { port = 110 } } ssl_cert = Hi.I'm very sorry for the repost but I forgot the subject. So,? I'm using dovecot 2.0.18 and I'm trying to authenticate through a CAS server (until now authentication was through MS Active Directory). I could not find anywhere some examples, so here is what i have done so far. -install phpcas and pam_cas -edit /etc/pam.d/dovecot ????????????????? auth??? sufficient????? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf -edit /etc.pam_cas.conf ????????????????? host mycas.mydomain.com ????????????????? port 443 ????????????????? uriValidate /cas/proxyValidate ????????????????? ssl on ????????????????? proxy ??????????????????????? ????????????????? trusted_ca /etc/cert/certificate.pem ????????????????? debug on - and finally dovecot.conf which I'm sure is complety wrong ????????????? userdb { ? ? ? ? ?? ?? args = /etc/dovecot/dovecot-ldap.conf ? ? ? ? ?? ?? driver = ldap ????????????? } ???????????? passdb { ???????????? driver = pam ? ? ? ? ? ?? args = cache_key=%u dovecot ???????????? } What I get in log is Oct 15 15:39:58 auth-worker: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): lookup service=dovecot Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): #1/1 style=1 msg=Password: Oct 15 15:39:58 auth-worker: Info: pam(user,127.0.0.1): pam_authenticate() failed: Permission denied Oct 15 15:40:00 auth: Debug: client out: FAIL??? 1??? user=user Oct 15 15:40:00 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 15 15:40:00 auth: Debug: auth client connected (pid=9019) Any ideas? Thanks. From user+dovecot at localhost.localdomain.org Mon Oct 15 20:04:19 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 15 Oct 2012 19:04:19 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> <20121014143830.GA8425@daniel.localdomain> Message-ID: <507C4213.8000205@localhost.localdomain.org> On 10/14/2012 05:58 PM Alessio Cecchi wrote: > Thanks, my dovecot installation is build from source. Dovecot start > fine with core dumps enabled but doveadm don't return "Core dumped" when > crash. In your terminal emulator enter the following commands: ulimit -c unlimited doveadm ? Regards, Pascal -- The trapper recommends today: decade.1228919 at localdomain.org From linuxpencil at hotmail.com Mon Oct 15 21:10:12 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 14:10:12 -0400 Subject: [Dovecot] Where'd the Mail Go? Message-ID: Hi; I ran this: echo "Hello me" | mail -s "Dovecot test" $USER then created a bash script: for mbox in /var/mail/$USER /var/spool/mail/$USER ~/mbox ~/mail/* ~/*; do grep -q "Dovecot test" $mbox && echo "mbox: $mbox" done grep -q "Dovecot test" ~/Maildir/new/* 2>/dev/null && echo "Maildir: ~/Maildir" and ran it but couldn't find where the mail went. I checked /var/mail/my_user_name and it wasn't there, either. How do I find it? TIA, John From slusarz at curecanti.org Mon Oct 15 22:00:11 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 15 Oct 2012 13:00:11 -0600 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Quoting Dave Abrahams : > on Mon Oct 15 2012, Dave Abrahams wrote: > >> on Sun Oct 14 2012, Michael M Slusarz wrote: >> >>> Quoting Dave Abrahams : >>> >>>> Hi, >>>> >>>> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >>>> if I do a search for "TO isocpp.org" it should find all the messages >>>> whose To: field contains the string "isocpp.org", but dovecot is >>>> returning me an empty list. However, a search for "TO tm at isocpp.org" >>>> produces a long list of messages. What am I doing wrong? >>> >>> First, you referenced the wrong RFC - RFC 2060 has been obsoleted >>> by RFC 3501. >> >> Thanks for pointing me to the right one. >> >>> Second, your assumption is correct - TO should do a substring search. >>> But this works fine for me (using version 2.1.10). >> >> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >> same empty result with either of these two commands: >> >> UID SEARCH TO isocpp.org >> >> UID SEARCH TO "isocpp.org" >> >> Am I formatting the command wrongly? > > Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the > same result. Lucene for sure does not support subtext searching. Squat used to... but IIRC things may have changed for v2.1. Try the wiki. michael From jbates at brightok.net Mon Oct 15 22:07:07 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 15 Oct 2012 14:07:07 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> Message-ID: <507C5EDB.7050401@brightok.net> On 10/12/2012 2:40 AM, Timo Sirainen wrote: > would probably complicate the code. > I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. > > Commented logs below. I did 3 different types of connections. Let me know what you think. Because I'm logging the proxy host itself, it can be IP or name depending on the configuration. If you like it, want minor changes, additional logging, let me know and I'll adjust the code. As is, this is a one liner. Jack Oct 12 19:03:45 compiler dovecot: lmtp(18568): Connect from ::1 Connection succeeds using static proxy to lmtp.example.com (default in this config). Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from lmtp.example.com(test): 250 2.5.0 command succeeded Connection succeeds but user invalid using director mapping Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from 192.168.1.3(test2): 550 5.1.1 User doesn't exist: test2 Current error reporting Oct 12 19:04:14 compiler dovecot: lmtp(18568): Error: lmtp client: connect(192.168.1.4, 7025) failed: No route to host Connection failed and what we returned to client Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from 192.168.1.4(test42): 451 4.4.0 Remote server not answering (connect) Oct 12 19:04:16 compiler dovecot: lmtp(18568): Disconnect from ::1: Client quit (in reset) From tom at whyscream.net Mon Oct 15 22:56:19 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 15 Oct 2012 21:56:19 +0200 Subject: [Dovecot] Dovecot Authentication Problem Can't Make it Work In-Reply-To: References: Message-ID: <507C6A63.2000301@whyscream.net> On 15/10/12 17:43, Howell Repaja wrote: > Hi All, > > I am struggling for 2 weeks solving authentication problem in dovecot. > > logs from /etc/mail/maillog > Oct 15 18:00:37 localhost dovecot: auth: > pam(howell at mydomain.info,88.22.197.66): > unknown user > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-279.el6.x86_64 x86_64 CentOS release 6.3 (Final) ext4 > passdb { > args = setcred=yes failure_show_msg=yes cache_key=%u dovecot > driver = pam > } > userdb { > driver = passwd > } Pam says that you have no such user 'howell at tourforyou.info'. Pam most probably talks to /etc/passwd (and friends), which means that you either need to login with a valid valid system username listed in /etc/passwd, or you need to setup some other userdb/passdb that supports full email addresses as usernames. -- Tom From jbates at brightok.net Mon Oct 15 23:10:59 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 15 Oct 2012 15:10:59 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507C5EDB.7050401@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> Message-ID: <507C6DD3.2000309@brightok.net> On 10/15/2012 2:07 PM, Jack Bates wrote: > On 10/12/2012 2:40 AM, Timo Sirainen wrote: >> would probably complicate the code. >> I don't think this would be difficult to implement. Probably just a >> few lines of code. Yeah, could be useful. >> >> > If there's no argument over the last email, confirm and check this patch. It's not the overall logging I would like, but the lmtp code isn't as mature as pop3/imap and the proxy is a quick and dirty on the lmtp code. Both need a good revamp, preferably with x-session support and perhaps logging rip/lip similar to how we do pop3/imap logins. I think we should also work on adjusting all logging for services using x-session to also log the proxy ip. rip,lip,pip. As I get time I'll look at it. This patch is just to keep us from having no useful logging in lmtp proxy. Based on lmtp pid, one can at least follow the connect, the proxy replies, and the disconnect of a session. --- dovecot-2.1.10/src/lmtp/lmtp-proxy.c 2012-10-12 19:46:49.688952484 +0000 +++ dovecot-2.1.10/src/lmtp/lmtp-proxy.c-new 2012-10-12 19:48:51.751932325 +0000 @@ -160,6 +160,8 @@ static bool lmtp_proxy_send_data_replies break; o_stream_send_str(proxy->client_output, t_strconcat(rcpt[i]->reply, "\r\n", NULL)); + i_info("proxy(%s): proxy host=%s: status=%s",rcpt[i]->address, + rcpt[i]->conn->set.host,rcpt[i]->reply); } o_stream_uncork(proxy->client_output); proxy->next_data_reply_idx = i; From daniel.parthey at informatik.tu-chemnitz.de Tue Oct 16 00:27:46 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 15 Oct 2012 23:27:46 +0200 Subject: [Dovecot] Where'd the Mail Go? In-Reply-To: References: Message-ID: <20121015212746.GA8899@daniel.localdomain> Hi John, John Reddy wrote: > I ran this: > echo "Hello me" | mail -s "Dovecot test" $USER > and ran it but couldn't find where the mail went. I checked /var/mail/my_user_name and it wasn't there, either. How do I find it? I'm sorry to tell you this is rather off-topic on the dovecot list, since the mail is routed and delivered by your MTA. In the logfiles of your MTA (mail transport agent) you should find hints where the mail went. /var/log/postfix/... /var/log/exim/... Regards Daniel -- https://plus.google.com/103021802792276734820 From dave at boostpro.com Tue Oct 16 01:20:29 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 15:20:29 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Message-ID: on Mon Oct 15 2012, Michael M Slusarz wrote: > Quoting Dave Abrahams : > >> on Mon Oct 15 2012, Dave Abrahams wrote: >> >>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>> >>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>> same empty result with either of these two commands: >>> >>> UID SEARCH TO isocpp.org >>> >>> UID SEARCH TO "isocpp.org" >>> >>> Am I formatting the command wrongly? >> >> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >> same result. > > Lucene for sure does not support subtext searching. Squat used to... > but IIRC things may have changed for v2.1. Try the wiki. Sorry, but what does "try the wiki" mean? Which indexer are you using, that successfully finds the substring match? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 01:35:06 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 15:35:06 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search Message-ID: According to the IMAP spec if I do a search for "TO isocpp.org" it should find all the messages whose To: field contains the string "isocpp.org", but dovecot is returning me an empty list. However, a search for "TO tm at isocpp.org" produces a long list of messages. This behavior is present if I *even load* the lucene fts plugin. Note that lucene isn't in use (fts = squat); it's merely loaded. This behavior goes away if I don't load fts_lucene. Dovecot configuration with dovecot -n: --8<---------------cut here---------------start------------->8--- # 2.1.6: /usr/local/stow/dovecot-2.1.6/etc/dovecot/dovecot.conf # OS: Darwin 11.4.2 x86_64 hfs default_internal_user = _dovecot default_login_user = _dovenull mail_gid = 20 mail_location = mdbox:/Users/dave/Library/Data/LocalIMAP/mdbox mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = fts fts_squat fts_lucene zlib mail_uid = 501 maildir_very_dirty_syncs = yes namespace { inbox = yes location = prefix = separator = . subscriptions = yes type = private } passdb { args = uid=501 gid=20 home=/Users/dave nopassword=y driver = static } plugin { fts = squat zlib_save = gz zlib_save_level = 6 } protocols = imap ssl = no protocol imap { mail_plugins = fts fts_squat fts_lucene zlib } --8<---------------cut here---------------end--------------->8--- Dovecot version: 2.1.6 Operating system or Linux distribution name: MacOS X 10.7, 10.8 CPU architecture (x86 or something else?): x86_64 Filesystem you used (especially if you use NFS or not): Mac Some kind of description of what you were doing and with what IMAP client.: Searching -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From tss at iki.fi Tue Oct 16 03:09:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:09:03 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <20121005194847.GA15222@daniel.localdomain> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> <20121005194847.GA15222@daniel.localdomain> Message-ID: <5FF6D55F-3B53-4CAD-ACBA-FC334E09F159@iki.fi> On 5.10.2012, at 22.48, Daniel Parthey wrote: > Timo Sirainen wrote: >> -i changes to dovecot.conf used by the given instance name > > This does not seem to work, at least not with version 2.1.10: Fixed: http://hg.dovecot.org/dovecot-2.1/rev/0262ede193e5 From tss at iki.fi Tue Oct 16 03:12:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:12:14 +0300 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Message-ID: <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> On 4.10.2012, at 16.00, Benedetto Vassallo wrote: > All works fine, but with the new version it seems that dovecot don't do hardlinks when deliver a message to multiple users. The hard linking is done only when the directory permissions match. > mail_location = maildir:~/MailDir:LAYOUT=fs > > I tryed using lmtp directly issuing 'telnet localhost 24' and sending a test message to 3 recipients. > Then issuing a 'ls -il' in the "new" directory of that users, I saw the inode was not the same. What are the permissions of the MailDir directory for user1/user2? ls -ld /home/user1/MailDir ls -ld /home/user2/MailDir From tss at iki.fi Tue Oct 16 03:15:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:15:05 +0300 Subject: [Dovecot] Dovecot Hangs while mutile users download mail for same account using pop3 In-Reply-To: <793760c2702e89acc526a66c0b543293@Coptics.org> References: <793760c2702e89acc526a66c0b543293@Coptics.org> Message-ID: On 8.10.2012, at 18.12, Robert JR wrote: > I have a weird problem in dovecot, Dovecot Hangs while multiple users download mail for same account using pop3 > > Three persons use 1 same email , and three of them use outlook express to check > That specific mail .. Also some times one of the three users check the mail for this > Account using imap (squirrel mail) Make sure you have pop3_lock_session=no and.. > to=, orig_to=, relay=local, delay=357, delays=338/0.01/0/19, dsn=4.2.0, status=deferred (cannot update mailbox /var/mail/sales for user sales. unable to lock for exclusive access: Resource temporarily unavailable) The problem may simply be that you're using mbox format. POP3 protocol itself wasn't meant for simultaneous access (it's actually disallowed by the RFC) and with mbox format Dovecot optimizes it in a way that probably locks the mailbox exclusively for the whole duration of the session. From linuxpencil at hotmail.com Tue Oct 16 03:15:05 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 20:15:05 -0400 Subject: [Dovecot] Where'd the Mail Go? In-Reply-To: <20121015212746.GA8899@daniel.localdomain> References: , <20121015212746.GA8899@daniel.localdomain> Message-ID: > In the logfiles of your MTA (mail transport agent) you should find > hints where the mail went. Hmm. I guess I'll work on postfix then, and come back later ;) John From tss at iki.fi Tue Oct 16 03:35:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:35:57 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> Message-ID: <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> On 13.10.2012, at 15.38, Alessio Cecchi wrote: > Il 2012-10-02 21:28 Timo Sirainen ha scritto: >> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >> >>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 This should fix it: http://hg.dovecot.org/dovecot-2.1/rev/41aac09497ee From tss at iki.fi Tue Oct 16 03:37:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:37:38 +0300 Subject: [Dovecot] POP UIDL In-Reply-To: References: Message-ID: <30E57D11-39A4-43FC-9CF2-91644ADEF950@iki.fi> On 14.10.2012, at 5.07, simon.buongiorno at gmail.com wrote: > I run a small mail server with Dovecot and postfix. Several of the accounts are popped by an external email provide I use, mostly out habit and also for a bit of redundancy. I have no details on that external server except that it uses exim. > > Lately, it's been repopping mail from accounts (mail is left on the server so I can use IMAP) at fairly frequent, but undetermined, intervals. Before I take it to them, I want to be sure it's not an error on my side. How can I be sure Dovecot does not have a problem with the UIDL list causing this external server to repop the mail? > > For the record, I'm not inclined to think it's a Dovecot issue, but since that's the bit I can fix, I'd like to be sure. dovecot -n output would have helped. Anyway, some POP3 clients become confused if there are duplicate UIDLs. Recent v2.1 versions have a pop3_uidl_duplicates setting to avoid those. From tss at iki.fi Tue Oct 16 03:41:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:41:14 +0300 Subject: [Dovecot] Plugin hooks in login process In-Reply-To: <1947528.35zxeZD9k1@orion> References: <1947528.35zxeZD9k1@orion> Message-ID: <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> On 15.10.2012, at 16.13, Simone Lazzaris wrote: > Problem is, if I execute the update on the backend, I miss the information > regarding the original IP, as I only see the IP of the proxies. This is easy to solve: Set login_trusted_networks setting to point to your proxies, and you'll see the original IP. > Looking in the dovecot source code, I noticed that there aren't any hooks in > the execution path used by the proxies; I am missing something ? I am the only > one missing the presence of this hooks in the auth/proxy process ? The login processes aren't really meant to have any plugins. From tss at iki.fi Tue Oct 16 03:42:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:42:51 +0300 Subject: [Dovecot] fts = squat solr In-Reply-To: References: Message-ID: <933F50FC-B8F6-4A02-B738-9109B83D9D33@iki.fi> On 15.10.2012, at 18.08, Dave Abrahams wrote: > I don't know if this was supposed to have changed with dovecot2, but > http://wiki.dovecot.org/Plugins/FTS shows Read wiki2 for Dovecot v2. > fts = squat solr > > so, since I have the lucene plugin?"fts = lucene" works by itself?I > tried > > fts = squat lucene > > but: > > $ doveadm index '*' > doveadm(dave): Error: fts: Failed to initialize backend 'squat lucene': Unknown backend > > So, is that syntax obsolete, is the wiki wrong, or am I doing something > wrong? The syntax is obsolete. From tss at iki.fi Tue Oct 16 03:52:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:52:50 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 1.35, Dave Abrahams wrote: > According to the IMAP spec if I do a search for "TO isocpp.org" it > should find all the messages whose To: field contains the string > "isocpp.org", but dovecot is returning me an empty list. However, a > search for "TO tm at isocpp.org" produces a long list of messages. This specific problem can be solved by: plugin { fts_lucene = whitespace_chars=@. } > This > behavior is present if I *even load* the lucene fts plugin. > Note that lucene isn't in use (fts = squat); it's merely loaded. This > behavior goes away if I don't load fts_lucene. I don't really see how that's possible. Although a quick test shows me that fts_squat seems to be completely broken with me for some reason. From dave at boostpro.com Tue Oct 16 06:44:10 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:44:10 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } Wow; OK, Google tells me that's documented at http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now because I knew what to look for. This might be good enough for me, but still doesn't make it conforming to the IMAP spec, right? IIUC the spec says you can search for arbitrary strings without regard to word boundaries. >> This behavior is present if I *even load* the lucene fts plugin. >> Note that lucene isn't in use (fts = squat); it's merely loaded. >> This behavior goes away if I don't load fts_lucene. > > I don't really see how that's possible. Although a quick test shows me > that fts_squat seems to be completely broken with me for some reason. I don't know what to tell ya. Tests confirm it for me. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 06:45:53 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:45:53 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } OK, Google tells me that's documented at http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now because I knew what to look for. I suggest doing something to make that more discoverable. This might be good enough for me, but still doesn't make it conforming to the IMAP spec, right? IIUC the spec says you can search for arbitrary strings without regard to word boundaries. >> This behavior is present if I *even load* the lucene fts plugin. >> Note that lucene isn't in use (fts = squat); it's merely loaded. >> This behavior goes away if I don't load fts_lucene. > > I don't really see how that's possible. Although a quick test shows me > that fts_squat seems to be completely broken with me for some reason. I don't know what to tell ya. Tests confirm it for me. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 06:51:40 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:51:40 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } Do I also need plugin { fts = lucene } or are these mutually exclusive, or...? It's not clear from http://wiki2.dovecot.org/Plugins/FTS/Lucene -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave.mehler at gmail.com Tue Oct 16 09:30:34 2012 From: dave.mehler at gmail.com (David Mehler) Date: Tue, 16 Oct 2012 02:30:34 -0400 Subject: [Dovecot] per-user quotas In-Reply-To: <20121012040136.GA13561@daniel.localdomain> References: <20121012001539.GA10473@daniel.localdomain> <20121012040136.GA13561@daniel.localdomain> Message-ID: Hello, Thanks for your replies so far. Still having issues with per-user quotas. To my Mysql virtual_users table I've added a column quota_kb and for a test user I've added in a value of 250000 going for a 250 megabyte quota. I've tried various sql queries they're returning empty sets not pulling out the information needed. Thanks. Dave. On 10/12/12, Daniel Parthey wrote: > Hi Dave, > > David Mehler wrote: >> Thanks for your reply. I've written you directly as it is sounding >> like at least for now this isn't dovecot it's well a mysql issue with >> design. I don't have anything in my database setup with regards quota >> I've included it below. I've got one virtual mail user called vmail >> with UID/GID of 5000 who owns all the virtual mailboxes. If I'm >> understanding what I've read in the link, the dovecot wiki and your >> message since anything in a user section of Mysql will override the >> global configuration in 90-quota.conf, that being the case I should >> add an extra column to virtual users? Once that's done adjust the >> userdb query for dovecot to return quota information? > > Yes, the quota should be stored in an additional userdb column, > or you need at least an SQL statement which takes a username > and returns a quota rule. > > Quota of 0 is interpreted as "unlimited" by dovecot. > As already said, if the value in the quota column for the > user is 0, your SQL statement could also return a different > value (default quota) instead of 0 (using MySQL CASE/ELSE statement). > > Here is a short documentation on how it is done with Tine 2.0 Groupware: > http://www.tine20.org/wiki/index.php/Admins/Mailserver_integration > (it is a simple example which does not support a default quota fallback, > since Tine 2.0 writes the default quota as a value into the column > when the user is created) > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From tss at iki.fi Tue Oct 16 09:43:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 09:43:08 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 6.45, Dave Abrahams wrote: >>> According to the IMAP spec if I do a search for "TO isocpp.org" it >>> should find all the messages whose To: field contains the string >>> "isocpp.org", but dovecot is returning me an empty list. However, a >>> search for "TO tm at isocpp.org" produces a long list of messages. >> >> This specific problem can be solved by: >> >> plugin { >> fts_lucene = whitespace_chars=@. >> } > > OK, Google tells me that's documented at > http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now > because I knew what to look for. I suggest doing something to make that > more discoverable. That is the only page where there is any information about fts-lucene. I made it a bit clearer in that page now that whitespace_chars should be used as default. > This might be good enough for me, but still doesn't make it conforming > to the IMAP spec, right? IIUC the spec says you can search for > arbitrary strings without regard to word boundaries. It doesn't conform to the IMAP spec, correct. But nobody cares about that anymore. Everyone violates it. From tss at iki.fi Tue Oct 16 09:43:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 09:43:41 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 6.51, Dave Abrahams wrote: >> plugin { >> fts_lucene = whitespace_chars=@. >> } > > Do I also need > > plugin { > fts = lucene > } > > or are these mutually exclusive, or...? It's not clear from > http://wiki2.dovecot.org/Plugins/FTS/Lucene fts setting selects which backend to use. fts_lucene gives settings to that backend. From benedetto.vassallo at unipa.it Tue Oct 16 10:11:53 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Tue, 16 Oct 2012 09:11:53 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> Message-ID: <20121016091153.15601eysq5n040qh@webmail.unipa.it> Def. Quota Timo Sirainen : > On 4.10.2012, at 16.00, Benedetto Vassallo wrote: > >> All works fine, but with the new version it seems that dovecot >> don't do hardlinks when deliver a message to multiple users. > > The hard linking is done only when the directory permissions match. > >> mail_location = maildir:~/MailDir:LAYOUT=fs >> >> I tryed using lmtp directly issuing 'telnet localhost 24' and >> sending a test message to 3 recipients. >> Then issuing a 'ls -il' in the "new" directory of that users, I saw >> the inode was not the same. > > What are the permissions of the MailDir directory for user1/user2? > > ls -ld /home/user1/MailDir > ls -ld /home/user2/MailDir > > Thank you for your reply. They are different groups: drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ I tryed to issue: chgrp -R mail /home/user1/MailDir chgrp -R mail /home/user2/MailDir chgrp -R mail /home/user3/MailDir but nothing changed. Any idea? Thank you. -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From thefantaman at gmail.com Tue Oct 16 10:28:31 2012 From: thefantaman at gmail.com (Fabrizio Monti) Date: Tue, 16 Oct 2012 09:28:31 +0200 Subject: [Dovecot] /var/run/dovecot/auth-userdb failed In-Reply-To: <20121014142422.GA8080@daniel.localdomain> References: <1350056339814-38093.post@n4.nabble.com> <20121014142422.GA8080@daniel.localdomain> Message-ID: Thank you very much Daniel, I solved the problem. In this moment I have a problem with a maildir, I used a format "/home/vmail/%d/%u" and the first user make maildir he is owner "/home/vmail/%d" and second user don't make a maildir. Now, I look for the solution. Best regards. 2012/10/14 Daniel Parthey > thefantaman wrote: > > I work on test server and if I send an email on log i read > > > > lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: > > Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: > > /var/run/dovecot/auth-userdb, euid is not dir owner) > > > > unix_listener auth-userdb { > > mode = 0600 > > user = root > > group = root > > } > > } > > The problem is that LDA (local delivery agent or lmtp service) > is not able to look up the destination mailbox in userdb. > > The socket /var/run/dovecot/auth-userdb is currently only > readable or writable by user root since mode is set to 0600, > not readable or writable by other groups. > > http://wiki2.dovecot.org/LDA#Virtual_users > > You'll need to set up a auth-userdb socket for dovecot-lda so it > knows where to find mailboxes for the users. LDA is running under > the virtual mailbox user and group "vmail", so you need to grant > this user or group access to /var/run/dovecot/auth-userdb. > > You could do this by using group memberships and set mode = 0660 > or simply make it world-readable-writable with mode = 0666: > > unix_listener auth-userdb { > mode = 0666 > user = root > group = root > } > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From raabe at froglogic.com Tue Oct 16 10:30:26 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Tue, 16 Oct 2012 09:30:26 +0200 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Message-ID: <507D0D12.6000205@froglogic.com> Am 10/16/2012 12:20 AM, schrieb Dave Abrahams: > > on Mon Oct 15 2012, Michael M Slusarz wrote: > >> Quoting Dave Abrahams : >> >>> on Mon Oct 15 2012, Dave Abrahams wrote: >>> >>>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>>> >>>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>>> same empty result with either of these two commands: >>>> >>>> UID SEARCH TO isocpp.org >>>> >>>> UID SEARCH TO "isocpp.org" >>>> >>>> Am I formatting the command wrongly? >>> >>> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >>> same result. >> >> Lucene for sure does not support subtext searching. Squat used to... >> but IIRC things may have changed for v2.1. Try the wiki. > > Sorry, but what does "try the wiki" mean? > Which indexer are you using, that successfully finds the substring match? I don't know what Michael had in mind, but I also seemed to recall that the 'Squat' plugin used to be the only FTS plugin which suppotred substring matches. http://wiki2.dovecot.org/Plugins/FTS/Squat explains: "The main difference between Squat indexes and the others is that Squat provides support for substring searches, while pretty much all other FTS indexes support only matching from the beginning of words. By strictly reading the IMAP RFC it requires substring matching, so to optimize regular TEXT and BODY searches you must use Squat with Dovecot v2.0. [..] However, almost all other commonly used IMAP servers no longer care about this requirement, so Dovecot v2.1 also no longer makes this distinction." I'm not sure how to read this, but I can imagine (and maybe that's what Michael was hinting at) that the Squat plugin for Dovecot >= 2.1 no longer supports substring matches as required by the IMAP RFC whereas previous versions do. P.S.: I wish this list would have a Reply-To configured. :-) -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dave at boostpro.com Tue Oct 16 12:35:42 2012 From: dave at boostpro.com (Dave Abrahams) Date: Tue, 16 Oct 2012 02:35:42 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> <507D0D12.6000205@froglogic.com> Message-ID: on Tue Oct 16 2012, Frerich Raabe wrote: > Am 10/16/2012 12:20 AM, schrieb Dave Abrahams: >> >> on Mon Oct 15 2012, Michael M Slusarz wrote: >> >>> Quoting Dave Abrahams : >>> > >>>> on Mon Oct 15 2012, Dave Abrahams wrote: >>>> >>>>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>>>> >>>>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>>>> same empty result with either of these two commands: >>>>> >>>>> UID SEARCH TO isocpp.org >>>>> >>>>> UID SEARCH TO "isocpp.org" >>>>> >>>>> Am I formatting the command wrongly? >>>> >>>> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >>>> same result. >>> >>> Lucene for sure does not support subtext searching. Squat used to... >>> but IIRC things may have changed for v2.1. Try the wiki. >> >> Sorry, but what does "try the wiki" mean? >> Which indexer are you using, that successfully finds the substring match? > > I don't know what Michael had in mind, but I also seemed to recall > that the 'Squat' plugin used to be the only FTS plugin which suppotred > substring matches. http://wiki2.dovecot.org/Plugins/FTS/Squat > explains: > > "The main difference between Squat indexes and the others is that > Squat provides support for substring searches, while pretty much all > other FTS indexes support only matching from the beginning of > words. By strictly reading the IMAP RFC it requires substring > matching, so to optimize regular TEXT and BODY searches you must use > Squat with Dovecot v2.0. [..] However, almost all other commonly used > IMAP servers no longer care about this requirement, so Dovecot v2.1 > also no longer makes this distinction." > > I'm not sure how to read this, but I can imagine (and maybe that's > what Michael was hinting at) that the Squat plugin for Dovecot >= 2.1 > no longer supports substring matches as required by the IMAP RFC > whereas previous versions do. Well, it worked for me in 2.1.6 and 2.1.9. However, http://wiki2.dovecot.org/Plugins/FTS claims squat is "obsolete" in 2.1.x, my colleague is reporting (to me) crashes with squat during indexing, and Timo just posted that squat "seems to be completely broken for some reason." -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From alessio at skye.it Tue Oct 16 16:36:37 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 16 Oct 2012 15:36:37 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> Message-ID: <507D62E5.60006@skye.it> Il 16/10/2012 02:35, Timo Sirainen ha scritto: > On 13.10.2012, at 15.38, Alessio Cecchi wrote: > >> Il 2012-10-02 21:28 Timo Sirainen ha scritto: >>> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >>> >>>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > This should fix it: http://hg.dovecot.org/dovecot-2.1/rev/41aac09497ee > Timo, with this patch works fine! Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From towern at gmail.com Tue Oct 16 16:39:13 2012 From: towern at gmail.com (tower) Date: Tue, 16 Oct 2012 15:39:13 +0200 Subject: [Dovecot] Problem with quota update in dovecot 1.2 Message-ID: <507D6381.2030703@gmail.com> Hi A have a problem with quota2 table, working under postfixadmin. When I login into imap server with thunderbird client everything works fine, quota is update when i receive a message, but when I move message to trash or any other folder value for messages field in table quota2 increases +1 instead decreases or have still this same value. Only if I close my imap client value descending -2. I wish to table quota2 will be updated immediately after any operation on inbox. Is there any mistake in my configuration? Thanks for any advice. ============================================================== dovecot -n print: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-686 i686 Debian 6.0.3 ext3 log_path: /var/log/dovecot/error.log info_log_path: /var/log/dovecot/info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login first_valid_uid: 100 mail_privileged_group: Debian-exim mail_uid: 101 mail_gid: 103 mail_location: maildir:/var/mail/virtual/%d/%n/Maildir mail_cache_min_mail_count: 100 mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): autocreate quota imap_quota trash mail_plugins(imap): autocreate quota imap_quota trash mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): tb-extra-mailbox-sep imap_client_workarounds(imap): tb-extra-mailbox-sep imap_client_workarounds(pop3): imap_id_log(default): * imap_id_log(imap): * imap_id_log(pop3): lda: postmaster_address: postmaster at mydomain.com mail_plugins: quota log_path: /var/log/dovecot/lda-errors.log info_log_path: /var/log/dovecot/deliver.log auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: Debian-exim plugin: autocreate: INBOX autocreate2: Sent autocreate3: Trash autocreate4: Drafts autocreate5: Junk autocreate6: Spam autosubscribe: INBOX autosubscribe2: Sent autosubscribe3: Trash autosubscribe4: Drafts autosubscribe5: Junk autosubscribe6: Spam quota: dict:user::proxy::quotadict quota_rule: Trash:storage=+20%% quota_rule2: Spam:ignore quota_rule3: Drafts:storage=+5%% quota_rule4: Sent:storage=+15%% quota_rule5: Junk:storage=+10%% quota_warning: storage=100%% quota-exceeded 100 %u quota_warning2: storage=95%% quota-warning 95 %u quota_warning3: storage=90%% quota-warning 90 %u quota_warning4: storage=85%% quota-warning 85 %u quota_warning5: storage=80%% quota-warning 80 %u quota_warning6: storage=75%% quota-warning 75 %u quota_warning7: storage=70%% quota-warning 70 %u quota_warning8: storage=65%% quota-warning 65 %u trash: /etc/dovecot/dovecot-trash.conf mail_log_events: delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_group_events: no mail_log_fields: uid box msgid size subject from dict: quotadict: mysql:/etc/dovecot/dovecot-dict-quota.conf ============================================= cat /etc/dovecot/dovecot-sql.conf driver = mysql connect = host=localhost dbname=eximdb user=eximdbadm password=************* default_pass_scheme = PLAIN password_query = \ SELECT username as user, domain, password \ FROM mailbox WHERE username= '%u' AND active = 1 user_query = \ SELECT CONCAT('/var/mail/virtual/', maildir) AS home, 101 AS uid, 103 AS gid, concat('dict:storage=', CAST(ROUND(quota / 1024) AS CHAR), '::proxy::quotadict') AS quota, CONCAT('*:storage=', CAST(quota AS CHAR), 'B') AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' ============================================= cat /etc/dovecot/dovecot-dict-quota.conf connect = host=localhost dbname=eximdb user=eximdbadm password=********* map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } ============================================== cat /etc/dovecot/dovecot-trash.conf 1 Trash 2 Spam 3 Sent 4 Draft From jbates at brightok.net Tue Oct 16 16:48:17 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 08:48:17 -0500 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D6381.2030703@gmail.com> References: <507D6381.2030703@gmail.com> Message-ID: <507D65A1.2060809@brightok.net> On 10/16/2012 8:39 AM, tower wrote: > Hi > > A have a problem with quota2 table, working under postfixadmin. When I > login into imap server with thunderbird client everything works fine, > quota is update when i receive a message, but when I move message to > trash or any other folder value for messages field in table quota2 > increases +1 instead decreases or have still this same value. Only if > I close my imap client value descending -2. I wish to table quota2 > will be updated immediately after any operation on inbox. Is there any > mistake in my configuration? > > Thanks for any advice. > Are you sure the client isn't just copying it to the other folder. When you exit, you may be expunging inbox (deleting the marked message) and emptying trash (deleting the copied message). Jack From gdelafond+dovecot at aquaray.com Tue Oct 16 16:56:00 2012 From: gdelafond+dovecot at aquaray.com (de Lafond Guillaume) Date: Tue, 16 Oct 2012 15:56:00 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> Message-ID: Hi, > Hi.I'm very sorry for the repost but I forgot the subject. > So, I'm > using dovecot 2.0.18 and I'm trying to authenticate through a CAS server > (until now authentication was through MS Active Directory). I could not > find anywhere some examples, so here is what i have done so far. > -install phpcas and pam_cas > -edit /etc/pam.d/dovecot > auth sufficient /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf > -edit > /etc.pam_cas.conf > host mycas.mydomain.com > port 443 > uriValidate /cas/proxyValidate > ssl on > proxy ??????????????????????? > trusted_ca /etc/cert/certificate.pem > debug > on > > - and finally dovecot.conf which I'm sure is complety wrong > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > passdb { > driver = pam > args = cache_key=%u dovecot > } Could you try with "failure_show_msg=yes" on passdb args ? You can try without the "proxy" line in pam_cas.conf and a static userdb with allow_all_users=yes (in place of ldap configuration). Hope this can help. You have some logs in /var/log/auth.log (depends of your distrib). -- Guillaume de Lafond Aqua Ray From towern at gmail.com Tue Oct 16 17:00:46 2012 From: towern at gmail.com (tower) Date: Tue, 16 Oct 2012 16:00:46 +0200 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D65A1.2060809@brightok.net> References: <507D6381.2030703@gmail.com> <507D65A1.2060809@brightok.net> Message-ID: <507D688E.6000903@gmail.com> On 10/16/2012 03:48 PM, Jack Bates wrote: > On 10/16/2012 8:39 AM, tower wrote: >> Hi >> >> A have a problem with quota2 table, working under postfixadmin. When >> I login into imap server with thunderbird client everything works >> fine, quota is update when i receive a message, but when I move >> message to trash or any other folder value for messages field in >> table quota2 increases +1 instead decreases or have still this same >> value. Only if I close my imap client value descending -2. I wish to >> table quota2 will be updated immediately after any operation on >> inbox. Is there any mistake in my configuration? >> >> Thanks for any advice. >> > Are you sure the client isn't just copying it to the other folder. > When you exit, you may be expunging inbox (deleting the marked > message) and emptying trash (deleting the copied message). > > > Jack Yes, now I see, that messages still is in folder from which I move it. In thunderbird I do drag and drop (FYI). Is there any switch in dovecot configuration which treats message as deleted from folder, from which I move that message? From s.lazzaris at interactive.eu Tue Oct 16 17:40:57 2012 From: s.lazzaris at interactive.eu (Simone Lazzaris) Date: Tue, 16 Oct 2012 16:40:57 +0200 Subject: [Dovecot] Plugin hooks in login process In-Reply-To: <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> References: <1947528.35zxeZD9k1@orion> <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> Message-ID: <2673453.lfcgdkNUMt@orion> In data marted? 16 ottobre 2012 03:41:14, Timo Sirainen ha scritto: > On 15.10.2012, at 16.13, Simone Lazzaris wrote: > > Problem is, if I execute the update on the backend, I miss the information > > regarding the original IP, as I only see the IP of the proxies. > > This is easy to solve: Set login_trusted_networks setting to point to your > proxies, and you'll see the original IP. Thanks, it works as a charm. Just one more thing. I wasn't be able to find this option in the wiki, and also now that I know the meaning, I can't successfully google for it. Maybe the docs needs some love ? I hate tamper the developers for something I should have found by myself on the first place. > > Looking in the dovecot source code, I noticed that there aren't any hooks > > in the execution path used by the proxies; I am missing something ? I am > > the only one missing the presence of this hooks in the auth/proxy process > > ? > The login processes aren't really meant to have any plugins. I see. Well, thanks anyway. -- Simone Lazzaris | Responsabile aree datacenter e VoIP Interactive Network srl | via Roggia Vignola 9, 24047 Treviglio (BG) Tel. 0363 1970352 | Fax 0363.1971971 | www.interactive.eu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From david.travers at cohenschemist.co.uk Tue Oct 16 19:39:54 2012 From: david.travers at cohenschemist.co.uk (David Travers) Date: Tue, 16 Oct 2012 16:39:54 +0000 Subject: [Dovecot] Per user quotas Message-ID: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> Hi All, I keep going round in circles with this. I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. I believe I need to add to a passwd file, but I have added to that and nothing has changed. # dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.5 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:~/mail:LAYOUT=fs:INBOX=~/mail/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 lda: mail_plugins: quota auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: quota: maildir:user quota quota_rule: *:storage=200M quota_rule2: Trash:storage=10%% quota_rule2: SPAM:ignore quota_warning: storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2: storage=80%% /usr/local/bin/quota-warning.sh 80 quota_exceeded_message: Quota exceeded, Please delete some emails Any help gratefully received Dave _________________________________________ This email has been scanned for malicious content. _________________________________________ From jbates at brightok.net Tue Oct 16 19:54:21 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 11:54:21 -0500 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D688E.6000903@gmail.com> References: <507D6381.2030703@gmail.com> <507D65A1.2060809@brightok.net> <507D688E.6000903@gmail.com> Message-ID: <507D913D.8070004@brightok.net> On 10/16/2012 9:00 AM, tower wrote: > Yes, now I see, that messages still is in folder from which I move it. > In thunderbird I do drag and drop (FYI). Is there any switch in > dovecot configuration which treats message as deleted from folder, > from which I move that message? Not to my knowledge, but someone else may know more. I setup the Trash folder with a +100MB quota per the wiki so that it could handle deleting emails when quota was near full. Jack From stocton12 at yahoo.com Tue Oct 16 21:12:37 2012 From: stocton12 at yahoo.com (b m) Date: Tue, 16 Oct 2012 11:12:37 -0700 (PDT) Subject: [Dovecot] CAS Authentication In-Reply-To: References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> Message-ID: <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login? with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. ________________________________ From: de Lafond Guillaume To: b m ; Dovecot Mailing List Sent: Tuesday, October 16, 2012 4:56 PM Subject: Re: [Dovecot] CAS Authentication Hi, > Hi.I'm very sorry for the repost but I forgot the subject. > So,? I'm > using dovecot 2.0.18 and I'm trying to authenticate through a CAS server > (until now authentication was through MS Active Directory). I could not > find anywhere some examples, so here is what i have done so far. > -install phpcas and pam_cas > -edit /etc/pam.d/dovecot >? ? ? ? ? ? ? ? ? auth? ? sufficient? ? ? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf > -edit > /etc.pam_cas.conf >? ? ? ? ? ? ? ? ? host mycas.mydomain.com >? ? ? ? ? ? ? ? ? port 443 >? ? ? ? ? ? ? ? ? uriValidate /cas/proxyValidate >? ? ? ? ? ? ? ? ? ssl on >? ? ? ? ? ? ? ? ? proxy ??????????????????????? >? ? ? ? ? ? ? ? ? trusted_ca /etc/cert/certificate.pem >? ? ? ? ? ? ? ? ? debug > on > > - and finally dovecot.conf which I'm sure is complety wrong >? ? ? ? ? ? ? userdb { >? ? ? ? ? ? ? args = /etc/dovecot/dovecot-ldap.conf >? ? ? ? ? ? ? driver = ldap >? ? ? ? ? ? ? } >? ? ? ? ? ? ? passdb { >? ? ? ? ? ? ? driver = pam >? ? ? ? ? ? ? args = cache_key=%u dovecot >? ? ? ? ? ? ? } Could you try with "failure_show_msg=yes" on passdb args ? You can try without the "proxy" line in pam_cas.conf and a static userdb with allow_all_users=yes (in place of ldap configuration). Hope this can help. You have some logs in /var/log/auth.log (depends of your distrib). -- Guillaume de Lafond Aqua Ray From jbates at brightok.net Tue Oct 16 23:43:00 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 15:43:00 -0500 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 Message-ID: <507DC6D4.7090902@brightok.net> Timo, Please check the code. I didn't add it, but a real_lip might be useful for some people as well. Also, I notice that pop3-proxy is doing a different xsession than the imap proxy. Is there an xsession standard that is different between the two, or just an oversight in the code? Both send the remote address/port, but only imap proxy sends the local address/port. This patch declares long variable %{real_rip} so that a backend server can declare a different login_log_format_elements login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c This is primarily useful for backend servers to log both the rip, lip, and in case of xsession, the real rip. I haven't done extensive testing yet, but as long as nothing does anything weird elsewhere in the code, it should be good. http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch dovecot: pop3-login: Login: user=, method=PLAIN, rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured Jack Bates From list at airstreamcomm.net Tue Oct 16 23:57:45 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 16 Oct 2012 15:57:45 -0500 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> Message-ID: <507DCA49.9010702@airstreamcomm.net> On 10/4/12 9:58 AM, Timo Sirainen wrote: > On 4.10.2012, at 17.41, list at airstreamcomm.net wrote: > >>> protocol lmtp { >>> userdb { >>> .. >>> } >>> } >>> protocol !lmtp { >>> userdb { >>> .. >>> } >>> } >>> >>> >> Forgot to mention I am running 2.0.17. > The above works in v2.1. > >> And I am getting the following error: >> >> auth: Fatal: No passdbs specified in configuration file. PLAIN >> mechanism needs one >> >> >> From a previous post it appears that Dovecot cannot run without a global lookups specified: >> >> http://www.dovecot.org/list/dovecot/2012-March/064407.html >> >> Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? > The !lmtp version avoids that fatal problem. So the solution is: upgrade. Timo, I upgraded to 2.1 and configured as recommended, however I am still getting an error: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one Doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_debug = yes auth_verbose = yes disable_plaintext_auth = no mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service pop3-login { inet_listener pop3 { port = 110 } } ssl_cert = References: <507DC6D4.7090902@brightok.net> Message-ID: <507DE508.8070705@brightok.net> Still have some problems with the patch. will have to research it more. Worked fine on localhost, but isn't logging at all for foreign hosts in a production environment. Jack On 10/16/2012 3:43 PM, Jack Bates wrote: > Timo, > > Please check the code. I didn't add it, but a real_lip might be useful > for some people as well. Also, I notice that pop3-proxy is doing a > different xsession than the imap proxy. Is there an xsession standard > that is different between the two, or just an oversight in the code? > Both send the remote address/port, but only imap proxy sends the local > address/port. > > This patch declares long variable %{real_rip} so that a backend server > can declare a different login_log_format_elements > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l > pip=%{real_rip} mpid=%e %c > > This is primarily useful for backend servers to log both the rip, lip, > and in case of xsession, the real rip. I haven't done extensive > testing yet, but as long as nothing does anything weird elsewhere in > the code, it should be good. > > http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch > > dovecot: pop3-login: Login: user=, method=PLAIN, > rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured > > > Jack Bates > From fred at fredk.com Wed Oct 17 02:14:04 2012 From: fred at fredk.com (Fred Kilbourn) Date: Tue, 16 Oct 2012 18:14:04 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS Message-ID: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> I've had squat running on dovecot 2.0 and have been updating all users mailbox indexes nighty via cron with this command: doveadm -v search -A text xyzzyx I've just updated to 2.1 and I'm migrating to lucene indexes, but reading the documentation I'm having a hard time understanding the semantic differences between indexing and rescanning. If I were to continue to run an all user all mailbox index every night, would that be appropriate? Would running this every night avoid the need to ever rescan? Should I run rescan instead of index? Should I run both rescan and index? In which order? Best Regards, FredK From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 17 02:18:56 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 17 Oct 2012 01:18:56 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users Message-ID: <20121016231856.GA10851@daniel.localdomain> # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-44-server x86_64 Ubuntu 10.04.4 LTS doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A shows the following message in the log when iterating the 49th user: Oct 17 00:47:17 10.129.3.233 dovecot: doveadm: Error: purge: invalid option -- 'e' Oct 17 00:47:17 10.129.3.233 dovecot: doveadm(someuser at example-ll.org): Error: doveadm purge: Client sent unknown parameter: ? Any ideas on how this error gets triggered? Regards Daniel -- https://plus.google.com/103021802792276734820 From calestyo at scientia.net Wed Oct 17 02:21:14 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 01:21:14 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? Message-ID: <1350429674.3360.27.camel@fermat.scientia.net> Hi. I'm migrating all my mail archive (some 60 GB) from Evolution (which is really a broken piece of software) into dovecot. Now I face the problem how to do this best... Evolution (which is still a old 2.32.x version) itself uses mbox files, in a special hierarchical structure to allow subfolders and that like. It also stores it's own status info in X-Evolution and X-Evolution-Source mail headers. Unfortunately,... much of the mail was earlier from a Thunderbird installation, which uses it's own status headers (X-Mozilla*) that were not recognised by Evolution. I have no idea which mbox subformat was always used throughout the different programs and versions... 1) Any way to check for that? To make things worse... Thunder(burden) seems to have used a modified From_ line syntax... "^From -
$" Ideally I'd like migrate all mail into dovecot (for storage reasons again, mbox) retaining the different status flags (read, forwarded, etc.) and getting rid of the proprietary headers (of course only when they were. First thing I tried was to simply copy mail within Evolution (i.e. dragging&dropping it from the local folders to the IMAP folders from dovecot). - that preserves the status from Evolution, but doesn't restore that from Thunderbird - it clutters up the information of all From_ lines... "
" becomes "
" - neither does it handle the special Thunderbird From_ lines - neither does it remove the Thunderbird or the X-Evolution-Source headers - if Evolution has already had corrupted index files (and this is extremely likely... as it happens even immediately during recreating all of them)... so I may loose mail So my idea was that I need a program that: - can parse all the different mbox formats (those that use the quoted ">From" style and those that use Conent-Length) - can differentiate message headers from body (so that I can drop the proprietary headers and replace them by what dovecot uses as headers) - must of course understand multiline message headers 2) Any idea for a tool like that? The meaning of the X-Evolution and X-Mozilla* headers are easy to find on the web.... so I can convert them. So I basically "just" need a tool that parses all kinds of mbox formats... allow me to drop/add headers... and spits out the rest unmodified. 3) dovecot uses some special headers like X-UID and X-IMAPD... will it create these on it's own, the first time it processes the new mbox file? I mean these headers won't be there after creation. 4) Should I drop out (during) conversion... any other mail headers.. that dovecot uses as it's own? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 17 04:21:46 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 20:21:46 -0500 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 In-Reply-To: <507DE508.8070705@brightok.net> References: <507DC6D4.7090902@brightok.net> <507DE508.8070705@brightok.net> Message-ID: <507E082A.9050508@brightok.net> Umm, yeah. Setting your rsyslog to pipe certain IP matches to another file really screws things up when you change the log to include that IP. Hours of work before I figured out that it was logging just fine but to another file. lol Jack On 10/16/2012 5:51 PM, Jack Bates wrote: > Still have some problems with the patch. will have to research it > more. Worked fine on localhost, but isn't logging at all for foreign > hosts in a production environment. > > > Jack > > On 10/16/2012 3:43 PM, Jack Bates wrote: >> Timo, >> >> Please check the code. I didn't add it, but a real_lip might be >> useful for some people as well. Also, I notice that pop3-proxy is >> doing a different xsession than the imap proxy. Is there an xsession >> standard that is different between the two, or just an oversight in >> the code? Both send the remote address/port, but only imap proxy >> sends the local address/port. >> >> This patch declares long variable %{real_rip} so that a backend >> server can declare a different login_log_format_elements >> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l >> pip=%{real_rip} mpid=%e %c >> >> This is primarily useful for backend servers to log both the rip, >> lip, and in case of xsession, the real rip. I haven't done extensive >> testing yet, but as long as nothing does anything weird elsewhere in >> the code, it should be good. >> >> http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch >> >> dovecot: pop3-login: Login: user=, method=PLAIN, >> rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured >> >> >> Jack Bates >> > From calestyo at scientia.net Wed Oct 17 05:00:36 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 04:00:36 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350429674.3360.27.camel@fermat.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> Message-ID: <1350439236.18957.13.camel@fermat.scientia.net> Hi again... Things are even much much worse... (oh how I hate Evolution right now). I found a bug in Evolution, where it apparently corrupts all mail by incorrectly (not) quoting From_ lines in headers/bodies... It quotes lines matching "^From (.*)$" as ">From \1" but it does not quote at all already quoted From_ lines, i.e. "^>+From .*$". Now that means AFAICS, that it's not possible to repair that corruption (you'll see my "happiness" about this, when reading the offensive bug report): Details here: https://bugzilla.gnome.org/show_bug.cgi?id=686258 I'm not sure how this affects any of my migration/conversion plans... any ideas? Thanks, a desperate Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 17 05:44:19 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 21:44:19 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: <507E1B83.7070104@brightok.net> On 10/16/2012 6:14 PM, Fred Kilbourn wrote: > I've had squat running on dovecot 2.0 and have been updating all users > mailbox indexes nighty via cron with this command: > > doveadm -v search -A text xyzzyx > > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > the documentation I'm having a hard time understanding the semantic > differences between indexing and rescanning. > > > > If I were to continue to run an all user all mailbox index every night, > would that be appropriate? > > > > Would running this every night avoid the need to ever rescan? > > 2 sets of indexes. dovecot indexes FTS indexes Performing the cron search will update the FTS indexes, although you should read up on 2.1's doveadm index command. The dovecot indexes should stay in sync. However, if they do lose track of the FTS indexes, you can do a rescan to sync them back up. Except for corruption or index changes made outside of dovecot, the two sets of indexes should stay in sync. Jack From jbates at brightok.net Wed Oct 17 06:01:17 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 22:01:17 -0500 Subject: [Dovecot] Per user quotas In-Reply-To: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> Message-ID: <507E1F7D.4080107@brightok.net> On 10/16/2012 11:39 AM, David Travers wrote: > Hi All, > > I keep going round in circles with this. > I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! > > I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. > > I believe I need to add to a passwd file, but I have added to that and nothing has changed. > > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields Note that the extra fields are prefixed with userdb_ when placed in a passwd file. Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). Jack From fred at fredk.com Wed Oct 17 06:09:20 2012 From: fred at fredk.com (Fred Kilbourn) Date: Tue, 16 Oct 2012 22:09:20 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <507E1B83.7070104@brightok.net> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> <507E1B83.7070104@brightok.net> Message-ID: <00b301cdac14$cda1f880$68e5e980$@fredk.com> > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] > On Behalf Of Jack Bates > Sent: Tuesday, October 16, 2012 9:44 PM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Difference between Indexing and Rescan in FTS > > On 10/16/2012 6:14 PM, Fred Kilbourn wrote: > > I've had squat running on dovecot 2.0 and have been updating all users > > mailbox indexes nighty via cron with this command: > > > > doveadm -v search -A text xyzzyx > > > > > > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > > the documentation I'm having a hard time understanding the semantic > > differences between indexing and rescanning. > > > > > > > > If I were to continue to run an all user all mailbox index every night, > > would that be appropriate? > > > > > > > > Would running this every night avoid the need to ever rescan? > > > > > 2 sets of indexes. > > dovecot indexes > FTS indexes > > Performing the cron search will update the FTS indexes, although you > should read up on 2.1's doveadm index command. The dovecot indexes > should stay in sync. However, if they do lose track of the FTS indexes, > you can do a rescan to sync them back up. Except for corruption or index > changes made outside of dovecot, the two sets of indexes should stay in > sync. > > > Jack Thanks Jack, So here are my takeaways, let me know if I'm wrong: - The FTS index is the actual search data - The dovecot index holds, among other information, which messages are indexed by FTS - The FTS index still doesn't update automatically, so my nightly cronjob should keep it in order - The dovecot index should stay in order under normal circumstances, and issuing a resync command shouldn't be needed unless something bad happens Assuming my understanding above is correct, how about these questions, to further clarify my original questions: - As a system administrator, what signs should I look for that a resync is needed? (aside from user complaints) - What exact impact does running the resync command have? - Is it worthwhile to resync periodically as a maintenance task? - Or, does resyncing reset all FTS indexing that has been done, causing it to have to be done again from scratch? And, I did catch the revision in the user docs for updating indexes. I plan on updating my maintenance script accordingly. Thanks, Fred From tss at iki.fi Wed Oct 17 06:15:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 17 Oct 2012 06:15:54 +0300 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: On 17.10.2012, at 2.14, Fred Kilbourn wrote: > I've had squat running on dovecot 2.0 and have been updating all users > mailbox indexes nighty via cron with this command: > > doveadm -v search -A text xyzzyx doveadm index is a bit more efficient. > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > the documentation I'm having a hard time understanding the semantic > differences between indexing and rescanning. doveadm fts rescan makes sure that 1) all of the old messages are indexed and 2) there are no extra (already deleted) messages indexed. So it's basically repairing fts index. You probably shouldn't run it automatically, or at least not very often. From dmalolepszy at optusnet.com.au Wed Oct 17 09:11:13 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 17 Oct 2012 17:11:13 +1100 Subject: [Dovecot] Dovecot failed logins delay all logins Message-ID: <507E4C01.6010303@optusnet.com.au> Hi all, I have observed with my Dovecot setup that unique failed logins cause legitimate correct logins to be slowed. I am running two servers, each with two Dovecot instances, a Proxy with Director, and a backend Dovecot. I suspect that the backend instance is throttling connections from the same IP, and because I am running a Proxy, the backend will only see either of the two server IPs. I confirmed this by directly connecting to the backend, to bypass the proxy and rule it. I initiated dozens of unique failed logins from one IP and separately attempted to login from the same IP, and experienced an extended delay during login. At the same time a login from a different IP suceeded imediately. I see nothing in the logs suggesting some sort of process limits were exceeded, however I do see the following proc title for the backend auth process: "dovecot/auth [7 wait, 0 passdb, 0 userdb]" I have increased the mail_max_userip_connections to a very large value however I believe that setting is a per username/ip limit. Is there any sort of setting in Dovecot that I can configure that stops this authentication throttling per IP? Below is the configuration of the backend Dovecot instance. # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.3 (Santiago) auth_cache_negative_ttl = 3 secs auth_cache_size = 100 M auth_cache_ttl = 10 mins auth_default_realm = example.com auth_failure_delay = 5 secs auth_mechanisms = plain login auth_verbose_passwords = sha1 auth_worker_max_count = 25 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 last_valid_gid = 12 last_valid_uid = 8 login_greeting = Hello there. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_fsync = always mail_gid = mail mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " stats" mail_uid = mail mmap_disable = yes namespace { inbox = yes location = maildir:%h/Maildir prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocols = pop3 imap service auth { unix_listener auth-userdb { group = mail mode = 0660 user = mail } } service imap-login { inet_listener imap { address = 0.0.0.0 port = 9143 } process_min_avail = 5 service_count = 0 vsz_limit = 256 M } service imap { process_limit = 1000 vsz_limit = 256 M } service pop3-login { inet_listener pop3 { address = 0.0.0.0 port = 9110 } process_min_avail = 5 service_count = 0 vsz_limit = 256 M } service pop3 { process_limit = 1000 vsz_limit = 256 M } service stats { fifo_listener stats-mail { mode = 0600 user = mail } inet_listener { address = 127.0.0.1 port = 24242 } } ssl = no stats_memory_limit = 64 M userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol imap { imap_logout_format = bytes_read=%i bytes_send=%o mail_max_userip_connections = 1000 mail_plugins = " stats " } protocol pop3 { mail_max_userip_connections = 1000 } Dominic From pipefab at mweb.co.za Wed Oct 17 09:26:30 2012 From: pipefab at mweb.co.za (Hendrik) Date: Wed, 17 Oct 2012 08:26:30 +0200 Subject: [Dovecot] dovecot died Message-ID: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Hi All I have been trying to get this website running for months now. I get this emails from cpanel and don't know how to fix it. If anyone can help me please contact me at pipefab at mweb.co.za. Kind regards Hendrik imap failed @ Fri Oct 12 05:51:18 2012. A restart was attempted automagically. Service Check Method: [check command] Number of Restart Attempts: 1 Cmd Service Check Raw Output: dovecot is not running Startup Log: /etc/init.d/dovecot: line 15: 6043 Alarm clock /usr/sbin/dovecot > /dev/null 2>&1 Syslog Messages: Oct 12 04:01:01 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3 yj43fuh...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:01:02 scene dovecot: IMAP(__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3y j43fuhjgeiqomc3dhlkyjwdq): Disconnected: Logged out bytes=11/340 Oct 12 04:06:06 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0 jaovpdr...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:06:06 scene dovecot: IMAP(__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0j aovpdrgqjnf0_rxyi0wncetn): Disconnected: Logged out bytes=11/313 Oct 12 04:11:01 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4z k8h05nx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:11:01 scene dovecot: IMAP(__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4zk 8h05nx2p9n4yfxhrp3a2gjhl): Disconnected: Logged out bytes=11/313 Oct 12 04:16:08 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigs c75cnov...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:16:09 scene dovecot: IMAP(__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigsc 75cnovslbll4702ue2veu2n3): Disconnected: Logged out bytes=11/318 Oct 12 04:21:10 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsf ou8d5ky...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:21:11 scene dovecot: IMAP(__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsfo u8d5kysrfeaqvlmjgx6afvnb): Disconnected: Logged out bytes=11/340 Oct 12 04:26:35 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoe oga08za...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:26:36 scene dovecot: IMAP(__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoeo ga08zaq_nh6yzqsmveqpvnxi): Disconnected: Logged out bytes=11/313 Oct 12 04:31:05 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaus tpvzl1q...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:31:05 scene dovecot: IMAP(__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaust pvzl1qjjei5uuoi1c4g8tpea): Disconnected: Logged out bytes=11/313 Oct 12 04:36:02 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjj fzonng9...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:36:03 scene dovecot: IMAP(__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjjf zonng9eqrsw5l5hg7xoejer2): Disconnected: Logged out bytes=11/340 Oct 12 04:41:03 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gw eibvov1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:41:04 scene dovecot: IMAP(__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gwe ibvov1_minfh7j4_4ejaz7v2): Disconnected: Logged out bytes=11/340 Oct 12 04:46:04 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhd rhafjur...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:46:05 scene dovecot: IMAP(__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhdr hafjureydiuxbtbk2jkpkvlo): Disconnected: Logged out bytes=11/340 Oct 12 04:51:06 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrq h1iztqo...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:51:06 scene dovecot: IMAP(__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrqh 1iztqosnzfwt28kkzv4riyd9): Disconnected: Logged out bytes=11/340 Oct 12 04:56:07 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan 2pdmdvg...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:56:07 scene dovecot: IMAP(__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan2 pdmdvg6jofzylncdb3ytjaaz): Disconnected: Logged out bytes=11/340 Oct 12 05:01:08 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvir quwcv1z...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:01:08 scene dovecot: IMAP(__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvirq uwcv1zhloqt12rqni_o2pqcj): Disconnected: Logged out bytes=11/340 Oct 12 05:06:09 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgr cpi3usb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:06:09 scene dovecot: IMAP(__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgrc pi3usbdjkiy2n8zy2bbvhhny): Disconnected: Logged out bytes=11/340 Oct 12 05:11:10 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg 4vew3df...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:11:10 scene dovecot: IMAP(__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg4 vew3dfbyrglsasuldldaspck): Disconnected: Logged out bytes=11/340 Oct 12 05:16:11 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0 e2bwtas...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:16:11 scene dovecot: IMAP(__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0e 2bwtastu0kton3azlhmmuhwi): Disconnected: Logged out bytes=11/340 Oct 12 05:21:12 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszq mf4h8z1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:21:12 scene dovecot: IMAP(__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszqm f4h8z1shjl34q9zpid3g4gsp): Disconnected: Logged out bytes=11/340 Oct 12 05:26:13 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaog ha3yc0w...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:26:13 scene dovecot: IMAP(__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaogh a3yc0w6t7vvgyf1snz8vechf): Disconnected: Logged out bytes=11/340 Oct 12 05:31:14 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62 mucudsm...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:31:15 scene dovecot: IMAP(__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62m ucudsmjfmyolzcpm9shakkiw): Disconnected: Logged out bytes=11/340 Oct 12 05:36:15 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxc kmn7fkd...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:36:16 scene dovecot: IMAP(__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxck mn7fkdpsvbrjptqanfuljfv2): Disconnected: Logged out bytes=11/340 Oct 12 05:41:15 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqun f1neeqb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:41:16 scene dovecot: IMAP(__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqunf 1neeqbmzas00uqbzmmjsxiab): Disconnected: Logged out bytes=11/340 Oct 12 05:46:17 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfre yewqvxx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:46:17 scene dovecot: IMAP(__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfrey ewqvxxeoo7yhyiki7ghukxss): Disconnected: Logged out bytes=11/340 Oct 12 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 seconds. This might cause a lot of problems, so I'll just kill myself now. http://wiki.dovecot.org/TimeMovedBackwards From fred at fredk.com Wed Oct 17 09:26:32 2012 From: fred at fredk.com (Fred Kilbourn) Date: Wed, 17 Oct 2012 01:26:32 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: <00e101cdac30$5ab63270$10229750$@fredk.com> > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] > On Behalf Of Timo Sirainen > Sent: Tuesday, October 16, 2012 10:16 PM > To: Fred Kilbourn > Cc: dovecot at dovecot.org > Subject: Re: [Dovecot] Difference between Indexing and Rescan in FTS > > On 17.10.2012, at 2.14, Fred Kilbourn wrote: > > > I've had squat running on dovecot 2.0 and have been updating all users > > mailbox indexes nighty via cron with this command: > > > > doveadm -v search -A text xyzzyx > > doveadm index is a bit more efficient. > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but > reading > > the documentation I'm having a hard time understanding the semantic > > differences between indexing and rescanning. > > doveadm fts rescan makes sure that 1) all of the old messages are > indexed and 2) there are no extra (already deleted) messages indexed. So > it's basically repairing fts index. You probably shouldn't run it > automatically, or at least not very often. Okay, you've clarified that for me. I understand that rescan isn't a nightly task, but could be run every now and then periodically. How often might be appropriate if I wanted to do this as a maintenance task? Once a month? Lastly, I'm trying to use the index command instead of the search command, but I can't figure out how to make it index every mailbox for every user. Is there a wildcard that can be used for the mailbox? Or do I need to iterate all the mailboxes with one command and run index however many times for each inbox? Thanks for your help From dmalolepszy at optusnet.com.au Wed Oct 17 09:44:10 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 17 Oct 2012 17:44:10 +1100 Subject: [Dovecot] Dovecot failed logins delay all logins In-Reply-To: <507E4C01.6010303@optusnet.com.au> References: <507E4C01.6010303@optusnet.com.au> Message-ID: <507E53BA.7030001@optusnet.com.au> I think I found a solution to this thanks to a post by Timo here: http://dovecot.org/list/dovecot/2011-December/062631.html service anvil { unix_listener anvil-auth-penalty { mode = 0 } } On 17/10/12 17:11, Dominic Malolepszy wrote: > Hi all, > > I have observed with my Dovecot setup that unique failed logins cause > legitimate correct logins to be slowed. I am running two servers, each > with two Dovecot instances, a Proxy with Director, and a backend > Dovecot. I suspect that the backend instance is throttling > connections from the same IP, and because I am running a Proxy, the > backend will only see either of the two server IPs. I confirmed this > by directly connecting to the backend, to bypass the proxy and rule > it. I initiated dozens of unique failed logins from one IP and > separately attempted to login from the same IP, and experienced an > extended delay during login. At the same time a login from a different > IP suceeded imediately. I see nothing in the logs suggesting some sort > of process limits were exceeded, however I do see the following proc > title for the backend auth process: > "dovecot/auth [7 wait, 0 passdb, 0 userdb]" > > I have increased the mail_max_userip_connections to a very large value > however I believe that setting is a per username/ip limit. Is there > any sort of setting in Dovecot that I can configure that stops this > authentication throttling per IP? Below is the configuration of the > backend Dovecot instance. > > > # 2.1.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Red Hat Enterprise Linux > Server release 6.3 (Santiago) > auth_cache_negative_ttl = 3 secs > auth_cache_size = 100 M > auth_cache_ttl = 10 mins > auth_default_realm = example.com > auth_failure_delay = 5 secs > auth_mechanisms = plain login > auth_verbose_passwords = sha1 > auth_worker_max_count = 25 > base_dir = /var/run/dovecot/ > disable_plaintext_auth = no > first_valid_gid = 12 > first_valid_uid = 8 > last_valid_gid = 12 > last_valid_uid = 8 > login_greeting = Hello there. > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > mail_fsync = always > mail_gid = mail > mail_location = maildir:%h/Maildir > mail_nfs_index = yes > mail_nfs_storage = yes > mail_plugins = " stats" > mail_uid = mail > mmap_disable = yes > namespace { > inbox = yes > location = maildir:%h/Maildir > prefix = INBOX. > separator = . > } > passdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > protocols = pop3 imap > service auth { > unix_listener auth-userdb { > group = mail > mode = 0660 > user = mail > } > } > service imap-login { > inet_listener imap { > address = 0.0.0.0 > port = 9143 > } > process_min_avail = 5 > service_count = 0 > vsz_limit = 256 M > } > service imap { > process_limit = 1000 > vsz_limit = 256 M > } > service pop3-login { > inet_listener pop3 { > address = 0.0.0.0 > port = 9110 > } > process_min_avail = 5 > service_count = 0 > vsz_limit = 256 M > } > service pop3 { > process_limit = 1000 > vsz_limit = 256 M > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = mail > } > inet_listener { > address = 127.0.0.1 > port = 24242 > } > } > ssl = no > stats_memory_limit = 64 M > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > verbose_proctitle = yes > protocol imap { > imap_logout_format = bytes_read=%i bytes_send=%o > mail_max_userip_connections = 1000 > mail_plugins = " stats " > } > protocol pop3 { > mail_max_userip_connections = 1000 > } > > > Dominic From amateo at um.es Wed Oct 17 10:24:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 17 Oct 2012 09:24:42 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> Message-ID: <507E5D3A.5030900@um.es> El 16/10/12 20:12, b m escribi?: > Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. > This is my config. In /etc/pam.d/dovecot I have: auth sufficient pam_cas_ssh.so -simap://localhost -f/etc/pam_cas.conf account sufficient pam_permit.so session sufficient pam_permit.so and /etc/pam_cas.conf host port 443 uriValidate /cas/proxyValidate ssl on debug off proxy trusted_ca in dovecot, I have these users dbs: userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } With this, it works fine. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From david.travers at cohenschemist.co.uk Wed Oct 17 10:42:06 2012 From: david.travers at cohenschemist.co.uk (David Travers) Date: Wed, 17 Oct 2012 07:42:06 +0000 Subject: [Dovecot] Per user quotas In-Reply-To: <507E1F7D.4080107@brightok.net> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> Message-ID: <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> Hi Jake, Yep, similar to what I had been trying, but it doesn't seem to be working. In my /etc/passwd file I had the line:- dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash I have changed it to show:- dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange Also yes, I noticed the numbering and have corrected. Thanks for your quick reply. Dave -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates Sent: 17 October 2012 04:02 To: dovecot at dovecot.org Subject: Re: [Dovecot] Per user quotas On 10/16/2012 11:39 AM, David Travers wrote: > Hi All, > > I keep going round in circles with this. > I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! > > I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. > > I believe I need to add to a passwd file, but I have added to that and nothing has changed. > > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields Note that the extra fields are prefixed with userdb_ when placed in a passwd file. Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). Jack _________________________________________ This email has been scanned for malicious content. _________________________________________ From CMarcus at Media-Brokers.com Wed Oct 17 12:59:30 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 17 Oct 2012 05:59:30 -0400 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <507DCA49.9010702@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> <507DCA49.9010702@airstreamcomm.net> Message-ID: <507E8182.3040904@Media-Brokers.com> On 2012-10-16 4:57 PM, list at airstreamcomm.net wrote: > > Doveconf -n: > > # 2.1.1: /etc/dovecot/dovecot.conf Ummm... latest is 2.1.10... try upgrading again... ;) From CMarcus at Media-Brokers.com Wed Oct 17 13:13:34 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 17 Oct 2012 06:13:34 -0400 Subject: [Dovecot] dovecot died In-Reply-To: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> References: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Message-ID: <507E84CE.60708@Media-Brokers.com> On 2012-10-17 2:26 AM, Hendrik wrote: > 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 > seconds. This might cause a lot of problems, so I'll just kill myself now. > http://wiki.dovecot.org/TimeMovedBackwards Presumably this is on a shared hosting service, and they haven't gotten the VM to sync time properly. It is absolutely critical that time be kept in sync on servers, especially mail servers, so dovecot assumes a serious problem exists when time leaps like this happen and kills itself. Fix the time sync problem on your server (you may have to work with the service/hosting provider) and dovecot will stop committing seppuku... Charles From raabe at froglogic.com Wed Oct 17 15:44:12 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 17 Oct 2012 14:44:12 +0200 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read Message-ID: <507EA81C.5060806@froglogic.com> Hi, I'm running Dovecot 1.2.17 on FreeBSD 9 to serve an archive of a few internal mailinglists. The archive is implemented using a public namespace: namespace private { separator = / prefix = inbox = yes } namespace public { separator = / prefix = Lists/ location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists subscriptions = no } As you can see, the CONTROL/INDEX files are stored per-user to allow private \Seen flags. The different mailinglists are all sent to the 'lists' user which has a Sieve script to file them into different folders, so I have directories on my harddisk like /home/vmail/lists/Maildir/.somelist /home/vmail/lists/Maildir/.anotherlist Now, I'd like to mark the mail in *one* of those folders as \Seen by default. If the INDEX files weren't per-user, it would simply be a matter of using 'addflag "\Seen";' in the Sieve script of the lists user. Alas, this has no effect. Hence my question - how can I have the mail of just one mailinglist get marked as "read" for all users? So far, the only option I see is to add a second public namespace, with a different prefix - and this namespace doesn't use private CONTROL/INDEX files. However, I'd like to keep using the "Lists" prefix if possible to avoid too many changes to the clients. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From jbates at brightok.net Wed Oct 17 16:02:47 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:02:47 -0500 Subject: [Dovecot] Dovecot failed logins delay all logins In-Reply-To: <507E53BA.7030001@optusnet.com.au> References: <507E4C01.6010303@optusnet.com.au> <507E53BA.7030001@optusnet.com.au> Message-ID: <507EAC77.6060401@brightok.net> On 10/17/2012 1:44 AM, Dominic Malolepszy wrote: > I think I found a solution to this thanks to a post by Timo here: > http://dovecot.org/list/dovecot/2011-December/062631.html > > service anvil { unix_listener anvil-auth-penalty { mode = 0 } } > You can also leave IP based penalties and set your other servers such as proxy and webmail as trusted. Jack From jbates at brightok.net Wed Oct 17 16:07:06 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:07:06 -0500 Subject: [Dovecot] dovecot died In-Reply-To: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> References: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Message-ID: <507EAD7A.2080100@brightok.net> Read the wiki that was linked. It could be anything from ntpd/OS/hardware bug. It isn't uncommon for there to be TSC timing issues as well. I have a cutting edge server that has a bug that breaks TSC and causes timing issues. Luckily, my OS is relatively good at not stepping backwards in time. Jack On 10/17/2012 1:26 AM, Hendrik wrote: > Hi All > > I have been trying to get this website running for months now. I get this > emails from cpanel and don't know how to fix it. If anyone can help me > please contact me at pipefab at mweb.co.za. > > > > Kind regards > > Hendrik > > imap failed @ Fri Oct 12 05:51:18 2012. A restart was attempted > automagically. > > > > Service Check Method: [check command] > > > > Number of Restart Attempts: 1 > > > > Cmd Service Check Raw Output: dovecot is not running > > > > > > Startup Log: /etc/init.d/dovecot: line 15: 6043 Alarm clock > /usr/sbin/dovecot > /dev/null 2>&1 > > > > > > Syslog Messages: Oct 12 04:01:01 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3 > yj43fuh...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:01:02 scene dovecot: > IMAP(__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3y > j43fuhjgeiqomc3dhlkyjwdq): Disconnected: Logged out bytes=11/340 Oct 12 > 04:06:06 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0 > jaovpdr...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:06:06 scene dovecot: > IMAP(__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0j > aovpdrgqjnf0_rxyi0wncetn): Disconnected: Logged out bytes=11/313 Oct 12 > 04:11:01 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4z > k8h05nx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:11:01 scene dovecot: > IMAP(__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4zk > 8h05nx2p9n4yfxhrp3a2gjhl): Disconnected: Logged out bytes=11/313 Oct 12 > 04:16:08 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigs > c75cnov...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:16:09 scene dovecot: > IMAP(__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigsc > 75cnovslbll4702ue2veu2n3): Disconnected: Logged out bytes=11/318 Oct 12 > 04:21:10 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsf > ou8d5ky...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:21:11 scene dovecot: > IMAP(__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsfo > u8d5kysrfeaqvlmjgx6afvnb): Disconnected: Logged out bytes=11/340 Oct 12 > 04:26:35 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoe > oga08za...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:26:36 scene dovecot: > IMAP(__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoeo > ga08zaq_nh6yzqsmveqpvnxi): Disconnected: Logged out bytes=11/313 Oct 12 > 04:31:05 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaus > tpvzl1q...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:31:05 scene dovecot: > IMAP(__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaust > pvzl1qjjei5uuoi1c4g8tpea): Disconnected: Logged out bytes=11/313 Oct 12 > 04:36:02 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjj > fzonng9...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:36:03 scene dovecot: > IMAP(__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjjf > zonng9eqrsw5l5hg7xoejer2): Disconnected: Logged out bytes=11/340 Oct 12 > 04:41:03 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gw > eibvov1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:41:04 scene dovecot: > IMAP(__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gwe > ibvov1_minfh7j4_4ejaz7v2): Disconnected: Logged out bytes=11/340 Oct 12 > 04:46:04 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhd > rhafjur...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:46:05 scene dovecot: > IMAP(__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhdr > hafjureydiuxbtbk2jkpkvlo): Disconnected: Logged out bytes=11/340 Oct 12 > 04:51:06 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrq > h1iztqo...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:51:06 scene dovecot: > IMAP(__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrqh > 1iztqosnzfwt28kkzv4riyd9): Disconnected: Logged out bytes=11/340 Oct 12 > 04:56:07 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan > 2pdmdvg...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:56:07 scene dovecot: > IMAP(__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan2 > pdmdvg6jofzylncdb3ytjaaz): Disconnected: Logged out bytes=11/340 Oct 12 > 05:01:08 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvir > quwcv1z...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:01:08 scene dovecot: > IMAP(__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvirq > uwcv1zhloqt12rqni_o2pqcj): Disconnected: Logged out bytes=11/340 Oct 12 > 05:06:09 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgr > cpi3usb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:06:09 scene dovecot: > IMAP(__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgrc > pi3usbdjkiy2n8zy2bbvhhny): Disconnected: Logged out bytes=11/340 Oct 12 > 05:11:10 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg > 4vew3df...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:11:10 scene dovecot: > IMAP(__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg4 > vew3dfbyrglsasuldldaspck): Disconnected: Logged out bytes=11/340 Oct 12 > 05:16:11 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0 > e2bwtas...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:16:11 scene dovecot: > IMAP(__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0e > 2bwtastu0kton3azlhmmuhwi): Disconnected: Logged out bytes=11/340 Oct 12 > 05:21:12 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszq > mf4h8z1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:21:12 scene dovecot: > IMAP(__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszqm > f4h8z1shjl34q9zpid3g4gsp): Disconnected: Logged out bytes=11/340 Oct 12 > 05:26:13 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaog > ha3yc0w...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:26:13 scene dovecot: > IMAP(__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaogh > a3yc0w6t7vvgyf1snz8vechf): Disconnected: Logged out bytes=11/340 Oct 12 > 05:31:14 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62 > mucudsm...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:31:15 scene dovecot: > IMAP(__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62m > ucudsmjfmyolzcpm9shakkiw): Disconnected: Logged out bytes=11/340 Oct 12 > 05:36:15 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxc > kmn7fkd...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:36:16 scene dovecot: > IMAP(__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxck > mn7fkdpsvbrjptqanfuljfv2): Disconnected: Logged out bytes=11/340 Oct 12 > 05:41:15 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqun > f1neeqb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:41:16 scene dovecot: > IMAP(__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqunf > 1neeqbmzas00uqbzmmjsxiab): Disconnected: Logged out bytes=11/340 Oct 12 > 05:46:17 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfre > yewqvxx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:46:17 scene dovecot: > IMAP(__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfrey > ewqvxxeoo7yhyiki7ghukxss): Disconnected: Logged out bytes=11/340 Oct 12 > 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 > seconds. This might cause a lot of problems, so I'll just kill myself now. > http://wiki.dovecot.org/TimeMovedBackwards > > > > From jbates at brightok.net Wed Oct 17 16:44:38 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:44:38 -0500 Subject: [Dovecot] Per user quotas In-Reply-To: <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> Message-ID: <507EB646.5090702@brightok.net> My recommendation is that you use Passwd-file instead of Passwd and specify /etc/passwd. I mention this, as Passwd can use NSS and may not give you the results you want. Passwd-file will guarantee you use the /etc/passwd file. Also, I'm not as familiar with v1.x, but I know in v2.1.10, a userdb lookup doesn't use the userdb_ prefix. So you can try it with and without that prefix. userdb_ prefix on v2 is for cases where you do a prefetch on the passdb. I hope this helps. I've been using Passwd-file only for proxy settings and ldap for my backends to handle quota. Jack On 10/17/2012 2:42 AM, David Travers wrote: > Hi Jake, > > Yep, similar to what I had been trying, but it doesn't seem to be working. > > In my /etc/passwd file I had the line:- > dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash > > I have changed it to show:- > dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M > > Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? > > Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange > > Also yes, I noticed the numbering and have corrected. > > Thanks for your quick reply. > > Dave > > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates > Sent: 17 October 2012 04:02 > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Per user quotas > > On 10/16/2012 11:39 AM, David Travers wrote: >> Hi All, >> >> I keep going round in circles with this. >> I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! >> >> I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. >> >> I believe I need to add to a passwd file, but I have added to that and nothing has changed. >> >> > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M > > Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields > > Note that the extra fields are prefixed with userdb_ when placed in a passwd file. > > Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). > > > Jack > > > > _________________________________________ > This email has been scanned for malicious content. > _________________________________________ > From dg at dguhl.org Wed Oct 17 17:51:44 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 17 Oct 2012 16:51:44 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350429674.3360.27.camel@fermat.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> Message-ID: <20121017145144.GA777@PC211.ikt.de> On Wed, Oct 17, 2012 at 01:21:14AM +0200, Christoph Anton Mitterer wrote: > Hi. [..] > First thing I tried was to simply copy mail within Evolution (i.e. > dragging&dropping it from the local folders to the IMAP folders from > dovecot). This seems to be the smartest idea. > - that preserves the status from Evolution, but doesn't restore that > from Thunderbird Why not use TB to copy the emails from your 'TB mboxes' to Dovecot? This way I moved around 25 GiB of emails from >> 50 mbox files, created with TB 3.6 way down to some 0.x beta, to Dovecot -- without any issues I could recall. [..] > - neither does it remove the Thunderbird or the X-Evolution-Source > headers If they bug you remove them with sed or awk or perl or python or ... Dennis [..] From calestyo at scientia.net Wed Oct 17 20:57:38 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 19:57:38 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121017145144.GA777@PC211.ikt.de> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1350496658.27003.6.camel@heisenberg.scientia.net> On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: > > First thing I tried was to simply copy mail within Evolution (i.e. > > dragging&dropping it from the local folders to the IMAP folders from > > dovecot). > This seems to be the smartest idea. Well as I've mentioned... on looses the info in the From_ lines (that is the RCPT TO address and the date of arrival) because Evolution does not correctly migrated them (actually I'm not sure whether IMAP would allow that). > > - that preserves the status from Evolution, but doesn't restore that > > from Thunderbird > Why not use TB to copy the emails from your 'TB mboxes' to Dovecot? > This way I moved around 25 GiB of emails from >> 50 mbox files, > created with TB 3.6 way down to some 0.x beta, to Dovecot -- without > any issues I could recall. Sorry... too late for that... cause back in the "old" days when I went away from TB I didn't notice that the used other mail headers for their statuses... so now everthing is already mixed together. > If they bug you remove them with sed or awk or perl or python or ... Yeah... but sed alone is not enough... cause such lines may also appear in the body... and I mustn't remove them... So in principle I'm looking for a smart parser of mbox which already gives me headers and body and I can modify either. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From rob0 at gmx.co.uk Wed Oct 17 21:12:36 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 17 Oct 2012 13:12:36 -0500 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350496658.27003.6.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> Message-ID: <20121017181236.GN3672@harrier.slackbuilds.org> On Wed, Oct 17, 2012 at 07:57:38PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: > > > First thing I tried was to simply copy mail within Evolution > > > (i.e. dragging&dropping it from the local folders to the IMAP > > > folders from dovecot). > > This seems to be the smartest idea. > Well as I've mentioned... on looses the info in the From_ lines > (that is the RCPT TO address and the date of arrival) because > Evolution does not correctly migrated them (actually I'm not sure > whether IMAP would allow that). Perhaps you mean the "^From " mbox delimiter line. You do not need mbox delimiters in maildir files. Did you mention whether or not you're using maildir? -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From calestyo at scientia.net Wed Oct 17 21:21:47 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 20:21:47 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121017181236.GN3672@harrier.slackbuilds.org> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> Message-ID: <1350498107.27003.10.camel@heisenberg.scientia.net> On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: > > Well as I've mentioned... on looses the info in the From_ lines > > (that is the RCPT TO address and the date of arrival) because > > Evolution does not correctly migrated them (actually I'm not sure > > whether IMAP would allow that). > Perhaps you mean the "^From " mbox delimiter line. Yes I meant them (the _ should have denoted the space) > You do not need > mbox delimiters in maildir files. I know.. > Did you mention whether or not > you're using maildir? The reason is mainly that I have gazillions of mail in a ~ 60 GB archive... even with an fs optimised for small files I'd loose far too much space per mail than I want to afford. Also, AFAIK full text search becomes much solver in maildir (as you need to open/close endless files). On the longterm view I want to have a look into things like dbmail/archiveopteryx... for the giant local archive... and keep dovecot "only" as the internet mail server. Ideally dovecot would have such an SQL backend...or incorporate that part from Archiveopteryx. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From rob0 at gmx.co.uk Wed Oct 17 21:36:47 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 17 Oct 2012 13:36:47 -0500 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350498107.27003.10.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> <1350498107.27003.10.camel@heisenberg.scientia.net> Message-ID: <20121017183647.GO3672@harrier.slackbuilds.org> On Wed, Oct 17, 2012 at 08:21:47PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: > > Did you mention whether or not you're using maildir? > The reason is mainly that I have gazillions of mail in a ~ 60 GB > archive... even with an fs optimised for small files I'd loose far > too much space per mail than I want to afford. Fine, maildir is not the perfect solution for everyone. But I'm confused about why Evolution/Thunderbird local folders to IMAP folders does not work. That should be the best approach. If it does not work, you're going to have some perl/python/ruby scripting to do. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From list at airstreamcomm.net Wed Oct 17 22:27:04 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 17 Oct 2012 14:27:04 -0500 Subject: [Dovecot] Dsync clustering Message-ID: <507F0688.6000707@airstreamcomm.net> I have not seen mention of using dsync for clustering Dovecot in some time on the mailing list, but I believe Timo was going to write a wiki page when v2.2 became more mature. Does this documentation exist yet, or are there any resources on what dsync replication is capable of at this point (looking on the wiki and google didn't reveal much)? Thank in advance. From roundcube222 at alaadin.org Wed Oct 17 21:33:42 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 17 Oct 2012 21:33:42 +0300 Subject: [Dovecot] Disconnected for inactivity time. Message-ID: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> Hello, After 10 mins of unactivity of pop3 , dovecot disconnect the user (-ERR Disconnected for inactivity.) What is the option in the config file which control the unactivity logout time ? becuase i want to decrease the inactivity time To 5 mins instead of 10 mins Please advise Robert JR From slusarz at curecanti.org Wed Oct 17 22:47:43 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Wed, 17 Oct 2012 13:47:43 -0600 Subject: [Dovecot] Disconnected for inactivity time. In-Reply-To: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> References: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> Message-ID: <20121017134743.Horde.af5lO4F5lbhQfwtfwboGUbA@bigworm.curecanti.org> Quoting Robert JR : > After 10 mins of unactivity of pop3 , dovecot disconnect the user > (-ERR Disconnected for inactivity.) > > What is the option in the config file which control the unactivity > logout time ? becuase i want to decrease the inactivity time > > To 5 mins instead of 10 mins You can't (at least without hacking the code). The POP3 specification **requires** a minimum of 10 minutes before an autologout occurs (RFC 1939 [3]): http://tools.ietf.org/html/rfc1939#section-3 michael From stephan at rename-it.nl Wed Oct 17 23:12:00 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 17 Oct 2012 22:12:00 +0200 Subject: [Dovecot] Dovecot sieve and duplicate email subjects In-Reply-To: References: Message-ID: <507F1110.3020701@rename-it.nl> On 8/27/2012 12:40 PM, Benjamin Thomas wrote: > I was wondering if it's possible to gracefully handle duplicate subjects > lines (within a given time frame) with sieve filters ? > > Ideally, I would like the first email of the day with subject ""AutoAlert > Type1..." to get moved to the subfolder given above. Subsequent "duplicate" > emails would then go into a "duplicate" subfolder. > > Is this possible with sieve filters ? I could not find anything striking me > as obvious while reading the page http://wiki.dovecot.org/LDA/Sieve. Although the above is still not possible with the standard Sieve features, I updated the Dovecot-specific vnd.dovecot.duplicate extension with support for the scenario you describe. Basically, I made it much more flexible for checking all kinds of string value duplicates and not only strictly based on the Message-ID header. Also, the user can now directly control the time frame within which duplicates are detected (within configurable limits). http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt It will be included in the next Pigeonhole v0.3 release. Regards, Stephan. From limon at koli.be Thu Oct 18 01:40:45 2012 From: limon at koli.be (Levent Dane) Date: Wed, 17 Oct 2012 17:40:45 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c Message-ID: <456733b1b04e92265fbd9ba8e005132c@koli.be> First, I don't know why but dovecot gots this error. Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) Then, when I tried to run SELECT "INBOX/Code" (UNSEEN) virtual plugin got a segfault. Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] I'm running gentoo with kernel 2.6.32.12. My dovecot version is 2.1.10. I applied http://hg.dovecot.org/dovecot-2.1/raw-diff/0306792cc843/src/lib-storage/mail-search.c But still, i'm getting this problem. Thanks, Levent Dane From mailadmin at cubixys.com Thu Oct 18 02:16:55 2012 From: mailadmin at cubixys.com (Fasil) Date: Thu, 18 Oct 2012 02:16:55 +0300 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> Message-ID: <507F3C67.5020900@cubixys.com> Thanks Frank. Followed the URL and could not find any luck. Is there a way to change the value of 'max open files' of dovecot. I have tried to set the value in /etc/default/dovecot by setting ulimit. But the value is not getting applied. Could anyone help on this regard. Fasil. On 08/16/2012 09:17 AM, Frank Bonnet wrote: > hello > > here some useful informations > > http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ > > Envoy? de mon iPhone. > > > Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : > >> Dear all, >> >> Thank you all for such a wonderful support. Hats off to all :) >> >> Few times I came across imap login issues where new users will not be allowed to login. >> /var/log/mail.err shows the error below >> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in creating login processes, slowing down for now >> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open files >> >> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >> >> # ulimit -Hn >> 1024 >> >> # ulimit -Sn >> 1024 >> >> # cat /proc/sys/fs/file-max >> 1238548 >> >> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >> Max open files 1024 1024 files >> >> Please advice how to get rid off this. >> >> Fasil. From jtam.home at gmail.com Thu Oct 18 02:46:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 17 Oct 2012 16:46:30 -0700 (PDT) Subject: [Dovecot] Disconnected for inactivity time. In-Reply-To: References: Message-ID: Robert JR writes: > After 10 mins of unactivity of pop3 , dovecot disconnect the user (-ERR > Disconnected for inactivity.) > > What is the option in the config file which control the unactivity > logout time ? becuase i want to decrease the inactivity time > > To 5 mins instead of 10 mins Looks like it's set in the source code pop3-client.c:#define CLIENT_IDLE_TIMEOUT_MSECS (10*60*1000) but the output of # doveconf -a ... service pop3 { ... idle_kill = 0 ... } maybe points to the config that overrides this. Easy enough to test. It's non-RFC compliant as one poster points out, so unless you got a good reason to do this like lots of zombie pop3 processes, leave it alone. Joseph Tam From web at guzman.com.ar Thu Oct 18 02:47:09 2012 From: web at guzman.com.ar (Ricardo) Date: Wed, 17 Oct 2012 20:47:09 -0300 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze Message-ID: Hello list apologize in advance for my bad English, this is the first time I write to a list if I mistake Excuse me I want to implement mail server with MySQL database, Postfix and Postfixadmin, dovecot-core, dovecot-mysql dovecot-imapd dovecot-pop3d for multiple domains. I have problems installing the daemon dovecot-core, dovecot-mysql dovecot-imapd dovecot-pop3d Debian squeeze, I'm using the repositories: deb http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze main deb-src http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze main deb http://security.debian.org/ squeeze/updates main deb-src http://security.debian.org/ squeeze/updates main # squeeze-updates, previously known as 'volatile' deb http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze-updates main deb-src http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze-updates main deb http://packages.dotdeb.org squeeze all deb-src http://packages.dotdeb.org squeeze all deb http://backports.debian.org/debian-backports squeeze-backports main to install dovecot-core, dovecot-mysql, install it without problems is the version (2.1.7-2 ~ bpo60 +1) of both packages, now wanting to install dovecot-imapd dovecot-pop3d (version 1.2.15-7) breaks the dovecot-core, dovecot-mysql, apparently must be the same version all packages. Debian Wheezy, installs without problems but installs the version (dovecot-core_2.1.7-2 ~ ppa12.04 +1 _i386.deb) all packets are the same version. What is the correct version for Debian squeeze? Look for San Google but eh had success. Can anybody help? Ricardo From rfs9999 at earthlink.net Wed Oct 17 18:53:24 2012 From: rfs9999 at earthlink.net (Rick Sanders) Date: Wed, 17 Oct 2012 15:53:24 +0000 (UTC) Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: Your best bet for a clean migration is to use an IMAP migration tool (assuming both of your servers support IMAP). It avoids all of the issues surrounding the underlying databases used to store the mailboxes and messages since everything is done through IMAP commands. There are lots of different IMAP tools out there, some free some not. Using an IMAP migration tool is usually straight-forward and simple. Here is a list of some of them. Full disclosure, imap_tools is mine. imapsync: http://imapsync.lamiral.info imap_tools: http://www.athensfbc.com/imap_tools offlineimap: https://github.com/nicolas33/offlineimap mbsync: http://isync.sourceforge.net/ mailsync: http://mailsync.sourceforge.net/ mailutil: http://www.washington.edu/imap/ part of the UW IMAP tookit. imaprepl: http://www.bl0rg.net/software/ http://freecode.com/projects/imap-repl/ imapcopy: http://home.arcor.de/armin.diehl/imapcopy/imapcopy.html migrationtool: http://sourceforge.net/projects/migrationtool/ imapmigrate: http://sourceforge.net/projects/cyrus-utils/ larch: https://github.com/rgrove/larch (derived from wonko_imapsync) wonko_imapsync: http://wonko.com/article/554 pop2imap: http://www.linux-france.org/prj/pop2imap/ exchange-away: http://exchange-away.sourceforge.net/ To copy all of a user's mailboxes from one IMAP server to another using my imapcopy tool is as simple as executing the following command: imapcopy.pl -S source/username/password -D destination/user/password Regards, Rick From jbates at brightok.net Thu Oct 18 05:13:25 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 21:13:25 -0500 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <507F3C67.5020900@cubixys.com> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> <507F3C67.5020900@cubixys.com> Message-ID: <507F65C5.3090803@brightok.net> I'm using RHEL6 instead of ubuntu, but check the startup scripts. In RHEL's case, the following file is sourced, so I updated it instead of the startup scripts. cat /etc/sysconfig/dovecot # Here you can specify your dovecot command line options. # #OPTIONS="" ulimit -n 4096 ulimit -u 5120 In addition, I had to also up the max allowed processes in the dovecot config. 2.x and 1.x are different on this. http://wiki1.dovecot.org/LoginProcess <-1.x method Jack On 10/17/2012 6:16 PM, Fasil wrote: > Thanks Frank. > Followed the URL and could not find any luck. > > Is there a way to change the value of 'max open files' of dovecot. > I have tried to set the value in /etc/default/dovecot by setting > ulimit. But the value is not getting applied. > Could anyone help on this regard. > > Fasil. > > On 08/16/2012 09:17 AM, Frank Bonnet wrote: >> hello >> >> here some useful informations >> >> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ >> >> Envoy? de mon iPhone. >> >> >> Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : >> >>> Dear all, >>> >>> Thank you all for such a wonderful support. Hats off to all :) >>> >>> Few times I came across imap login issues where new users will not >>> be allowed to login. >>> /var/log/mail.err shows the error below >>> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in creating >>> login processes, slowing down for now >>> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> >>> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >>> >>> # ulimit -Hn >>> 1024 >>> >>> # ulimit -Sn >>> 1024 >>> >>> # cat /proc/sys/fs/file-max >>> 1238548 >>> >>> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >>> Max open files 1024 1024 files >>> >>> Please advice how to get rid off this. >>> >>> Fasil. > > From jbates at brightok.net Thu Oct 18 05:30:58 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 21:30:58 -0500 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze In-Reply-To: References: Message-ID: <507F69E2.7050801@brightok.net> On 10/17/2012 6:47 PM, Ricardo wrote: > > to install dovecot-core, dovecot-mysql, install it without problems is > the version (2.1.7-2 ~ bpo60 +1) of both packages, > now wanting to install dovecot-imapd dovecot-pop3d (version 1.2.15-7) > breaks the dovecot-core, dovecot-mysql, > apparently must be the same version all packages. > > I think something is wrong with your debian setup. 1.2 is normal version. 2.1 is the backports version. You should be getting this: Package: dovecot-imapd Priority: optional Section: mail Installed-Size: 559 Maintainer: Dovecot Maintainers Architecture: i386 Source: dovecot Version: 1:2.1.7-2~bpo60+1 Provides: imap-server Depends: dovecot-core (= 1:2.1.7-2~bpo60+1), libc6 (>= 2.4), ucf (>= 2.0020) That's from the Packages list http://backports.debian.org/debian-backports/dists/squeeze-backports/main/binary-i386/ Jack From tss at iki.fi Thu Oct 18 06:06:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:06:27 +0300 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <456733b1b04e92265fbd9ba8e005132c@koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> Message-ID: <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> On 18.10.2012, at 1.40, Levent Dane wrote: > First, I don't know why but dovecot gots this error. > Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) > Then, when I tried to run > SELECT "INBOX/Code" (UNSEEN) > virtual plugin got a segfault. > Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html From tss at iki.fi Thu Oct 18 06:07:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:07:39 +0300 Subject: [Dovecot] Dsync clustering In-Reply-To: <507F0688.6000707@airstreamcomm.net> References: <507F0688.6000707@airstreamcomm.net> Message-ID: On 17.10.2012, at 22.27, list at airstreamcomm.net wrote: > I have not seen mention of using dsync for clustering Dovecot in some time on the mailing list, but I believe Timo was going to write a wiki page when v2.2 became more mature. Does this documentation exist yet, or are there any resources on what dsync replication is capable of at this point (looking on the wiki and google didn't reveal much)? Thank in advance. You can probably find some mails from this mailing list. Try searching for "dsync replication". From tss at iki.fi Thu Oct 18 06:10:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:10:03 +0300 Subject: [Dovecot] Per user quotas In-Reply-To: <507EB646.5090702@brightok.net> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> <507EB646.5090702@brightok.net> Message-ID: Correct, except I wouldn't go modifying /etc/passwd directly in any case. Other software besides Dovecot might not like it. Better to create a whole new /etc/dovecot/passwd or something. On 17.10.2012, at 16.44, Jack Bates wrote: > My recommendation is that you use Passwd-file instead of Passwd and specify /etc/passwd. I mention this, as Passwd can use NSS and may not give you the results you want. Passwd-file will guarantee you use the /etc/passwd file. Also, I'm not as familiar with v1.x, but I know in v2.1.10, a userdb lookup doesn't use the userdb_ prefix. So you can try it with and without that prefix. userdb_ prefix on v2 is for cases where you do a prefetch on the passdb. > > I hope this helps. I've been using Passwd-file only for proxy settings and ldap for my backends to handle quota. > > Jack > > On 10/17/2012 2:42 AM, David Travers wrote: >> Hi Jake, >> >> Yep, similar to what I had been trying, but it doesn't seem to be working. >> >> In my /etc/passwd file I had the line:- >> dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash >> >> I have changed it to show:- >> dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M >> >> Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? >> >> Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange >> >> Also yes, I noticed the numbering and have corrected. >> >> Thanks for your quick reply. >> >> Dave >> >> -----Original Message----- >> From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates >> Sent: 17 October 2012 04:02 >> To: dovecot at dovecot.org >> Subject: Re: [Dovecot] Per user quotas >> >> On 10/16/2012 11:39 AM, David Travers wrote: >>> Hi All, >>> >>> I keep going round in circles with this. >>> I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! >>> >>> I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. >>> >>> I believe I need to add to a passwd file, but I have added to that and nothing has changed. >>> >>> >> user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M >> >> Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields >> >> Note that the extra fields are prefixed with userdb_ when placed in a passwd file. >> >> Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). >> >> >> Jack >> >> >> >> _________________________________________ >> This email has been scanned for malicious content. >> _________________________________________ >> > From tss at iki.fi Thu Oct 18 06:14:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:14:19 +0300 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <00e101cdac30$5ab63270$10229750$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> <00e101cdac30$5ab63270$10229750$@fredk.com> Message-ID: On 17.10.2012, at 9.26, Fred Kilbourn wrote: >> doveadm fts rescan makes sure that 1) all of the old messages are >> indexed and 2) there are no extra (already deleted) messages indexed. So >> it's basically repairing fts index. You probably shouldn't run it >> automatically, or at least not very often. > > Okay, you've clarified that for me. > > I understand that rescan isn't a nightly task, but could be run every now > and then periodically. How often might be appropriate if I wanted to do > this as a maintenance task? Once a month? I don't know, depends on if you have problems related to it. I think the most common answer would be "never". > Lastly, I'm trying to use the index command instead of the search command, > but I can't figure out how to make it index every mailbox for every user. > Is there a wildcard that can be used for the mailbox? Or do I need to > iterate all the mailboxes with one command and run index however many times > for each inbox? doveadm index '*' works in new versions. I don't remember from which version. From tss at iki.fi Thu Oct 18 06:18:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:18:49 +0300 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121016091153.15601eysq5n040qh@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> <20121016091153.15601eysq5n040qh@webmail.unipa.it> Message-ID: <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> On 16.10.2012, at 10.11, Benedetto Vassallo wrote: >> What are the permissions of the MailDir directory for user1/user2? >> >> ls -ld /home/user1/MailDir >> ls -ld /home/user2/MailDir >> >> > > Thank you for your reply. > They are different groups: > > drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ > drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ > drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ Not very secure permissions.. Maybe would be easiest to just have one vmail user for everyone? > I tryed to issue: > chgrp -R mail /home/user1/MailDir > chgrp -R mail /home/user2/MailDir > chgrp -R mail /home/user3/MailDir Dovecot doesn't do hard linking when it looks like the permissions aren't compatible. The current code checks that if the owner UIDs are different, then the group needs to be writable. From tss at iki.fi Thu Oct 18 06:23:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:23:58 +0300 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 In-Reply-To: <507DC6D4.7090902@brightok.net> References: <507DC6D4.7090902@brightok.net> Message-ID: <23D747EC-045A-44B0-8A85-29B76B66B969@iki.fi> On 16.10.2012, at 23.43, Jack Bates wrote: > Please check the code. I didn't add it, but a real_lip might be useful for some people as well. Also, I notice that pop3-proxy is doing a different xsession than the imap proxy. Is there an xsession standard that is different between the two, or just an oversight in the code? Both send the remote address/port, but only imap proxy sends the local address/port. > > This patch declares long variable %{real_rip} so that a backend server can declare a different login_log_format_elements > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c > > This is primarily useful for backend servers to log both the rip, lip, and in case of xsession, the real rip. I haven't done extensive testing yet, but as long as nothing does anything weird elsewhere in the code, it should be good. > > http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch Added: http://hg.dovecot.org/dovecot-2.1/rev/92364817f4ba From tss at iki.fi Thu Oct 18 06:31:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:31:19 +0300 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read In-Reply-To: <507EA81C.5060806@froglogic.com> References: <507EA81C.5060806@froglogic.com> Message-ID: On 17.10.2012, at 15.44, Frerich Raabe wrote: > I'm running Dovecot 1.2.17 on FreeBSD 9 to serve an archive of a few internal mailinglists. The archive is implemented using a public namespace: > > namespace private { > separator = / > prefix = > inbox = yes > } > > namespace public { > separator = / > prefix = Lists/ > location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > subscriptions = no > } > > As you can see, the CONTROL/INDEX files are stored per-user to allow private \Seen flags. The different mailinglists are all sent to the 'lists' user which has a Sieve script to file them into different folders, so I have directories on my harddisk like > > /home/vmail/lists/Maildir/.somelist > /home/vmail/lists/Maildir/.anotherlist > > Now, I'd like to mark the mail in *one* of those folders as \Seen by default. If the INDEX files weren't per-user, it would simply be a matter of using 'addflag "\Seen";' in the Sieve script of the lists user. Alas, this has no effect. > > Hence my question - how can I have the mail of just one mailinglist get marked as "read" for all users? You can't with the above settings. It would require writing the seen flag to all users' index files. Not easy to do and definitely not efficient to do. Maybe if there was some kind of a mixed hybrid of accepting seen flag changes from the shared index, but no such code exists (also private/shared index separation doesn't exist before v2.2). > So far, the only option I see is to add a second public namespace, with a different prefix - and this namespace doesn't use private CONTROL/INDEX files. However, I'd like to keep using the "Lists" prefix if possible to avoid too many changes to the clients. Use: prefix=Lists/anotherlist/ location = maildir:/home/vmail/lists/sharedseen/Maildir Then deliver the mails to /home/vmail/lists/sharedseen/Maildir root directly. Of course this means that you need to create a namespace for each such list. Alternative would be to use prefix=Lists/sharedseen/ and create lists under it. From jbates at brightok.net Thu Oct 18 06:48:42 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 22:48:42 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507C6DD3.2000309@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> Message-ID: <507F7C1A.9030301@brightok.net> Timo, How do you feel about parent pointers in child structures? I'm curious as the proxy structure is passed the input channel, but it doesn't know much else about the input client. Rather than pass additional information in the structure, I think it'd be better to just place a pointer back to the input client so we can access all it's details. I ask, as that might solve the problem of lack of information in logging from some of the various functions in the proxy code. I know I was limited in the quick patch I did below for my own use. I'm afraid to change it too much. You have already started xclient work in v2.2 which would necessitate a lot of changes to the lmtp/proxy code. I've actually debated backporting it to 2.1 for my own use. :) Jack On 10/15/2012 3:10 PM, Jack Bates wrote: > On 10/15/2012 2:07 PM, Jack Bates wrote: >> On 10/12/2012 2:40 AM, Timo Sirainen wrote: >>> would probably complicate the code. >>> I don't think this would be difficult to implement. Probably just a >>> few lines of code. Yeah, could be useful. >>> >>> >> > > If there's no argument over the last email, confirm and check this > patch. It's not the overall logging I would like, but the lmtp code > isn't as mature as pop3/imap and the proxy is a quick and dirty on the > lmtp code. Both need a good revamp, preferably with x-session support > and perhaps logging rip/lip similar to how we do pop3/imap logins. > > I think we should also work on adjusting all logging for services > using x-session to also log the proxy ip. rip,lip,pip. As I get time > I'll look at it. > > This patch is just to keep us from having no useful logging in lmtp > proxy. Based on lmtp pid, one can at least follow the connect, the > proxy replies, and the disconnect of a session. > > --- dovecot-2.1.10/src/lmtp/lmtp-proxy.c 2012-10-12 > 19:46:49.688952484 +0000 > +++ dovecot-2.1.10/src/lmtp/lmtp-proxy.c-new 2012-10-12 > 19:48:51.751932325 +0000 > @@ -160,6 +160,8 @@ static bool lmtp_proxy_send_data_replies > break; > o_stream_send_str(proxy->client_output, > t_strconcat(rcpt[i]->reply, "\r\n", > NULL)); > + i_info("proxy(%s): proxy host=%s: > status=%s",rcpt[i]->address, > + rcpt[i]->conn->set.host,rcpt[i]->reply); > } > o_stream_uncork(proxy->client_output); > proxy->next_data_reply_idx = i; > > From tss at iki.fi Thu Oct 18 07:07:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 07:07:03 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507F7C1A.9030301@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> Message-ID: On 18.10.2012, at 6.48, Jack Bates wrote: > How do you feel about parent pointers in child structures? I'm curious as the proxy structure is passed the input channel, but it doesn't know much else about the input client. Rather than pass additional information in the structure, I think it'd be better to just place a pointer back to the input client so we can access all it's details. Generally speaking it's cleaner to keep things as separate as possible. Maybe instead of proxy getting lmtp_client pointer both of them could contain a shared struct lmtp_client_info or something like that. But in any case I'll probably more or less rewrite the whole LMTP code at some point, because I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) From limon at koli.be Thu Oct 18 09:03:55 2012 From: limon at koli.be (Levent Dane) Date: Thu, 18 Oct 2012 01:03:55 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> Message-ID: <20121018060354.GA2528@leningrad.koli.be> On 10/18, Timo Sirainen wrote: >On 18.10.2012, at 1.40, Levent Dane wrote: >> First, I don't know why but dovecot gots this error. >> Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) >> Then, when I tried to run >> SELECT "INBOX/Code" (UNSEEN) >> virtual plugin got a segfault. >> Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] > >I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html in Code/dovecot-virtual: Archive inthread refs keyword code not deleted dovecot -n http://pastebin.com/6CQd7tJK My mail client is Mutt-hg with sidebar patch I tried to take coredump but i didn't compile with debug flags. http://pastebin.com/CMbiYJeK If you can't reproduce this error. Tomorrow, I'll compile with debug flags. Thanks, -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 From janfrode at tanso.net Thu Oct 18 11:05:44 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 18 Oct 2012 10:05:44 +0200 Subject: [Dovecot] trash plugin not doing it's job Message-ID: I enabled the trash plugin yesterday, adding "trash" to mail_plugins, and configuring the plugin setting "trash = /etc/dovecot/dovecot-trash.conf.ext". But I still see users with lots of files in INBOX.Trash getting bounced because of quota exceeded: postfix/lmtp[26273]:: C89F490061: to=, relay=loadbalancers.example.net[192.168.42.15]:24, delay=1.2, delays=0.61/0.02/0/0.54, dsn=5.2.2, status=bounced (host loadbalancers.example.net[192.168.42.15] said: 552 5.2.2 Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) dovecot:: lmtp(19730, XXXXXXX at example.no): Error: BErxFCyrf1ASTQAAWNPRnw: sieve: msgid=: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) $ sudo doveadm quota get -u XXXXXXXX at example.no Quota name Type Value Limit % UserQuota STORAGE 1048559 1048576 99 UserQuota MESSAGE 4487 - 0 Postfix if delivering via LMTP trough dovecot director. Anybody see anything obvious in my config: ------------------------------------------------------------ # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) auth_cache_size = 100 M auth_verbose = yes auth_verbose_passwords = sha1 disable_plaintext_auth = no login_trusted_networks = 192.168.0.0/16 109.247.114.192/27 mail_gid = 3000 mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u mail_max_userip_connections = 20 c = quota zlib trash mail_uid = 3000 maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = dict:UserQuota::file:%h/dovecot-quota sieve = /sieve/%1Lu/%1.1Lu/%Lu/.dovecot.sieve sieve_before = /etc/dovecot/sieve/dovecot.sieve sieve_dir = /sieve/%1Lu/%1.1Lu/%Lu sieve_max_script_size = 1M trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at example.net protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 4521 unix_listener auth-userdb { group = mode = 0600 user = atmail } } service imap-login { inet_listener imap { address = * port = 143 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login /usr/local/sbin/imap-postlogin.sh } service imap { executable = imap imap-postlogin process_limit = 2048 } service lmtp { client_limit = 1 inet_listener lmtp { address = * port = 24 } process_limit = 25 process_min_avail = 10 } service managesieve-login { inet_listener sieve { address = * port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service pop3-postlogin { executable = script-login /usr/local/sbin/pop3-postlogin.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 2048 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota zlib trash sieve } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota zlib trash imap_quota } protocol pop3 { mail_plugins = quota zlib trash pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v } protocol sieve { managesieve_logout_format = bytes=%i/%o } ------------------------------------------------------------ and my trash config: $ cat /etc/dovecot/dovecot-trash.conf.ext # Spam mailbox is emptied before Trash 1 INBOX.Spam # Trash mailbox is emptied before Sent 2 INBOX.Trash Global sieve script: $ cat /etc/dovecot/sieve/dovecot.sieve require ["comparator-i;ascii-numeric","relational","fileinto","mailbox"]; if allof ( not header :matches "x-spam-score" "-*", header :value "ge" :comparator "i;ascii-numeric" "x-spam-score" "10" ) { discard; stop; } elsif allof ( not header :matches "x-spam-score" "-*", header :value "ge" :comparator "i;ascii-numeric" "x-spam-score" "6" ) { fileinto :create "INBOX.Spam"; } -jf From stocton12 at yahoo.com Thu Oct 18 11:33:25 2012 From: stocton12 at yahoo.com (b m) Date: Thu, 18 Oct 2012 01:33:25 -0700 (PDT) Subject: [Dovecot] CAS Authentication In-Reply-To: <507E5D3A.5030900@um.es> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> <507E5D3A.5030900@um.es> Message-ID: <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> Thanks for the configuration files. I have a question. In pam_cas.conf I don't know what to put in "proxy ". In some examples I have seen something like http:///proxy.php Do I need a php file in my webmail to handle the cas tickets and if so where can I find it? ________________________________ From: Angel L. Mateo To: dovecot at dovecot.org Sent: Wednesday, October 17, 2012 10:24 AM Subject: Re: [Dovecot] CAS Authentication El 16/10/12 20:12, b m escribi?: > Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login? with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. > ??? This is my config. In /etc/pam.d/dovecot I have: auth? ? sufficient??? pam_cas_ssh.so -simap://localhost -f/etc/pam_cas.conf account sufficient??? pam_permit.so session sufficient??? pam_permit.so ??? and /etc/pam_cas.conf host port 443 uriValidate /cas/proxyValidate ssl on debug off proxy??? trusted_ca ??? in dovecot, I have these users dbs: userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } passdb { ? args = session=yes cache_key=%n dovecot ? driver = pam } ??? With this, it works fine. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From rs at sys4.de Thu Oct 18 11:42:56 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 18 Oct 2012 10:42:56 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350498107.27003.10.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> <1350498107.27003.10.camel@heisenberg.scientia.net> Message-ID: <507FC110.1040809@sys4.de> Am 17.10.2012 20:21, schrieb Christoph Anton Mitterer: > On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: >>> Well as I've mentioned... on looses the info in the From_ lines >>> (that is the RCPT TO address and the date of arrival) because >>> Evolution does not correctly migrated them (actually I'm not sure >>> whether IMAP would allow that). >> Perhaps you mean the "^From " mbox delimiter line. > Yes I meant them (the _ should have denoted the space) > > >> You do not need >> mbox delimiters in maildir files. > I know.. > > >> Did you mention whether or not >> you're using maildir? > The reason is mainly that I have gazillions of mail in a ~ 60 GB > archive... even with an fs optimised for small files I'd loose far too > much space per mail than I want to afford. > > Also, AFAIK full text search becomes much solver in maildir (as you need > to open/close endless files). On the longterm view I want to have a look > into things like dbmail/archiveopteryx... for the giant local archive... > and keep dovecot "only" as the internet mail server. > > Ideally dovecot would have such an SQL backend...or incorporate that > part from Archiveopteryx. > > > Cheers, > Chris. > this may help too http://www.stchman.com/export_evolution.html http://www.ubuntugeek.com/how-to-export-your-mails-from-evolution-to-thunderbird.html http://ubuntuforums.org/showthread.php?t=1760469 http://ubuntuforums.org/showthread.php?t=1870445 http://jaisejames.wordpress.com/2012/03/15/to-activate-maildir-in-thunderbird/ http://realtechtalk.com/ThunderbirdMBOX_to_IMAPMaildir_migration_done_easy_with_mb2md-1134-articles -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From fabio.ferrari at unimore.it Thu Oct 18 11:51:37 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Thu, 18 Oct 2012 10:51:37 +0200 Subject: [Dovecot] Problem with process_limit In-Reply-To: References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: <370893b18f6c82ba13f4cb31d19ea259.squirrel@webmail2.unimore.it> Yes, thanks, it seems that this configuration changed something, but I think there is something else. Now this particular warning in the dovecot.log disappeared, but it shows these lines instead: Oct 17 10:55:57 imap-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 17 10:55:57 imap-login: Fatal: Couldn't connect to anvil Oct 17 10:56:12 pop3-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 17 10:56:12 pop3-login: Fatal: Couldn't connect to anvil the result is quite the same, I have to reload the dovecot because it does'n accept connections. I tried to add these lines in /etc/dovecot/conf.d/10-master.conf: service anvil { client_limit = 5000 } but without good results. Any ideas? thanks in advance Fabio Ferrari > On 1.10.2012, at 12.15, FABIO FERRARI wrote: > >> Occasionally, it happens that the dovecot.log shows this line: >> master: Warning: service(imap): process_limit reached, client >> connections >> are being dropped > .. >> Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the >> line >> process_limit = 1500 > > But did you set it inside service imap {}? All of the services have > process_limit parameter. > > From CMarcus at Media-Brokers.com Thu Oct 18 14:22:35 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 18 Oct 2012 07:22:35 -0400 Subject: [Dovecot] lmtp proxy logging In-Reply-To: References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> Message-ID: <507FE67B.4030705@Media-Brokers.com> On 2012-10-18 12:07 AM, Timo Sirainen wrote: > I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) Hey Timo, I hope this means what it sounds like it means... Can you confirm that this 'submission server' would support the ability to automatically add a copy of all emails sent using it to the designated 'Sent' folder, so that email clients could simply disable the 'Save a copy to Sent folder' feature (that causes the client to upload the message to the server twice, once to send the message, and again to save the Sent copy)? This is one feature of gmail that I simply love... Thanks as always, -- Best regards, Charles From amateo at um.es Thu Oct 18 14:23:47 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 18 Oct 2012 13:23:47 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> <507E5D3A.5030900@um.es> <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <507FE6C3.80702@um.es> El 18/10/12 10:33, b m escribi?: > Thanks for the configuration files. I have a question. In pam_cas.conf I don't know what to put in "proxy ". In some examples I have seen something like http:///proxy.php > Do I need a php file in my webmail to handle the cas tickets and if so where can I find it? > You need this script at your webmail server. This script depend on the webmail you are using. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From tss at iki.fi Thu Oct 18 14:30:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 14:30:00 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507FE67B.4030705@Media-Brokers.com> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> <507FE67B.4030705@Media-Brokers.com> Message-ID: <141971D9-6DC7-4BEF-B4AE-B1EBD5314499@iki.fi> On 18.10.2012, at 14.22, Charles Marcus wrote: > On 2012-10-18 12:07 AM, Timo Sirainen wrote: >> I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) > > Hey Timo, > > I hope this means what it sounds like it means... > > Can you confirm that this 'submission server' would support the ability to automatically add a copy of all emails sent using it to the designated 'Sent' folder, so that email clients could simply disable the 'Save a copy to Sent folder' feature (that causes the client to upload the message to the server twice, once to send the message, and again to save the Sent copy)? That's not the intended reason for creating it, but easy enough to add as an option, assuming \Sent SPECIAL-USE mailbox is defined. Anyway, I don't know when I'll actually start implementing it. Mainly just a "would be nice to have some day" thing to support LEMONADE SMTP extensions. From dg at dguhl.org Thu Oct 18 15:34:41 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 18 Oct 2012 14:34:41 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350496658.27003.6.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> Message-ID: <20121018123440.GA29330@laptop-dg.leere.eu> On Wed, Oct 17, 2012 at 07:57:38PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: [move through Evolution to IMAP] > Well as I've mentioned... on looses the info in the From_ lines (that is > the RCPT TO address and the date of arrival) because Evolution does not The date and time of arrival can be concluded from the last Received: header. The RCPT TO need to be converted to a X-Original-To: header. [..] > > If they bug you remove them with sed or awk or perl or python or ... > Yeah... but sed alone is not enough... cause such lines may also appear > in the body... and I mustn't remove them... > So in principle I'm looking for a smart parser of mbox which already > gives me headers and body and I can modify either. I think, like Rob suggested, you are in need of some serious scripting. Dennis From dg at dguhl.org Thu Oct 18 17:24:02 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 18 Oct 2012 16:24:02 +0200 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze In-Reply-To: References: Message-ID: <20121018142400.GA1261@PC211.ikt.de> On Wed, Oct 17, 2012 at 08:47:09PM -0300, Ricardo wrote: > Hello list [..] > I have problems installing the daemon dovecot-core, dovecot-mysql > dovecot-imapd dovecot-pop3d [..] > to install dovecot-core, dovecot-mysql, install it without problems > is the version (2.1.7-2 ~ bpo60 +1) of both packages, now wanting to > install dovecot-imapd dovecot-pop3d (version 1.2.15-7) breaks the > dovecot-core, dovecot-mysql, apparently must be the same version all > packages. Yea, of course you must use the same version for all packages. Upstream there is only one package for dovecot. It's part of Debian's philosophy to split monolithic packages into a bunch separate packages. > Debian Wheezy, installs without problems but installs the version > (dovecot-core_2.1.7-2 ~ ppa12.04 +1 _i386.deb) all packets are the > same version. This no Debian version schema but from Ubuntu 12.04. I don't know if they work correct on Debian. > What is the correct version for Debian squeeze? Stock Squeeze ships Dovecot in Debian version 1.2.15-7. Squeeze Backports offers version 2.1.7-2~bpo60+1. To install Dovecot 2.1 for Debian Squeeze: % sudo apt-get update % apt-get -s -t squeeze-backports install dovecot-imapd dovecot-pop3d dovecot-mysql The '-s' switch simulates the installation and works without root privileges. If you get no error and apt shows to install version 2.1.7-2~bpo60+1 repeat the command with a preceding 'sudo ' and no '-s'. Dennis From alessio at skye.it Thu Oct 18 17:29:50 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 18 Oct 2012 16:29:50 +0200 Subject: [Dovecot] Add S= to maildirfile Message-ID: <5080125E.5020904@skye.it> Hi, in some old Maildir/ I have file without the S= in file name. Is possibile to add the size to the file name with some tools like doveadm? Are there other methods to update these file? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From rs at sys4.de Thu Oct 18 19:22:39 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 18 Oct 2012 18:22:39 +0200 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <5080125E.5020904@skye.it> References: <5080125E.5020904@skye.it> Message-ID: <50802CCF.3000200@sys4.de> Am 18.10.2012 16:29, schrieb Alessio Cecchi: > Hi, > > in some old Maildir/ I have file without the S= in file name. > > Is possibile to add the size to the file name with some tools like doveadm? > > Are there other methods to update these file? > > Thanks > perhaps this helps for ideas http://wiki2.dovecot.org/HowTo/RefilterMail perhaps you can use dsync also , but i am really not sure if this works http://wiki2.dovecot.org/Tools/Dsync however its easy to test -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From jbates at brightok.net Thu Oct 18 19:49:05 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 18 Oct 2012 11:49:05 -0500 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50802CCF.3000200@sys4.de> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> Message-ID: <50803301.4060508@brightok.net> On 10/18/2012 11:22 AM, Robert Schetterer wrote: > Am 18.10.2012 16:29, schrieb Alessio Cecchi: >> Hi, >> >> in some old Maildir/ I have file without the S= in file name. >> >> Is possibile to add the size to the file name with some tools like doveadm? >> >> Are there other methods to update these file? >> >> Thanks >> > perhaps this helps for ideas > > http://wiki2.dovecot.org/HowTo/RefilterMail > > perhaps you can use dsync also , but i am really not sure > if this works > > http://wiki2.dovecot.org/Tools/Dsync > > however its easy to test Dsync would be the best option, I believe. It should work moving from maildir to maildir, but if necessary, you could also convert it to another format and then put it back to maildir. Jack From nanovox at gmail.com Thu Oct 18 23:32:15 2012 From: nanovox at gmail.com (Steven Kiehl) Date: Thu, 18 Oct 2012 16:32:15 -0400 Subject: [Dovecot] Emails from invalid local accounts Message-ID: Hi, I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues with receiving spam where the "from" header contains an address like accounting at mydomain.com. Is there some way I can filter out these emails coming from outside our network with an account associated with our network which doesn't exist? Do I just need to configure some custom process to evaluate these addresses, or is there some way either in dovecot or spamassassin to do this? Thanks, Steve K From noeldude at gmail.com Fri Oct 19 00:00:21 2012 From: noeldude at gmail.com (Noel) Date: Thu, 18 Oct 2012 16:00:21 -0500 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: Message-ID: <50806DE5.6050904@gmail.com> On 10/18/2012 3:32 PM, Steven Kiehl wrote: > Hi, > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > with receiving spam where the "from" header contains an address like > accounting at mydomain.com. Is there some way I can filter out these emails > coming from outside our network with an account associated with our network > which doesn't exist? Do I just need to configure some custom process to > evaluate these addresses, or is there some way either in dovecot or > spamassassin to do this? > > Thanks, > > Steve K > This should be dealt with in postfix or SpamAssassin, not dovecot, and there are likely other, better ways to detect this particular spam rather than mucking with the From: header. Sometimes mail arrives with a header something like From: accounting and postfix appends @$myorigin to the unqualified address while passing the mail through your content_filter. The fix for that is to set in your postfix main.cf remote_header_rewrite_domain = domain.invalid so that unqualified addresses will be rewritten with a known domain. Don't be tempted to reject such mail outright since you'll reject a significant amount of non-spam mail. Another thing to consider setting in postfix main.cf is: smtpd_reject_unlisted_sender = yes which will reject invalid envelope senders in your domain. (Note the difference between envelope sender and the From: header.) -- Noel Jones From ben at morrow.me.uk Fri Oct 19 00:00:27 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 18 Oct 2012 22:00:27 +0100 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: Message-ID: <20121018210026.GB5388@anubis.morrow.me.uk> At 4PM -0400 on 18/10/12 you (Steven Kiehl) wrote: > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > with receiving spam where the "from" header contains an address like > accounting at mydomain.com. Is there some way I can filter out these emails > coming from outside our network with an account associated with our network > which doesn't exist? Do I just need to configure some custom process to > evaluate these addresses, or is there some way either in dovecot or > spamassassin to do this? You want to do this in Postfix, with either the smtpd_reject_unlisted_sender parameter or the reject_unlisted_sender policy in smtpd_sender_restrictions. You will need to make sure Postfix has access to the list of valid mailboxes at your domain, which it should have already for recipient checking. Ben From stephan at rename-it.nl Fri Oct 19 02:01:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 19 Oct 2012 01:01:43 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <507BBE00.9010007@dada.eu> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> <507BBE00.9010007@dada.eu> Message-ID: <50808A57.8040201@rename-it.nl> On 10/15/2012 9:40 AM, Sandro Tosi wrote: > Hi Stephan, > thanks a lot for your reply. > > On 10/11/2012 10:35 PM, Stephan Bosch wrote: >> On 10/10/2012 11:23 AM, Sandro Tosi wrote: >>> Hello, >>> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >>> as this will be provided with an upcoming Debian package) with MySQL >>> dict lookups with the mail setup we're designing. >>> >>> Our (main) goals are: >>> >>> 1. store the filters on the database >> That is possible with some limitations. > > Are the ones below the only limitatios (ie one script per user) or are > there any other worth knowing? You cannot currently use ManageSieve when the active script is located in a dict database. And 'one script per user' is not an fully accurate description. It is technically possible to access multiple different scripts from the dict database. It is however not possible to use dict support combination with multiscript support ( http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Executing_Multiple_Scripts_Sequentially) to execute multiple scripts in a sequence. Multiscript currently only works for Sieve scripts that are located in the filesystem. > In our situation, what would you suggest? We're now thinking of > keeping the scripts list on a separate table, and merge the "user > selected ones" in a single script to write in the filters table. Is > that what would you suggest? Is there a better solution? You can use the include extension (https://tools.ietf.org/html/draft-ietf-sieve-include-05) to access scripts in a dict database from a main active script to combine them. I believe you could even dynamically construct that main script in SQL using some string manipulation in the query, but that is a bit ugly. Could you send me an overview of your configuration, including your database layout? Provided that I have some time in the next week, I could investigate building a simple working configuration for the sake of example. Regards, Stephan. From calestyo at scientia.net Fri Oct 19 02:32:59 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Fri, 19 Oct 2012 01:32:59 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1350603179.3391.21.camel@fermat.scientia.net> Hi Rick and Robert. Thanks for the tools... I'll have a look over them. :) On Wed, 2012-10-17 at 15:53 +0000, Rick Sanders wrote: > Your best bet for a clean migration is to use an IMAP migration tool (assuming > both of your servers support IMAP). It avoids all of the issues surrounding the > underlying databases used to store the mailboxes and messages since everything > is done through IMAP commands. Well the problem is that a) the mboxes are already mixed up (with respect to different formats), which was basically my fault. b) Evolution is severely broken, amongst others for this https://bugzilla.gnome.org/show_bug.cgi?id=686258 reason. So I cannot really trust that automatic migration will work. > imapsync: http://imapsync.lamiral.info > imap_tools: http://www.athensfbc.com/imap_tools > offlineimap: https://github.com/nicolas33/offlineimap > mbsync: http://isync.sourceforge.net/ > mailsync: http://mailsync.sourceforge.net/ > mailutil: http://www.washington.edu/imap/ part of the UW IMAP tookit. > imaprepl: http://www.bl0rg.net/software/ http://freecode.com/projects/imap-repl/ > imapcopy: http://home.arcor.de/armin.diehl/imapcopy/imapcopy.html > migrationtool: http://sourceforge.net/projects/migrationtool/ > imapmigrate: http://sourceforge.net/projects/cyrus-utils/ > larch: https://github.com/rgrove/larch (derived from wonko_imapsync) > wonko_imapsync: http://wonko.com/article/554 > pop2imap: http://www.linux-france.org/prj/pop2imap/ > exchange-away: http://exchange-away.sourceforge.net/ For most of them, I unfortunately didn't found information on whether they support the different subformats of mbox... what about your MboxtoIMAP.pl ? Right now I tent to create my own converter based on mb2md... just that I don't write out maildir but again mbox. Timo, when you're reading this: I'm not sure though, on which headers I must/should stripe for dovecot? From http://wiki.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata I'd guess that I have to drop all X-IMAPbase, X-IMAP and X-UID. (Will dovcote recreate them, when it indexes the mbox file the first time?) And I have to manually create/calculate, Status, X-Status, X-Keyword (based on what either Evolution or Thunderbird set) and also Content-Length... the "From_" lines in the mails need then to be _not_ quoted. Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Fri Oct 19 02:38:38 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Fri, 19 Oct 2012 01:38:38 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121018123440.GA29330@laptop-dg.leere.eu> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121018123440.GA29330@laptop-dg.leere.eu> Message-ID: <1350603518.3391.23.camel@fermat.scientia.net> On Thu, 2012-10-18 at 14:34 +0200, Dennis Guhl wrote: > [move through Evolution to IMAP] Seriously... I can just suggest anyone to never trust this piece of crap ;) Don't know which daemons led me to using it... > I think, like Rob suggested, you are in need of some serious > scripting. Yeah... guess that's what it will end up with. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From nanovox at gmail.com Fri Oct 19 04:59:40 2012 From: nanovox at gmail.com (Steven Kiehl) Date: Thu, 18 Oct 2012 21:59:40 -0400 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: <50806DE5.6050904@gmail.com> References: <50806DE5.6050904@gmail.com> Message-ID: This is great information on some options I should look into further, however adding the "smtpd_reject_unlisted_sender" option doesn't seem to eliminate the problem. What these spammers are doing is forging the "from" header to be a full address like "accounting at mydomain.com" and they are sending to a real address like "webmaster at mydomain.com". So even if the envelope sender is valid or coming from an outside domain, the visible originating from address is invalid and is in my own domain. And I'm absolutely positive any mail received from these forged from addresses are spam that shouldn't even be delivered. This is also complicated further by the use of virtual domains and virtual alias mapping (all sql based) in the Postfix configuration. Some of my problem may be that Postfix might not be able to get a comprehensive list of valid mailboxes and aliases to deliver to the virtual transport. I've tried to define the virtual mailbox maps, but every time I do that the aliases stop working. On Thu, Oct 18, 2012 at 5:00 PM, Noel wrote: > On 10/18/2012 3:32 PM, Steven Kiehl wrote: > > Hi, > > > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > > with receiving spam where the "from" header contains an address like > > accounting at mydomain.com. Is there some way I can filter out these > emails > > coming from outside our network with an account associated with our > network > > which doesn't exist? Do I just need to configure some custom process to > > evaluate these addresses, or is there some way either in dovecot or > > spamassassin to do this? > > > > Thanks, > > > > Steve K > > > > This should be dealt with in postfix or SpamAssassin, not dovecot, > and there are likely other, better ways to detect this particular > spam rather than mucking with the From: header. > > Sometimes mail arrives with a header something like > From: accounting > and postfix appends @$myorigin to the unqualified address while > passing the mail through your content_filter. > > The fix for that is to set in your postfix main.cf > remote_header_rewrite_domain = domain.invalid > so that unqualified addresses will be rewritten with a known > domain. Don't be tempted to reject such mail outright since you'll > reject a significant amount of non-spam mail. > > Another thing to consider setting in postfix main.cf is: > smtpd_reject_unlisted_sender = yes > which will reject invalid envelope senders in your domain. (Note > the difference between envelope sender and the From: header.) > > > > -- Noel Jones > From noeldude at gmail.com Fri Oct 19 06:50:30 2012 From: noeldude at gmail.com (Noel) Date: Thu, 18 Oct 2012 22:50:30 -0500 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: <50806DE5.6050904@gmail.com> Message-ID: <5080CE06.1080706@gmail.com> On 10/18/2012 8:59 PM, Steven Kiehl wrote: > This is great information on some options I should look into > further, however adding the "smtpd_reject_unlisted_sender" > option doesn't seem to eliminate the problem. [This is OT for the dovecot list, and my last post in this thread. Please send all followups to the appropriate postfix, amavisd-new, or spamassassin list in consideration of other list members. Thank you.] smtpd_reject_unlisted_sender works with the envelope address; this option has no effect on headers. > What these spammers are doing is forging the "from" header to be > a full address like "accounting at mydomain.com > " Possible, but I doubt it. The only way you'll ever see the more likely original "From: accounting" header is by running postfix in debug mode (which is not recommended) or by using a tcp sniffer in front of postfix. That's why I recommend setting "remote_header_rewrite_domain = domain.invalid". Also, this setting requires a non-ancient postfix, but I don't remember which version; if it shows up in "postconf -n" output, you're OK. > and they are sending to a real address like > "webmaster at mydomain.com ". So even > if the envelope sender is valid or coming from an outside domain, > the visible originating from address is invalid and is in my own > domain. And I'm absolutely positive any mail received from these > forged from addresses are spam that shouldn't even be delivered. If there are a few frequently-abused addresses, you can add them to a header_checks rule. But don't get too tied up in wack-a-mole header_checks; that's a great time waster for limited benefit. > This is also complicated further by the use of virtual domains and > virtual alias mapping (all sql based) in the Postfix > configuration. Some of my problem may be that Postfix might not > be able to get a comprehensive list of valid mailboxes and aliases > to deliver to the virtual transport. I've tried to define the > virtual mailbox maps, but every time I do that the aliases stop > working. If your postfix is not able to properly validate recipients, you should ask about that on the postfix list. That is a serious problem. http://www.postfix.org/DEBUG_README.html#mail The point you're missing is that there is no way to validate the From: header. Look at other features of the unwanted mail for ways to reject it. -- Noel Jones From tomislav.mihalicek at gmail.com Fri Oct 19 10:40:50 2012 From: tomislav.mihalicek at gmail.com (tmihalicek) Date: Fri, 19 Oct 2012 00:40:50 -0700 (PDT) Subject: [Dovecot] Dovecot quota postgres dictionary problems Message-ID: <1350632450161-38234.post@n4.nabble.com> I have a strange errors in .err log file, but the postgres seem to be filling with quota changes, i will also put configs in Oct 19 09:23:52 mailstore-node-01 dovecot: imap(test at example.net): Error: read(/var/run/dovecot/dict) failed: Timeout after 30 seconds Oct 19 09:24:22 mailstore-node-01 dovecot: imap(test at example.net): Error: read(/var/run/dovecot/dict) failed: Timeout after 30 seconds Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Panic: file dict-client.c: line 270 (client_dict_finish_transaction): assertion failed: (dict->async_commits > 0) Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x484ea) [0x7fbed405d4ea] -> /usr/lib/dovecot/libdovecot.so.0(+0x48536) [0x7fbed405d536] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fbed4030eaf] -> /usr/lib/dovecot/libdovecot.so.0(+0x22337) [0x7fbed4037337] -> /usr/lib/dovecot/libdovecot.so.0(+0x2236b) [0x7fbed403736b] -> /usr/lib/dovecot/libdovecot.so.0(+0x22e78) [0x7fbed4037e78] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0x8a3f) [0x7fbed2c76a3f] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7fbed2c73262] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(quota_transaction_commit+0x1e7) [0x7fbed2c738d7] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xb39f) [0x7fbed2c7939f] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xb4f4) [0x7fbed2c794f4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x2a) [0x7fbed432396a] -> dovecot/imap [test at example.net 10.84.34.2 expunge](imap_sync_deinit+0x4d) [0x418edd] -> dovecot/imap [test at example.net 10.84.34.2 expunge]() [0x41918c] -> dovecot/imap [test at example.net 10.84.34.2 expunge](cmd_sync_delayed+0x1f5) [0x4195b5] -> dovecot/imap [test at example.net 10.84.34.2 expunge](client_handle_input+0x1fd) [0x41127d] -> dovecot/imap [test at example.net 10.84.34.2 expunge](client_input+0x5f) [0x411adf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7fbed40696c6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7fbed406a6ff] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7fbed4069668] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fbed4055043] -> dovecot/imap [test at example.net 10.84.34.2 expunge](main+0x2a4) [0x419d24] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fbed3cd1c8d] -> dovecot/imap [test at example.net 10.84.34.2 expunge]() [0x409059] doveconf.txt dovecot-dict-sql.conf.ext -- View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-quota-postgres-dictionary-problems-tp38234.html Sent from the Dovecot mailing list archive at Nabble.com. From amateo at um.es Fri Oct 19 15:38:36 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 19 Oct 2012 14:38:36 +0200 Subject: [Dovecot] Auth caching and password changes Message-ID: <508149CC.9070004@um.es> Hello, In my system I have configured auth caching. The problem I have is that whenever a user changes his password, he/she can't login to dovecot after a while and the scenarios described at http://wiki2.dovecot.org/Authentication/Caching are not applied. I have tried also with "doveadm auth cache flush ", but it didn't work. He also could to login again if he waits for a time or if I run "doveadm auth cache flush" in the server, flushing all auth information from cache. I have attached the log I had when I changed my password (and suffered the problem). I have attached my doveconf -n too. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 -------------- next part -------------- A non-text attachment was scrubbed... Name: cambioclave.log Type: text/x-log Size: 1349 bytes Desc: not available URL: -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota zlib mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 3075 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From alessio at skye.it Fri Oct 19 19:12:26 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 19 Oct 2012 18:12:26 +0200 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50803301.4060508@brightok.net> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> <50803301.4060508@brightok.net> Message-ID: <50817BEA.6090201@skye.it> Il 18/10/2012 18:49, Jack Bates ha scritto: > On 10/18/2012 11:22 AM, Robert Schetterer wrote: >> Am 18.10.2012 16:29, schrieb Alessio Cecchi: >>> Hi, >>> >>> in some old Maildir/ I have file without the S= in file name. >>> >>> Is possibile to add the size to the file name with some tools like >>> doveadm? >>> >>> Are there other methods to update these file? >>> >>> Thanks >>> >> perhaps this helps for ideas >> >> http://wiki2.dovecot.org/HowTo/RefilterMail >> >> perhaps you can use dsync also , but i am really not sure >> if this works >> >> http://wiki2.dovecot.org/Tools/Dsync >> >> however its easy to test > > Dsync would be the best option, I believe. It should work moving from > maildir to maildir, but if necessary, you could also convert it to > another format and then put it back to maildir. > > Jack > Dsync could be a good idea but I need to "sync" maildir to the same maildir and I don't think can be done this. My situation is a Maildir/cur/ with old email files without S=size and, in the same Maildir/cur/, some files with S=size. Can doveadm do this? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From wamp at promax.media.pl Fri Oct 19 15:32:20 2012 From: wamp at promax.media.pl (wamp) Date: Fri, 19 Oct 2012 05:32:20 -0700 (PDT) Subject: [Dovecot] Dovecot 2 quota limit and actual size (mysql) Message-ID: <1350649940026-38235.post@n4.nabble.com> Hello, Can You explain to me how dovecot-lda knows actual size of virtual user directory? I want to keep max size of user directory in mysql - should I also use some kind of script to upgrade actual size information in mysql ? I read docs from wiki but still dont know it. thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-2-quota-limit-and-actual-size-mysql-tp38235.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Fri Oct 19 19:17:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 19 Oct 2012 19:17:04 +0300 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50817BEA.6090201@skye.it> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> <50803301.4060508@brightok.net> <50817BEA.6090201@skye.it> Message-ID: <9BE333EF-2120-4581-9A51-79C08EAF9085@iki.fi> On 19.10.2012, at 19.12, Alessio Cecchi wrote: >>>> in some old Maildir/ I have file without the S= in file name. >>>> >>>> Is possibile to add the size to the file name with some tools like doveadm? Not directly. >>>> Are there other methods to update these file? A script that renames the files and updates dovecot-uidlist. No such script exists as far as I know. You could also switch from Maildir++ quota to dict-file quota and this wouldn't be a problem. > Dsync could be a good idea but I need to "sync" maildir to the same maildir and I don't think can be done this. You could sync to another maildir, rm -rf the original, sync back to original. > My situation is a Maildir/cur/ with old email files without S=size and, in the same Maildir/cur/, some files with S=size. > > Can doveadm do this? No. From tobias at maffert.net Fri Oct 19 21:47:20 2012 From: tobias at maffert.net (Tobias Maffert) Date: Fri, 19 Oct 2012 20:47:20 +0200 Subject: [Dovecot] Question about salted hashes Message-ID: <5081A038.8070908@maffert.net> Hello. I'm switching from b1gmail to my own setup which consists of Postfix+Dovecot+MySQL (and maybe VBoxAdm). There are two problems: - b1gmail is using unsalted MD5 hashes. Is there any good way to make my new setup backward compatible? So I don't have to force all of my 50k users to change their password. - How do I change my setup to salted SHA256 (or an even better algorithm). And how do I make the hashes compatible between Postfix, Dovecot, MySQL (and maybe VBoxAdm)? My system: Debian Squeeze Postfix version: 2.7.1 Dovecot version: 1.2.15 ----------------------------------------------- dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.6 protocols: imap pop3 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_access_groups: vmail mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): UID%u-%v lda: postmaster_address: auth_socket_path: /var/run/dovecot/auth-master mail_plugins: quota sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain login user: vmail passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir ----------------------------------------------- driver = mysql connect = host=127.0.0.1 dbname=smail user=smail password=mypw default_pass_scheme = CRYPT password_query = SELECT username AS user, password_enc AS password, CONCAT(homedir, maildir) AS userdb_home, uid AS userdb_uid, gid AS userdb_gid, CONCAT('maildir:', homedir, maildir) AS userdb_mail, CONCAT('maildir:storage=', (quota*1024)) as userdb_quota FROM mail_users WHERE (username = '%u' OR email = '%u') AND ((imap = 1 AND '%Ls' = 'imap') OR (pop3 = 1 AND '%Ls' = 'pop3') OR '%Ls' = 'smtp') user_query = SELECT CONCAT(homedir, maildir) AS home, CONCAT('maildir:', homedir, maildir) AS mail, uid, gid, CONCAT('*:storage=', (quota*1024)) as quota_rule FROM mail_users WHERE (username = '%u' OR email = '%u') ----------------------------------------------- Regards, Tobi M. From h.reindl at thelounge.net Fri Oct 19 21:50:28 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 19 Oct 2012 20:50:28 +0200 Subject: [Dovecot] Question about salted hashes In-Reply-To: <5081A038.8070908@maffert.net> References: <5081A038.8070908@maffert.net> Message-ID: <5081A0F4.9090704@thelounge.net> Am 19.10.2012 20:47, schrieb Tobias Maffert: > Hello. > > I'm switching from b1gmail to my own setup which consists of > Postfix+Dovecot+MySQL (and maybe VBoxAdm). There are two problems: - > b1gmail is using unsalted MD5 hashes. Is there any good way to make my > new setup backward compatible? So I don't have to force all of my 50k > users to change their password. - How do I change my setup to salted > SHA256 (or an even better algorithm). And how do I make the hashes > compatible between Postfix, Dovecot, MySQL (and maybe VBoxAdm)? > > My system: > > Debian Squeeze > Postfix version: 2.7.1 > Dovecot version: 1.2.15 besides your question you REALLY do not want to START with 1.2.15 while dovecot-2.1.10 is the recent version postfix is somehow OK but recent is postfix-2.9.4 these outdated versions usually results in many questions on mailing-lists because several howtos do not work or bugs fixed since years are still there -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From dave.mehler at gmail.com Fri Oct 19 23:11:07 2012 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 19 Oct 2012 16:11:07 -0400 Subject: [Dovecot] still having difficulties with per-user quotas Message-ID: Hello, I am trying to get per-user quotas working. My thanks to all who have helped so far. To recap I am running Dovecot 2.1 and Mysql where I've got my virtual users. All virtual users are under the system user vmail with a UID and GID of 5000. Looking over the wiki docs I've added a quota table and got the dict service working, I am not having problems with permissions or the login username and password, all that is working fine. Here's my current doveconf -n output it is producing the following debug error related to the userdb sql query: # 2.1.10: /etc/dovecot/dovecot.conf # XXX dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 hostname = XXX last_valid_gid = 5000 last_valid_uid = 5000 mail_location = maildir:/home/vmail/%d/%n:LAYOUT=fs mail_plugins = " quota" namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = xxx xxxx ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_cert = , method=PLAIN, rip=::1, lip=::1, mpid=29282, secured, session= Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User is missing UID (see mail_uid setting) Oct 19 15:23:52 imap(xxx): Error: Internal error occurred. Refer to server log for more information. I am wanting a majority of my users to have the global 1GB quota, but the users in the quota table to have given quotas. Here's what the virtual_users and quota tables look like: mysql> describe virtual_users; +-----------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | domain_id | int(11) | NO | MUL | NULL | | | user | varchar(40) | NO | | NULL | | | password | varchar(128) | NO | | NULL | | +-----------+--------------+------+-----+---------+----------------+ 4 rows in set (0.00 sec) mysql> describe quota; +----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ | username | varchar(100) | NO | PRI | NULL | | | bytes | bigint(20) | NO | | 0 | | | messages | int(11) | NO | | 0 | | +----------+--------------+------+-----+---------+-------+ 3 rows in set (0.00 sec) I'd appreciate any help. Thanks. Dave. From emailbuilder88 at yahoo.com Fri Oct 19 23:43:29 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Fri, 19 Oct 2012 13:43:29 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? Message-ID: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Hello, I'm having some problems getting LDA to work without userdb lookups and have a few related questions. This system has all users in MySQL, each user with unique UID/GID, no local users at all.? Installation is from apt-get. 1) If LDA is invoked without lookups, is it correct to assume that the "service auth" and "service auth-worker" can be completely removed from dovecot master configuration? (I have tried commenting them out and logging into IMAP, which seems to work, not sure if anyone else needs the auth service) 2) If LDA is invoked without lookups, will I be unable to use Dovecot quota plugin? Does it need to have a user lookup to get quota info? (haven't added quota support, need to take this one step at a time) 3) The interesting part -- I am invoking LDA from Maildrop. See: http://thread.gmane.org/gmane.mail.imap.dovecot/65473 So when invoked, Maildrop has already dropped to the destination UID/GID and the needed paths are available in the environment.? However, using as many permutations of calling LDA as I can think of (based on ??? http://wiki2.dovecot.org/LDA ), I always get this: (command line usage error. Command output: lda: Fatal: Couldn't lookup our username (uid=2500) ) The UID is correct for the target user. If I add "-d $LOGNAME" to my LDA callout, I get permission denied on the userdb lookup, which I guess is another issue to work out if I want to go with lookups. But right now I am trying not to. Why does LDA seem to try for a lookup even when I follow the wiki instructions how to call it without a lookup? 3.5) Related question, my users have separate homedir and maildir, both paths are looked up by Maildrop. I think I need to call LDA with "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 14:45:20 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 13:45:20 +0200 Subject: [Dovecot] Dovecot 2 quota limit and actual size (mysql) In-Reply-To: <1350649940026-38235.post@n4.nabble.com> References: <1350649940026-38235.post@n4.nabble.com> Message-ID: <20121020114520.GA26196@daniel.localdomain> Use LMTP instead of lda. The dovecot lmtp service automatically cares about updating quota values in mysql database when mail arrives through the lmtp socket. Regards Daniel wamp wrote: > Hello, > Can You explain to me how dovecot-lda knows actual size of virtual user > directory? I want to keep > max size of user directory in mysql - should I also use some kind of script > to upgrade actual size information in mysql ? > > I read docs from wiki but still dont know it. > > > thanks > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-2-quota-limit-and-actual-size-mysql-tp38235.html > Sent from the Dovecot mailing list archive at Nabble.com. > -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 15:47:49 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 14:47:49 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: Message-ID: <20121020124749.GA26942@daniel.localdomain> David Mehler wrote: > Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User is missing UID (see mail_uid setting) Set the following options in your dovecot.conf: mail_uid = vmail mail_gid = vmail Also see section "Mail users" at http://wiki2.dovecot.org/UserIds Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 16:51:44 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 15:51:44 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: References: Message-ID: <20121020135144.GA28609@daniel.localdomain> Jan-Frode Myklebust wrote: > $ cat /etc/dovecot/dovecot-trash.conf.ext > # Spam mailbox is emptied before Trash > 1 INBOX.Spam > # Trash mailbox is emptied before Sent > 2 INBOX.Trash Are you sure the Trash Folder of the affected users is located below "INBOX"? doveadm mailbox list -u user at domain | grep -iE "trash|spam" Example at http://wiki2.dovecot.org/Plugins/Trash omits "INBOX." Have you tried INBOX/Trash as mailbox name? Regards Daniel -- https://plus.google.com/103021802792276734820 From sven at svenhartge.de Sat Oct 20 19:39:22 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sat, 20 Oct 2012 18:39:22 +0200 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive Message-ID: <09718hdveev8@mids.svenhartge.de> Hi! I am about to migrate a perdition-based IMAP/POP3 proxy to Dovecot. Unfortunately some users are behind a firewall/NAT setup which throws away seemingly idle TCP connections sooner than the established default of 24 hours (more likely after 30 minutes ...) resulting in all kinds of weird client behavior. And unfortunately? this firewall/NAT setup is outside of my control and I have no means of correcting this (in my opinion) flawed configuration. Now, with perdition I was able to use the --tcp_keepalive option which totally solved the mentioned weird client behavior. My question is: does Dovecot2 use TCP-Keepalive on its sockets per default or do I need to enable it some way I have not yet discovered? The manual and wiki only talk about "keepalive" in connection with the IMAP protocol and IDLE and my C-fu is too weak to understand the source code. Gr??e, Sven. -- Sigmentation fault. Core dumped. From tss at iki.fi Sat Oct 20 20:02:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 20 Oct 2012 20:02:36 +0300 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive In-Reply-To: <09718hdveev8@mids.svenhartge.de> References: <09718hdveev8@mids.svenhartge.de> Message-ID: <9881D0A3-70EB-454B-A71D-9C7A253AE21E@iki.fi> On 20.10.2012, at 19.39, Sven Hartge wrote: > I am about to migrate a perdition-based IMAP/POP3 proxy to Dovecot. > > Unfortunately some users are behind a firewall/NAT setup which throws > away seemingly idle TCP connections sooner than the established default > of 24 hours (more likely after 30 minutes ...) resulting in all kinds of > weird client behavior. > > And unfortunately? this firewall/NAT setup is outside of my control and > I have no means of correcting this (in my opinion) flawed configuration. > > Now, with perdition I was able to use the --tcp_keepalive option which > totally solved the mentioned weird client behavior. > > My question is: does Dovecot2 use TCP-Keepalive on its sockets per > default or do I need to enable it some way I have not yet discovered? It's the default yes. Of course Linux's default keepalive interval is something like 90 minutes, so have you changed that already?.. > The manual and wiki only talk about "keepalive" in connection with the > IMAP protocol and IDLE and my C-fu is too weak to understand the source > code. imap_idle_notify_interval (default 2 min) causes Dovecot to send data to IDLEing connections, which pretty much makes the TCP keepalive irrelevant. For non-IDLE connections Dovecot has a disconnect timeout of 30 minutes. From dave.mehler at gmail.com Sat Oct 20 20:06:59 2012 From: dave.mehler at gmail.com (David Mehler) Date: Sat, 20 Oct 2012 13:06:59 -0400 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: <20121020124749.GA26942@daniel.localdomain> References: <20121020124749.GA26942@daniel.localdomain> Message-ID: Hello, Thank you for your reply. Adding mail_uid and mail_gid fixed it. I now have quotas going but I don't know if I have them right or just don't like my setup. My first issue is from what it is looking like I have to define all my users in the quota database not just the ones whose values I want to override the global quota declaration in 90-quota.conf. If I just add the user at domain to the database the bytes and messages columns have zero as default, this means those values override global quota in 90-quota.conf and they effectively have unlimited access. My second issue is I have entered a quota of 250 megabytes for a test user. This works but he seems to get more space everytime he logs in, started out at 250, on the next login it was 255, then 269 on the third, and so forth. I've checked the quota table and yes the value in the bytes column is increasing. Thanks for any help. Dave. On 10/20/12, Daniel Parthey wrote: > David Mehler wrote: >> Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User >> is missing UID (see mail_uid setting) > > Set the following options in your dovecot.conf: > > mail_uid = vmail > mail_gid = vmail > > Also see section "Mail users" at > http://wiki2.dovecot.org/UserIds > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From sven at svenhartge.de Sat Oct 20 20:15:25 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sat, 20 Oct 2012 19:15:25 +0200 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive References: <09718hdveev8@mids.svenhartge.de> <9881D0A3-70EB-454B-A71D-9C7A253AE21E@iki.fi> Message-ID: <1971afdveev8@mids.svenhartge.de> Timo Sirainen wrote: > On 20.10.2012, at 19.39, Sven Hartge wrote: >> My question is: does Dovecot2 use TCP-Keepalive on its sockets per >> default or do I need to enable it some way I have not yet discovered? > It's the default yes. Of course Linux's default keepalive interval is > something like 90 minutes, so have you changed that already?.. Yes, I did. For those systems it is set to 15 minutes right now. >> The manual and wiki only talk about "keepalive" in connection with >> the IMAP protocol and IDLE and my C-fu is too weak to understand the >> source code. > imap_idle_notify_interval (default 2 min) causes Dovecot to send data > to IDLEing connections, which pretty much makes the TCP keepalive > irrelevant. For non-IDLE connections Dovecot has a disconnect timeout > of 30 minutes. This is fine. As long as the client notices the termination of the connection, everything should be OK. Before I switched keepalive on for Perdition, the firewall/NAT would internally throw away a connection but neither the client or the server would notice this. Then if the client tried to do something with this connection, like select or save a message, the firewall/NAT would send a RST and the client would then bug the user with a meaningless message like "folder does not exist" which caused a lot of confusion for the end-user and created quite the bit of trouble tickets. This problem mostly happend with an IMAP connection to the "Sent Messages" folder which normally does not see much changes until the users writes and sends a mail. Then after the mail was sent via SMTP the client tries to save the message, gets sent an RST from the firewall/NAT and presents the user with a wrong and confusing error message. The user then thinks his mail was not sent and sends it again. This time the client opens a new connection to select the "Sent Messages" folder and everything works. But the recipient gets the mail twice. Again resulting in confusion and trouble tickets to be dealt with. By switching to TCP keepalive (and reducing the keepalive time to 15 minutes) all those problems were solved and my users (and support staff) were happy again ;) Gr??e, Sven. -- Sigmentation fault. Core dumped. From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 21:51:24 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 20:51:24 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> Message-ID: <20121020185124.GA2609@daniel.localdomain> David Mehler wrote: > My first issue is from what it is looking like I have to define all my > users in the quota database not just the ones whose values I want to > override the global quota declaration in 90-quota.conf. If I just add > the user at domain to the database the bytes and messages columns have > zero as default, this means those values override global quota in > 90-quota.conf and they effectively have unlimited access. This is expected behavior. If the userdb returns a quota rule, it overrides the global quota rule. Extend your SQL query to return a default quota_rule for rows without quota entry. > My second issue is I have entered a quota of 250 megabytes for a test > user. This works but he seems to get more space everytime he logs in, > started out at 250, on the next login it was 255, then 269 on the > third, and so forth. I've checked the quota table and yes the value in > the bytes column is increasing. Please show output of doveconf -n and any external (sql/dict) includes related to quota or quota_rules. Regards Daniel -- https://plus.google.com/103021802792276734820 From emailbuilder88 at yahoo.com Sat Oct 20 22:17:12 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Sat, 20 Oct 2012 12:17:12 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Message-ID: <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> > 1) If LDA is invoked without > lookups, is it correct to assume that the "service auth" and > "service > auth-worker" can be completely removed from dovecot master > configuration? (I have tried commenting them out and logging into IMAP, > which seems to work, not sure if anyone else needs the auth service) Any confirmation on this? > 2) > If LDA is invoked without lookups, will I be unable to use Dovecot > quota plugin? Does it need to have a user lookup to get quota info? > (haven't added quota support, need to take this one step at a time) I'm especially interested if someone can comment on this, since maybe it makes my efforts here wasted > 3) The interesting part -- I am invoking LDA from Maildrop. See: > http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So > when invoked, Maildrop has already dropped to the destination UID/GID > and the needed paths are available in the environment.? However, using > as many permutations of calling LDA as I can think of (based on ??? > http://wiki2.dovecot.org/LDA ), I always get this: > > (command line usage error. Command output: lda: Fatal: Couldn't lookup our > username (uid=2500) ) I could not find anything in the mailing list archives to help me, but I googled and found a link to a source file: http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 I read the source, it looks like it's not exactly a userdb lookup - LDA is trying to get the unix username for the given UID. In my case, UIDs are "virtual" so there isn't a unix username. The source doesn't really use the username that it looks up except in a call "open_logfile." Is it possible to avoid this problem? It looks like the answer is no, I have to use -d which also forces a userdb lookup. Maybe this limitation can be removed in the future? Now I suppose I have to go understand the problems of userdb lookup permissions, but I think there are solutions for that. Am I on the right understanding?? ? > The > UID is correct for the target user. If I add "-d $LOGNAME" to my LDA > callout, I get permission denied on the userdb lookup, which I guess is > another issue to work out if I want to go with lookups. But right now I > am trying not to. Why does LDA seem to try for a lookup even when I > follow the wiki instructions how to call it without a lookup? > > 3.5) > Related question, my users have separate homedir and maildir, both > paths are looked up by Maildrop. I think I need to call LDA with > "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? > From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 23:04:32 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 22:04:32 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> <20121020185124.GA2609@daniel.localdomain> Message-ID: <20121020200432.GA3161@daniel.localdomain> David Mehler wrote: > Thanks for your reply. So with the extending of the query to return a > default quota rule, do you have an example of that by the way, does > that mean I only have to put the overrided users in the quota table? Assuming that quota values are in the dovecot_users table... # passdb with userdb prefetch and default quota of 1024M for quota=0 rows # The userdb_ prefix is for prefetch userdb entries in password_query password_query = SELECT username AS user, \ password AS password, \ home AS userdb_home, \ uid AS userdb_uid, \ gid AS userdb_gid, \ CASE quota \ WHEN 0 \ THEN '*:bytes=1024M:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota AS CHAR), 'M:messages=', CAST(quota_message AS CHAR)) \ END AS `userdb_quota_rule` \ FROM dovecot_users \ WHERE username='%u'; # user_query with default quota of 1024M for quota=0 rows user_query = SELECT username AS user, \ home AS home, \ uid AS uid, \ gid as gid, \ CASE quota \ WHEN 0 \ THEN '*:bytes=1024M:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota AS CHAR), 'M:messages=', CAST(quota_message AS CHAR)) \ END AS `quota_rule` \ FROM dovecot_users \ WHERE username='%u'; Your user_query needs to return a row if the user exists, otherwise dovecot will assume that the user does not exist and the mail or user will be rejected. Regards Daniel -- https://plus.google.com/103021802792276734820 From emailbuilder88 at yahoo.com Sat Oct 20 23:34:12 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Sat, 20 Oct 2012 13:34:12 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> Message-ID: <1350765252.74118.YahooMailNeo@web39301.mail.mud.yahoo.com> >> 3) The interesting part -- I am invoking LDA from Maildrop. See: >> http://thread.gmane.org/gmane.mail.imap.dovecot/65473 >> So >> when invoked, Maildrop has already dropped to the destination UID/GID >> and the needed paths are available in the environment.? However, using >> as many permutations of calling LDA as I can think of (based on ??? >> http://wiki2.dovecot.org/LDA ), I always get this: >> >> (command line usage error. Command output: lda: Fatal: Couldn't lookup > our >> username (uid=2500) ) > > I could not find anything in the mailing list archives to help me, but I googled > and found a link to a source file: > > http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 > > I read the source, it looks like it's not exactly a userdb lookup - LDA is > trying to get the unix username for the given UID. In my case, UIDs are > "virtual" so there isn't a unix username. The source doesn't > really use the username that it looks up except in a call > "open_logfile." > > Is it possible to avoid this problem? It looks like the answer is no, I have to > use -d which also forces a userdb lookup. Maybe this limitation can be removed > in the future? Now I suppose I have to go understand the problems of userdb > lookup permissions, but I think there are solutions for that. FWIW, in this scenario, "service auth" in master config has to have its mode relaxed to 0606 to make userdb lookups work.? So ANYONE on the machine can see all userdb lookups.? I don't have local users here, so it's probably safe anyway(?). Can anyone explain if there are other security risks of running the auth service at 0606? From jeff at bubble.org Sun Oct 21 04:52:01 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Sat, 20 Oct 2012 21:52:01 -0400 Subject: [Dovecot] Configuring Dovecot & Snarf plugin for the first time Message-ID: <50835541.8000808@bubble.org> I've been using uw-imap for some time on my linux system and have been running into issues with it so I've decided to move to Dovecote, so far it seems to have solved the issues I've been having however I need/want to move the incoming emails out of /var/spool/mail/{user} in the same (or similar fashion) that uw-imap did, and I found the snarf plugin. However whenever I enable the snarf plugin using the example on the wiki page my email is not loaded and when I remove my configuration for snarf my email re-appears. Based upon what I can tell the snarf plugin is either not loading (but I see it listed in the logs) or simply not working (which is probably because its not configured properly). The system is Fedora 16 (x86_64), the MTA is Exim, and Dovecot is 2.0.21 (version supplied with Fedora). I know I'm making a newbie mistake. Any guidance would be appreciated. Thanks, Jeff dovecot -n provides the following: [root at xyzzy conf.d]# more /tmp/dovecot.changes # 2.0.21: /etc/dovecot/dovecot.conf # OS: Linux 3.4.11-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mbox mail_plugins = " zlib" mbox_write_locks = fcntl namespace default { inbox = yes location = prefix = separator = / } namespace snarf { hidden = yes list = no location = mbox:/run/dovecot/empty:INBOX=/var/spool/mail/%u prefix = /snarf separator = / } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename snarf = = /snarf/INBOX } service imap-login { inet_listener imap { address = localhost } } service pop3-login { inet_listener pop3 { address = localhost } } ssl_cert = , method=PLAIN, rip=98.109.156.118, lip=132.238.254.34, mpid=19627, TLS Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= Oct 20 21:29:46 xyzzy dovecot: imap-login: Login: user=, method=PLAIN, rip=98.109.156.118, lip=132.238.254.34, mpid=19629, TLS Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= From nicolas at devels.es Sun Oct 21 14:15:37 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 12:15:37 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log Message-ID: <5083D959.5090407@devels.es> Hi list! I'm using Dovecot along with Postfix (with MySQL) and I'm having some curious error messages in the mail log. Everything runs normally and suddenly I start viewing messages like these: Oct 12 14:24:23 dovecot: last message repeated 5 times Oct 12 14:25:23 dovecot: last message repeated 6 times Oct 12 14:26:23 dovecot: last message repeated 6 times Oct 12 14:27:23 dovecot: last message repeated 6 times Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory Oct 12 14:30:23 dovecot: last message repeated 5 times Oct 12 14:31:23 dovecot: last message repeated 6 times Oct 12 14:33:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory Oct 12 14:34:23 dovecot: last message repeated 5 times Oct 12 14:35:23 dovecot: last message repeated 6 times Oct 12 14:36:01 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory This file which Dovecot references to indeed exists, is accesible, etc. Once I get the first of these messages, the user stops receiving e-mails (the mail server keeps receiving them normally, though) until he restarts the mail client. Then the error msg is gone and he receives all their unreceived messages. This error appears in 6-12h. intervals once the user starts their client. Version is 2.0.19, and dovecot -n is: root at mail:~# dovecot -n # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-32-generic-pae i686 Ubuntu 12.04.1 LTS ext4 disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir mailbox_idle_check_interval = 2 mins namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = root } ssl_cert = Hi everyone, short version: Is there no built in failover mechanism for the director service to handle a backend failure? Long version: I have a frontend server running the director service and two backends. Due to maintenance I had to shut down one of the backends which caused connection errors for the users being directed to this backend. I was very surprised as I expected the director to redirect these users to the remaining backend. Am I wrong or is the director not working as expected? Regards Patrick # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain login director_mail_servers = 172.17.1.1 172.17.1.2 director_servers = 172.17.1.3 172.17.1.4 lmtp_proxy = yes log_path = /var/log/dovecot.log managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 172.17.1.3 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 172.17.1.3 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = References: <5083D959.5090407@devels.es> Message-ID: On 21.10.2012, at 14.15, Nicol?s wrote: > Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? From nicolas at devels.es Sun Oct 21 14:29:39 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 12:29:39 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: References: <5083D959.5090407@devels.es> Message-ID: <5083DCA3.8000808@devels.es> El 21/10/2012 12:26, Timo Sirainen escribi?: > On 21.10.2012, at 14.15, Nicol?s wrote: > >> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory > fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? > No, on that machine I'm just using local filesystem, no NFS. From tss at iki.fi Sun Oct 21 16:58:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 21 Oct 2012 16:58:36 +0300 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <5083DCA3.8000808@devels.es> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> Message-ID: <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> On 21.10.2012, at 14.29, Nicol?s wrote: > El 21/10/2012 12:26, Timo Sirainen escribi?: >> On 21.10.2012, at 14.15, Nicol?s wrote: >> >>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >> > > No, on that machine I'm just using local filesystem, no NFS. Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. From janfrode at tanso.net Sun Oct 21 19:45:18 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 21 Oct 2012 18:45:18 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: <20121020135144.GA28609@daniel.localdomain> References: <20121020135144.GA28609@daniel.localdomain> Message-ID: On Sat, Oct 20, 2012 at 3:51 PM, Daniel Parthey wrote: > Jan-Frode Myklebust wrote: >> $ cat /etc/dovecot/dovecot-trash.conf.ext >> # Spam mailbox is emptied before Trash >> 1 INBOX.Spam >> # Trash mailbox is emptied before Sent >> 2 INBOX.Trash > > Are you sure the Trash Folder of the affected users is located below "INBOX"? > doveadm mailbox list -u user at domain | grep -iE "trash|spam" $ sudo doveadm mailbox list -u XXXXX at example.no INBOX INBOX.Drafts INBOX.Sent INBOX.Spam INBOX.Trash > Example at http://wiki2.dovecot.org/Plugins/Trash omits "INBOX." > Have you tried INBOX/Trash as mailbox name? No, should I, when my prefix is "INBOX." and separator is "." ? namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } BTW: I think it's mostly working.. as the number or quota exceeded messages has clearly dropped since implementing it, but I do find a few users that get quota exceeded and has lots of messages in INBOX.Trash og INBOX.Spam.. -jf From nicolas at devels.es Sun Oct 21 20:58:46 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 18:58:46 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> Message-ID: <508437D6.8020305@devels.es> El 21/10/2012 14:58, Timo Sirainen escribi?: > On 21.10.2012, at 14.29, Nicol?s wrote: > >> El 21/10/2012 12:26, Timo Sirainen escribi?: >>> On 21.10.2012, at 14.15, Nicol?s wrote: >>> >>>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >>> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >>> >> No, on that machine I'm just using local filesystem, no NFS. > Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. > Well, at least it's good to know it's not a misconfiguration :-) If there's any additional info I can provide feel free to ask. Regards, Nicol?s From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 21 22:43:20 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 21 Oct 2012 21:43:20 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <5083D963.3000700@wk-serv.de> References: <5083D963.3000700@wk-serv.de> Message-ID: <20121021194320.GA7977@daniel.localdomain> Patrick Westenberg wrote: > Is there no built in failover mechanism for the director service to > handle a backend failure? No, the director's job is to keep a hash table and direct the connection for each user to its associated backend. Currently, there is no built-in backend monitoring. In order to handle maintenance of backends, you will need the poolmon daemon, which enables/disables backends in the director depending on their availability: https://github.com/brandond/poolmon Regards Daniel -- https://plus.google.com/103021802792276734820 From tss at iki.fi Sun Oct 21 22:45:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 21 Oct 2012 22:45:46 +0300 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <508437D6.8020305@devels.es> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> <508437D6.8020305@devels.es> Message-ID: On 21.10.2012, at 20.58, Nicol?s wrote: > El 21/10/2012 14:58, Timo Sirainen escribi?: >> On 21.10.2012, at 14.29, Nicol?s wrote: >> >>> El 21/10/2012 12:26, Timo Sirainen escribi?: >>>> On 21.10.2012, at 14.15, Nicol?s wrote: >>>> >>>>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >>>> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >>>> >>> No, on that machine I'm just using local filesystem, no NFS. >> Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. >> > > Well, at least it's good to know it's not a misconfiguration :-) If there's any additional info I can provide feel free to ask. Any way you can reproduce this somewhat easily? For example with http://imapwiki.org/ImapTest ? From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 21 23:49:19 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 21 Oct 2012 22:49:19 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> Message-ID: <20121021204919.GA9191@daniel.localdomain> Hi Dave, David Mehler wrote: > Thanks for all your help so far. I have per-user quotas now working, I > had to also alter my dict config file as well. I am having two > outstanding issues, actually one outstanding issue and one question. > > Here's the question, given that the userdb sql query returns a default > quota entry for rows of zero in quota and quota_messages is the > default quota section needed in 90-quota.conf? > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } You need quota_rule2 to give the user some additional space in the Trash folder if he/she wants to delete messages when over quota. The dict is also needed for quota lookup from the database. The only thing which might be omitted is the global quota_rule since it is returned by the userdb/passwd in any case, but I'm not sure what happens if you only configure a "quota_rule2" without configuring a "quota_rule". > My outstanding issue is whenever I as the root mysql user update a > user's quota the other user also gets an update, I noticed with one > the messages column on the other user went from 0 to 2, another time > the quota value went up from 0 to 3500 it seems random. You should not be accounting the actual mailbox usage in the same virtual_users table as the quota is read from. Use *different* column or table name in your dict file where dovecot may write the current storage/message count. > dovecot-dict-sql.conf.ext > > map { > pattern = priv/quota/storage > table = virtual_users > username_field = user > value_field = quota value_field should be current_quota_storage (writable column) > } > map { > pattern = priv/quota/messages > table = virtual_users > username_field = user > value_field = quota_messages value_field should be current_quota_messages (writable column) > I'd like to know why these columns are updating. Dovecot stores the current storage and mailcount in there. These columns should be different from the columns defining the maximum limit. Regards Daniel From daniel.parthey at informatik.tu-chemnitz.de Mon Oct 22 01:22:07 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 22 Oct 2012 00:22:07 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> <20121021204919.GA9191@daniel.localdomain> Message-ID: <20121021222207.GA10903@daniel.localdomain> David Mehler wrote: > Thanks, so if I understand what your saying the reason I'm getting the > column update issues is Dovecot is reading from and writing to the > quota and quota_messages columns in my virtual_users table? > > My database user I believe only has select permissions on that table. > > So, I either need another table and to adjust my dovecot-dict.sql file > for that table, that's where Dovecot will write to, or two more > columns in the virtual_users table? > > Which way do you recommend? I would recommend to create a new table for dovecot_usage where dovecot is granted write permission. Regards Daniel From dmalolepszy at optusnet.com.au Mon Oct 22 02:08:01 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Mon, 22 Oct 2012 10:08:01 +1100 Subject: [Dovecot] Dovecot LDA message save logging Message-ID: <50848051.6050308@optusnet.com.au> Hi, Is there any option in Dovecot that enables logging the full path of where a message is saved in the backend? Dominic From dave.mehler at gmail.com Mon Oct 22 02:14:56 2012 From: dave.mehler at gmail.com (David Mehler) Date: Sun, 21 Oct 2012 19:14:56 -0400 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: <20121021222207.GA10903@daniel.localdomain> References: <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> <20121021204919.GA9191@daniel.localdomain> <20121021222207.GA10903@daniel.localdomain> Message-ID: Hello, Thanks. I've created a quota table as described in dovecot-dict sql configuration file and granted the mail user select, insert, update, and delete rights to that table, while the virtual_users table select rights only. I configured for the new table. Here's my config, have I got it? Are the columns now going to stay where I put them? mysql> show create table virtual_users; +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | virtual_users | CREATE TABLE `virtual_users` ( `id` int(11) NOT NULL AUTO_INCREMENT, `domain_id` int(11) NOT NULL, `user` varchar(40) NOT NULL, `password` varchar(128) NOT NULL, `quota` bigint(20) NOT NULL DEFAULT '0', `quota_messages` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`id`), UNIQUE KEY `UNIQUE_EMAIL` (`domain_id`,`user`), CONSTRAINT `virtual_users_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `virtual_domains` (`id`) ON DELETE CASCADE ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 | +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> show create table quota; +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | quota | CREATE TABLE `quota` ( `username` varchar(100) NOT NULL, `bytes` bigint(20) NOT NULL DEFAULT '0', `messages` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`username`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) dovecot dictionary configuration map { pattern = priv/quota/storage table = quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = username value_field = messages } Thanks. Dave. On 10/21/12, Daniel Parthey wrote: > David Mehler wrote: >> Thanks, so if I understand what your saying the reason I'm getting the >> column update issues is Dovecot is reading from and writing to the >> quota and quota_messages columns in my virtual_users table? >> >> My database user I believe only has select permissions on that table. >> >> So, I either need another table and to adjust my dovecot-dict.sql file >> for that table, that's where Dovecot will write to, or two more >> columns in the virtual_users table? >> >> Which way do you recommend? > > I would recommend to create a new table for dovecot_usage > where dovecot is granted write permission. > > Regards > Daniel > From dovecot at knutejohnson.com Mon Oct 22 04:14:56 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 21 Oct 2012 18:14:56 -0700 Subject: [Dovecot] Anybody recognize these log lines? Message-ID: <50849E10.1080404@knutejohnson.com> WARN: Duplicate profile 'Dovecot POP3', using last found WARN: Duplicate profile 'Dovecot Secure POP3', using last found WARN: Duplicate profile 'Dovecot IMAP', using last found WARN: Duplicate profile 'Dovecot Secure IMAP', using last found Anybody know if these are dovecot generated? Thanks, -- Knute Johnson From mcguire at neurotica.com Mon Oct 22 04:17:07 2012 From: mcguire at neurotica.com (Dave McGuire) Date: Sun, 21 Oct 2012 21:17:07 -0400 Subject: [Dovecot] Anybody recognize these log lines? In-Reply-To: <50849E10.1080404@knutejohnson.com> References: <50849E10.1080404@knutejohnson.com> Message-ID: <50849E93.8040706@neurotica.com> On 10/21/2012 09:14 PM, Knute Johnson wrote: > WARN: Duplicate profile 'Dovecot POP3', using last found > WARN: Duplicate profile 'Dovecot Secure POP3', using last found > WARN: Duplicate profile 'Dovecot IMAP', using last found > WARN: Duplicate profile 'Dovecot Secure IMAP', using last found > > Anybody know if these are dovecot generated? Looks like output from the "ufw" firewall package. -Dave -- Dave McGuire, AK4HZ New Kensington, PA From alessio at skye.it Mon Oct 22 08:51:54 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 22 Oct 2012 07:51:54 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> Message-ID: <5084DEFA.6060404@skye.it> Il 13/10/2012 10:42, Timo Sirainen ha scritto: > On 13.10.2012, at 11.16, Alessio Cecchi wrote: > >> I'm running dovecot 2.1.10 on Debian 6. >> >> When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" >> >> [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] > The most helpful way to get this fixed is to get a gdb backtrace: http://dovecot.org/bugreport.html > Hi, after run "dovecot quota recalc -A" no error from "doveadm expunge -A mailbox Trash savedbefore 30d" is appeared and so have not been able to generate the "dump". -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From raabe at froglogic.com Mon Oct 22 10:21:51 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Mon, 22 Oct 2012 09:21:51 +0200 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read In-Reply-To: References: <507EA81C.5060806@froglogic.com> Message-ID: <5084F40F.7070601@froglogic.com> Am 10/18/2012 5:31 AM, schrieb Timo Sirainen: > Use: > > prefix=Lists/anotherlist/ > location = maildir:/home/vmail/lists/sharedseen/Maildir > > Then deliver the mails to /home/vmail/lists/sharedseen/Maildir root directly. Of course this means that you need to create a namespace for each such list. > > Alternative would be to use prefix=Lists/sharedseen/ and create lists under it. Thanks, the second version is basically what I did! I added a new namespace namespace public { separator = / prefix = Lists/Archive/ location = maildir:/home/vmail/lists/archive/Maildir subscriptions = no } ...and then had my Sieve script fileinto that. Works fine! Thanks for your help! -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From amateo at um.es Mon Oct 22 13:58:10 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 22 Oct 2012 12:58:10 +0200 Subject: [Dovecot] Auth caching and password changes In-Reply-To: <508149CC.9070004@um.es> References: <508149CC.9070004@um.es> Message-ID: <508526C2.8030403@um.es> El 19/10/12 14:38, Angel L. Mateo escribi?: > Hello, > > In my system I have configured auth caching. The problem I have is > that whenever a user changes his password, he/she can't login to dovecot > after a while and the scenarios described at > http://wiki2.dovecot.org/Authentication/Caching are not applied. > > I have tried also with "doveadm auth cache flush ", but it > didn't work. He also could to login again if he waits for a time or if I > run "doveadm auth cache flush" in the server, flushing all auth > information from cache. > > I have attached the log I had when I changed my password (and > suffered the problem). I have attached my doveconf -n too. > I think I have found part of the problem. My problem is that my authentication chain is first try by ldap (for normal clients authentication), and if it failed, then try with pam_cas (for webmail accesses with SSO). My change password application forms part of webmail, which also uses an imapproxy, so when I change the password, automatically seems to enter in the "Early change scenario" (I still haven't found the concrete reason for this). But I have tried to manually change the password in my ldap servers, and it works fine. So my point is that something related with this authentication chain provokes this scenario. My question now is there any way to configure authentication so a mechanism is only use when connections coming from a set of IPs? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From tss at iki.fi Mon Oct 22 15:39:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 22 Oct 2012 15:39:34 +0300 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <20120926180633.GE80443@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: On 26.9.2012, at 21.06, Kelsey Cummings wrote: > 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) > 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) > 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) Looks like I had broken this in v2.1.8. http://hg.dovecot.org/dovecot-2.1/rev/e4c337f38ed6 fixes this. I also added a bunch of other things to give better error messages and to try to fix any unexpected problems. From berni at birkenwald.de Mon Oct 22 17:23:18 2012 From: berni at birkenwald.de (Bernhard Schmidt) Date: Mon, 22 Oct 2012 14:23:18 +0000 (UTC) Subject: [Dovecot] auth timeout state=2, bad? Message-ID: Hello, we run Dovecot 2.1.7 as SASL backend for our Postfix SMTP-AUTH farm and see this error message occasionally Oct 22 16:15:32 lxmhs52 dovecot: auth: PLAIN(?,xx.xx.xx.xx): Request 0.21 timeouted after 150 secs, state=2 Since it is mostly the same IP repeating I'm assuming it's a client issue. Is that correct? What could cause this? Can we safely ignore it? Thanks, Bernhard From busseniu at in.tum.de Mon Oct 22 17:33:33 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 22 Oct 2012 16:33:33 +0200 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 Message-ID: <5085593D.3080403@in.tum.de> Hi, Thunderbird uses the following IMAP command to list shared folders: . list "" INBOX.shared.%.% Dovecot 2.1.10 does not list any folders in response to this command. Dovecot 2.0.21 does list them: * LIST (\HasNoChildren) "." "INBOX.shared.user1.folder" . OK List completed. Both versions list the folders if "*" is used instead of "INBOX.shared.%.%". Because of this issue, shared folders are not shown in Thunderbird. The ACL of the folder in question is ACL "keilrwtscd" for user1 (who is trying to access them using Thunderbird). config: # 2.1.10: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mail mail_plugins = acl mail_uid = vmail namespace { inbox = no list = children location = mdbox:%%h/mail prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } namespace default { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/mail/shared-mailboxes } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Mon Oct 22 18:59:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 22 Oct 2012 18:59:52 +0300 Subject: [Dovecot] auth timeout state=2, bad? In-Reply-To: References: Message-ID: <9790CECF-1FCE-4125-929B-CE0A53483495@iki.fi> On 22.10.2012, at 17.23, Bernhard Schmidt wrote: > we run Dovecot 2.1.7 as SASL backend for our Postfix SMTP-AUTH farm and > see this error message occasionally > > Oct 22 16:15:32 lxmhs52 dovecot: auth: PLAIN(?,xx.xx.xx.xx): Request > 0.21 timeouted after 150 secs, state=2 state 2 = "waiting for auth data from client", so what it means is that the client sent: AUTH PLAIN and then just didn't do anything. > Since it is mostly the same IP repeating I'm assuming it's a client > issue. Is that correct? What could cause this? Can we safely ignore it? You can safely ignore it. It should have been logged with "info" level and only with auth_verbose=yes. Although I guess the message could be a bit nicer. This is better I think: http://hg.dovecot.org/dovecot-2.1/rev/49bb6cc43d03 From kgc at corp.sonic.net Tue Oct 23 01:29:21 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 22 Oct 2012 15:29:21 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: <20121022222921.GE3370@corp.sonic.net> On Mon, Oct 22, 2012 at 03:39:34PM +0300, Timo Sirainen wrote: > On 26.9.2012, at 21.06, Kelsey Cummings wrote: > > > 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) > > 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) > > 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) > > Looks like I had broken this in v2.1.8. http://hg.dovecot.org/dovecot-2.1/rev/e4c337f38ed6 fixes this. I also added a bunch of other things to give better error messages and to try to fix any unexpected problems. Thanks Timo! -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From dovecot at knutejohnson.com Tue Oct 23 03:29:54 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Mon, 22 Oct 2012 17:29:54 -0700 Subject: [Dovecot] Anybody recognize these log lines? In-Reply-To: <50849E93.8040706@neurotica.com> References: <50849E10.1080404@knutejohnson.com> <50849E93.8040706@neurotica.com> Message-ID: <5085E502.3080802@knutejohnson.com> On 10/21/2012 6:17 PM, Dave McGuire wrote: > On 10/21/2012 09:14 PM, Knute Johnson wrote: >> WARN: Duplicate profile 'Dovecot POP3', using last found >> WARN: Duplicate profile 'Dovecot Secure POP3', using last found >> WARN: Duplicate profile 'Dovecot IMAP', using last found >> WARN: Duplicate profile 'Dovecot Secure IMAP', using last found >> >> Anybody know if these are dovecot generated? > > Looks like output from the "ufw" firewall package. > > -Dave > Thanks, I'll look at that. -- Knute Johnson From list at airstreamcomm.net Tue Oct 23 08:49:47 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 23 Oct 2012 00:49:47 -0500 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <20121021194320.GA7977@daniel.localdomain> References: <5083D963.3000700@wk-serv.de> <20121021194320.GA7977@daniel.localdomain> Message-ID: <50862FFB.2060108@airstreamcomm.net> On 10/21/12 2:43 PM, Daniel Parthey wrote: > Patrick Westenberg wrote: >> Is there no built in failover mechanism for the director service to >> handle a backend failure? > No, the director's job is to keep a hash table and direct > the connection for each user to its associated backend. > Currently, there is no built-in backend monitoring. > > In order to handle maintenance of backends, you will need the > poolmon daemon, which enables/disables backends in the director > depending on their availability: > > https://github.com/brandond/poolmon > > Regards > Daniel Considering the intention of the director was to alleviate locking issues in a shared storage environment are there any current solutions to improving the scalability/availability of Dovecot by implementing an alternative message storage systems such as nosql or maybe object storage that could abstract away the complexity of replicating data? We would love to finally have the ability to set our mail cluster on top of a storage subsystem that can span multiple geographic regions and do away with the NFS backend. From alessio at skye.it Tue Oct 23 09:00:52 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 23 Oct 2012 08:00:52 +0200 Subject: [Dovecot] Dovecot LDA message save logging In-Reply-To: <50848051.6050308@optusnet.com.au> References: <50848051.6050308@optusnet.com.au> Message-ID: <50863294.2010404@skye.it> Il 22/10/2012 01:08, Dominic Malolepszy ha scritto: > Hi, > > Is there any option in Dovecot that enables logging the full path of > where a message is saved in the backend? > > Dominic > With mail_debug=yes you can see it. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From eliezer at ngtech.co.il Tue Oct 23 18:19:04 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Tue, 23 Oct 2012 17:19:04 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 Message-ID: <5086B568.1010905@ngtech.co.il> Since I have lots of filtering rules in thunderbird I was thinking of using sieve instead. I want to filter incoming mail into subdirectories. like "from" store at folder "old". the script is: require ["fileinto", "envelope"]; if envelope :is "from" "eliezer at test.dom" { fileinto "old"; } else { # The rest goes into INBOX # default is "implicit keep", we do it explicitly here keep; } the result is that the mail is stored in two folders instead of just one, INBOX and old. the logs shows: Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing script from /home/vmail/domain/eliezer/home/.dovecot.svbin Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' from unknown reason(or I didnt understood how sieve works?) plugin section from dovecot -n plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.debug +imapflags +relational +comparator-i;ascii-numeric } Thanks, Eliezer From stsiol at yahoo.co.uk Tue Oct 23 19:03:38 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 23 Oct 2012 17:03:38 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb [Resolved] Message-ID: <1351008218.44057.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello all, I would like to thank you all for your kind replies and feedback in regards to migrating from a smaller hdd to a bigger one (namely from 72gb to 146gb). I finally found a painless way of doing this. Since I believe that this is still an off-topic post, if anyone is interested in the solution i've adopted for this, let me know by replying to me privately. If, however, you don't mind me posting here, let me know. Many many thanks go to?Alexander Hoogerhuis, the "mad Norwegian" :-) who helped me on this too much to describe here. Alex, you are a true sport. Thank you again people. All the Best, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From dg at dguhl.org Tue Oct 23 19:21:27 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 23 Oct 2012 18:21:27 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <20121023162125.GA7983@PC211.ikt.de> On Tue, Oct 23, 2012 at 05:19:04PM +0200, Eliezer Croitoru wrote: [..] > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; stop; # seems to be needed with explicit keep > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } Comparing various sieve scripts I concluded (but I didn't actually test it) that you need a stop; in your if or elsif clauses in case you have an else clause with an explicit keep; -- IMHO a behaviour violating RFC 5228. RFC 5228, Section 3.1 ... If the test of the "if" is false, it evaluates the test of the first "elsif" (if any). If the test of "elsif" is true, it runs the elsif's block. An elsif may be followed by an elsif, in which case, the interpreter repeats this process until it runs out of elsifs. When the interpreter runs out of elsifs, there may be an "else" case. If there is, and none of the if or elsif tests were true, the interpreter runs the else's block. ... Stefan, can you enlighten us? Dennis [..] From john.roman at dreamhost.com Tue Oct 23 20:04:12 2012 From: john.roman at dreamhost.com (john roman) Date: Tue, 23 Oct 2012 10:04:12 -0700 Subject: [Dovecot] index files created improperly in dovecot 1.2.16 Message-ID: Greetings, It seems to be a problem that ive seen occasionally on the web with few results as to a solution, but im experiencing it as well. Namely, dovecot creates index logs for users with a 600 permission, when it should create with a 700 permission. My indexes are stored in /var/indexes with the directory at 777 permission, the users are identified in MySQL. The error is as follows: Oct 23 09:03:13 mailer01 dovecot: POP3 (johnr at testing.com): stat(/var/indexes/j/johnr at testing.com/.INBOX) failed: Permission denied (euid=10509305(x10509305) egid=81607(pg199275) missing +x perm: /var/indexes/j) the permissions of /var/indexes/j are 700, owned by the euid and egid effectively. the only file that does not have this permission is in /var/indexes/j/johnr at testing.com/.INBOX/dovecot.index.log, at 600. From trashcan at odo.in-berlin.de Tue Oct 23 22:06:51 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 23 Oct 2012 21:06:51 +0200 Subject: [Dovecot] [2.2-UNSTABLE] compilation error: 'POSIX_FADV_WILLNEED' undeclared Message-ID: <198596C8-3989-4041-B96A-5D5AFEE8E3D0@odo.in-berlin.de> Hi -- I am trying to compile 2.2 (acd76b5272e9) at FreeBSD 9.0: | libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I/usr/local/include -std=gnu99 -O2 -Wall -W -Wmissing-prototypes \ -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 \ -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 \ -I/usr/local/include -MT fs-posix.lo -MD -MP -MF .deps/fs-posix.Tpo \ -c fs-posix.c -fPIC -DPIC -o .libs/fs-posix.o | fs-posix.c: In function 'fs_posix_prefetch': | fs-posix.c:298: warning: implicit declaration of function 'posix_fadvise' | fs-posix.c:298: error: 'POSIX_FADV_WILLNEED' undeclared (first use in this function) | fs-posix.c:298: error: (Each undeclared identifier is reported only once | fs-posix.c:298: error: for each function it appears in.) | gmake[3]: *** [fs-posix.lo] Error 1 | gmake[3]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src/lib-fs' | gmake[2]: *** [all-recursive] Error 1 | gmake[2]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src' | gmake[1]: *** [all-recursive] Error 1 | gmake[1]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2' | gmake: *** [all] Error 2 From configure logfile: | checking for posix_fadvise... no After a modification (stolen from src/lib-storage/index/index-mail.c) ... | --- dovecot-2.2-modified/src/lib-fs/fs-posix.c 2012-10-23 20:27:31.348919455 +0200 | +++ dovecot-2.2/src/lib-fs/fs-posix.c 2012-10-23 20:26:39.435300269 +0200 | @@ -295,10 +295,12 @@ | return TRUE; | } | | +#if defined(HAVE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED) | if (posix_fadvise(file->fd, 0, length, POSIX_FADV_WILLNEED) < 0) { | i_error("posix_fadvise(%s) failed: %m", _file->path); | return TRUE; | } | +#endif | return FALSE; | } ... the compilations runs to completion, and dovecot-2.2 UNSTABLE is running. But: I do not have the knowledge to judge if that "fix" will be the right one. Just to let you know and with regards, Michael From stephan at rename-it.nl Tue Oct 23 22:40:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 23 Oct 2012 21:40:43 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <5086F2BB.7010704@rename-it.nl> On 10/23/2012 5:19 PM, Eliezer Croitoru wrote: > Since I have lots of filtering rules in thunderbird I was thinking of > using sieve instead. > I want to filter incoming mail into subdirectories. > like "from" store at folder "old". > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } > > the result is that the mail is stored in two folders instead of just > one, INBOX and old. > the logs shows: > Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing > script from /home/vmail/domain/eliezer/home/.dovecot.svbin > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' > > from unknown reason(or I didnt understood how sieve works?) This behavior would definitely be a bug, but I cannot reproduce it even with Dovecot 2.0.17 and Pigeonhole v0.2.5: stephan at host:~/src/devel/dovecot-2.0-pigeonhole$ src/sieve-tools/sieve-test -t - -Tlevel=matching -e ~/frop.sieve ~/frop.eml ## Started executing script 'frop' 2: envelope test 2: starting `:is' match with `i;ascii-casemap' comparator: 2: getting `from' part from message envelope 2: extracting `all' part from address `eliezer at test.dom' 2: matching value `eliezer at test.dom' 2: with key `eliezer at test.dom' => 1 2: finishing match with result: matched 2: jump if result is false 2: not jumping 3: fileinto action 3: store message in mailbox `old' 3: jumping to line 7 ## Finished executing script 'frop' info: msgid=unspecified: stored mail into mailbox 'old'. sieve-test(stephan): Info: final result: success Could you test this at your end? Be careful, the above command adds a message to the user's mailbox, so read the sieve-test man page first before you try anything. Also, I executed this from my development tree, because I haven't got an operational Dovecot v2.0 installation. I don't remember any bug that was solved since that version that could explain what you're seeing. What is your Pigeonhole version? I've tried with v0.2.5 at this end. Also, could you provide your full configuration as output from `dovecot -n` ? Regards, Stephan. From benedetto.vassallo at unipa.it Tue Oct 23 23:18:06 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Tue, 23 Oct 2012 22:18:06 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> <20121016091153.15601eysq5n040qh@webmail.unipa.it> <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> Message-ID: <20121023221806.43795tha204qxgfy@webmail.unipa.it> Def. Quota Timo Sirainen : > On 16.10.2012, at 10.11, Benedetto Vassallo wrote: > >>> What are the permissions of the MailDir directory for user1/user2? >>> >>> ls -ld /home/user1/MailDir >>> ls -ld /home/user2/MailDir >>> >>> >> >> Thank you for your reply. >> They are different groups: >> >> drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ >> drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ >> drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ > > Not very secure permissions.. Maybe would be easiest to just have > one vmail user for everyone? > >> I tryed to issue: >> chgrp -R mail /home/user1/MailDir >> chgrp -R mail /home/user2/MailDir >> chgrp -R mail /home/user3/MailDir > > Dovecot doesn't do hard linking when it looks like the permissions > aren't compatible. The current code checks that if the owner UIDs > are different, then the group needs to be writable. On my production server with dovecot 2.0.13 I have same permissions and it works. I changed my permissions in any mode, changed the owner, the group but it still don't work. Any suggestion? Thank you -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From marc at perkel.com Tue Oct 23 23:51:39 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 23 Oct 2012 13:51:39 -0700 Subject: [Dovecot] Can Dovecot authenticate against an external email server? Message-ID: <5087035B.7060208@perkel.com> Just wondering if anyone has done this. I have a spam filtering service where I am now storing spam for users I filter for. It's a filter and forward service so I don't control the recipient's email server. What I would like to do somehow is have the user enter their email address and password and then look up their POP/IMAP server from a database and try to authenticate from it. If sucessful then the user will be able to access their stored spam using Dovecot and Squirrelmail. Any suggestions? Thanks in advance From troy at troyvit.com Tue Oct 23 23:52:45 2012 From: troy at troyvit.com (Troy Vitullo) Date: Tue, 23 Oct 2012 14:52:45 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver Message-ID: <20121023145245.124dd362@hrafn> Hi, My server uses a system comprised of postfix, dovecot and dspam to filter and deliver mail. Postfix used the following flags in calling spamc and dovecot: flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} after an upgrade from Debian lenny to squeeze we were able to get everything working except spam filtering. Spamassassin is able to judge whether the mail coming in is spam but everything stops there. In mail.err I see: pamc[3608]: exec failed: Permission denied spamc shows the same thing in syslog: exec failed: Permission denied postfix delays the email: postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system resource problem) Here are the permissions for deliver: -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver Here are the relevant groups: s1:~# grep dovecot /etc/group secmail:x:119:postfix,spamd,dovecot dovecot:x:111: here's the dovecot user: s1:~# grep dovecot /etc/passwd dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false here's dovecot -n: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 base_dir: /var/run/dovecot/ protocols: imap imaps pop3s pop3 ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert ssl_key_file: /etc/ssl/private/s1.troyvit.com.key ssl_cipher_list: ALL:!LOW disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_location: maildir:%h/Maildir/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s namespace: type: private separator: / inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmaster at sphere.local auth_socket_path: /var/run/dovecot/auth-master mail_plugin_dir: /usr/lib/dovecot/modules/lda/ mail_plugins: sieve auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: pam args: dovecot passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 438 user: dovecot plugin: sieve_global_path: /etc/dovecot/default.sieve sieve: /srv/%d/mail/%n/%n.sieve Many thanks in advance for any advice you can give. Troy From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 24 01:12:33 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 24 Oct 2012 00:12:33 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <50862FFB.2060108@airstreamcomm.net> References: <5083D963.3000700@wk-serv.de> <20121021194320.GA7977@daniel.localdomain> <50862FFB.2060108@airstreamcomm.net> Message-ID: <20121023221233.GA22084@daniel.localdomain> list at airstreamcomm.net wrote: > Considering the intention of the director was to alleviate locking > issues in a shared storage environment are there any current > solutions to improving the scalability/availability of Dovecot by > implementing an alternative message storage systems such as nosql or > maybe object storage that could abstract away the complexity of > replicating data? We would love to finally have the ability to set > our mail cluster on top of a storage subsystem that can span > multiple geographic regions and do away with the NFS backend. Key/value object store is planned for Dovecot v2.2 and has been discussed in this thread: http://dovecot.org/list/dovecot/2012-September/068257.html Regards Daniel -- https://plus.google.com/103021802792276734820 From Bill at KnoxvilleChristian.org Wed Oct 24 04:06:17 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:06:17 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121023145245.124dd362@hrafn> References: <20121023145245.124dd362@hrafn> Message-ID: <50873F09.4070604@KnoxvilleChristian.org> On 10/23/2012 4:52 PM, Troy Vitullo wrote: > Hi, > > My server uses a system comprised of postfix, dovecot and dspam to filter and deliver mail. > > Postfix used the following flags in calling spamc and dovecot: > > flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} > > after an upgrade from Debian lenny to squeeze we were able to get everything working except spam filtering. Spamassassin is able to judge whether the mail coming in is spam but everything stops there. > > In mail.err I see: > > pamc[3608]: exec failed: Permission denied > > spamc shows the same thing in syslog: > > exec failed: Permission denied > > postfix delays the email: > > postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system resource problem) > > Here are the permissions for deliver: > > -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver > > Here are the relevant groups: > > s1:~# grep dovecot /etc/group > secmail:x:119:postfix,spamd,dovecot > dovecot:x:111: > > here's the dovecot user: > s1:~# grep dovecot /etc/passwd > dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false > > here's dovecot -n: > > # 1.2.15: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 > base_dir: /var/run/dovecot/ > protocols: imap imaps pop3s pop3 > ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert > ssl_key_file: /etc/ssl/private/s1.troyvit.com.key > ssl_cipher_list: ALL:!LOW > disable_plaintext_auth: no > verbose_ssl: yes > login_dir: /var/run/dovecot/login > login_executable(default): /usr/lib/dovecot/imap-login > login_executable(imap): /usr/lib/dovecot/imap-login > login_executable(pop3): /usr/lib/dovecot/pop3-login > mail_location: maildir:%h/Maildir/ > mbox_write_locks: fcntl dotlock > mail_executable(default): /usr/lib/dovecot/imap > mail_executable(imap): /usr/lib/dovecot/imap > mail_executable(pop3): /usr/lib/dovecot/pop3 > mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > pop3_enable_last(default): no > pop3_enable_last(imap): no > pop3_enable_last(pop3): yes > pop3_client_workarounds(default): > pop3_client_workarounds(imap): > pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh > pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s > namespace: > type: private > separator: / > inbox: yes > list: yes > subscriptions: yes > lda: > postmaster_address: postmaster at sphere.local > auth_socket_path: /var/run/dovecot/auth-master > mail_plugin_dir: /usr/lib/dovecot/modules/lda/ > mail_plugins: sieve > auth default: > mechanisms: plain login > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: pam > args: dovecot > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: passwd > userdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: postfix > master: > path: /var/run/dovecot/auth-master > mode: 438 > user: dovecot > plugin: > sieve_global_path: /etc/dovecot/default.sieve > sieve: /srv/%d/mail/%n/%n.sieve > > Many thanks in advance for any advice you can give. > > Troy What is your mailbox_command in main.cf? I just use: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" I don't need anything in master.cf. But you should be using -u ${user} for spamc. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 04:15:34 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:15:34 -0400 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <50874136.7000903@KnoxvilleChristian.org> On 10/23/2012 11:19 AM, Eliezer Croitoru wrote: > Since I have lots of filtering rules in thunderbird I was thinking of > using sieve instead. > I want to filter incoming mail into subdirectories. > like "from" store at folder "old". > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } > > the result is that the mail is stored in two folders instead of just > one, INBOX and old. > the logs shows: > Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing > script from /home/vmail/domain/eliezer/home/.dovecot.svbin > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' > > from unknown reason(or I didnt understood how sieve works?) > > plugin section from dovecot -n > > plugin { > ... > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > sieve_extensions = +vnd.dovecot.debug +imapflags +relational > +comparator-i;ascii-numeric > } > > Thanks, > Eliezer Why to people bother with all these complex if...elsif...else structures. I just use stop a lot. This is included from my Main.seive: # # 2012-07-05 # require "include"; require "fileinto"; require "copy"; #require "body"; #require "imap4flags"; # put this in main #if header :contains "list-id" "dovecot.dovecot.org" { include "Dovecot"; } if address :is :localpart "to" "dovecot" { fileinto :copy "SystemFolders.Ham"; fileinto "Lists.Dovecot"; stop; } fileinto "Lists"; stop; As you can see, I also train Spamassassin with the mail from the list. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 04:32:59 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:32:59 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50873F09.4070604@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> Message-ID: <5087454B.5030704@KnoxvilleChristian.org> On 10/23/2012 9:06 PM, Bill Shirley wrote: > > On 10/23/2012 4:52 PM, Troy Vitullo wrote: >> Hi, >> >> My server uses a system comprised of postfix, dovecot and dspam to >> filter and deliver mail. >> >> Postfix used the following flags in calling spamc and dovecot: >> >> flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} >> -e /usr/lib/dovecot/deliver -d ${recipient} >> >> after an upgrade from Debian lenny to squeeze we were able to get >> everything working except spam filtering. Spamassassin is able to >> judge whether the mail coming in is spam but everything stops there. >> >> In mail.err I see: >> >> pamc[3608]: exec failed: Permission denied >> >> spamc shows the same thing in syslog: >> >> exec failed: Permission denied >> >> postfix delays the email: >> >> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system >> resource problem) >> >> Here are the permissions for deliver: >> >> -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver >> >> Here are the relevant groups: >> >> s1:~# grep dovecot /etc/group >> secmail:x:119:postfix,spamd,dovecot >> dovecot:x:111: >> >> here's the dovecot user: >> s1:~# grep dovecot /etc/passwd >> dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false >> >> here's dovecot -n: >> >> # 1.2.15: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 >> base_dir: /var/run/dovecot/ >> protocols: imap imaps pop3s pop3 >> ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert >> ssl_key_file: /etc/ssl/private/s1.troyvit.com.key >> ssl_cipher_list: ALL:!LOW >> disable_plaintext_auth: no >> verbose_ssl: yes >> login_dir: /var/run/dovecot/login >> login_executable(default): /usr/lib/dovecot/imap-login >> login_executable(imap): /usr/lib/dovecot/imap-login >> login_executable(pop3): /usr/lib/dovecot/pop3-login >> mail_location: maildir:%h/Maildir/ >> mbox_write_locks: fcntl dotlock >> mail_executable(default): /usr/lib/dovecot/imap >> mail_executable(imap): /usr/lib/dovecot/imap >> mail_executable(pop3): /usr/lib/dovecot/pop3 >> mail_plugin_dir(default): /usr/lib/dovecot/modules/imap >> mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap >> mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 >> pop3_enable_last(default): no >> pop3_enable_last(imap): no >> pop3_enable_last(pop3): yes >> pop3_client_workarounds(default): >> pop3_client_workarounds(imap): >> pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh >> pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s >> pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s >> pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s >> namespace: >> type: private >> separator: / >> inbox: yes >> list: yes >> subscriptions: yes >> lda: >> postmaster_address: postmaster at sphere.local >> auth_socket_path: /var/run/dovecot/auth-master >> mail_plugin_dir: /usr/lib/dovecot/modules/lda/ >> mail_plugins: sieve >> auth default: >> mechanisms: plain login >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: pam >> args: dovecot >> passdb: >> driver: sql >> args: /etc/dovecot/dovecot-sql.conf >> userdb: >> driver: passwd >> userdb: >> driver: sql >> args: /etc/dovecot/dovecot-sql.conf >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: postfix >> master: >> path: /var/run/dovecot/auth-master >> mode: 438 >> user: dovecot >> plugin: >> sieve_global_path: /etc/dovecot/default.sieve >> sieve: /srv/%d/mail/%n/%n.sieve >> >> Many thanks in advance for any advice you can give. >> >> Troy > > What is your mailbox_command in main.cf? I just use: > mailbox_command = /usr/bin/spamc -u "$USER" -e > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" > > I don't need anything in master.cf. But you should be using -u > ${user} for spamc. > > Bill > Forgot to ask, are you using Spamassassin's per-user configs? If you're not, that probably is your problem. It's probably trying to update bayes tokens and it doesn't have permission. I use per-user configs which are nice. One man's spam is another man's ham. Plus each user can have his/her own whitelist. I use these spamd args: -d -c -m10 --user-config You usually can find the args in /etc/sysconfig. Bill From rs at sys4.de Wed Oct 24 09:33:26 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 08:33:26 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5087454B.5030704@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> Message-ID: <50878BB6.2090309@sys4.de> Am 24.10.2012 03:32, schrieb Bill Shirley: > What is your mailbox_command in main.cf? I just use: > mailbox_command = /usr/bin/spamc -u "$USER" -e > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" > > I don't need anything in master.cf. But you should be using -u ${user} > for spamc. long time ago i tested this with dovecot lda postfix master.cf with a total virtual setup dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} but i strongly do not recommand this !!! use spamass-milter, amavis etc with dovecot lmtp as described on many sites Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From carsten.delellis at delellis.net Wed Oct 24 12:48:34 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Wed, 24 Oct 2012 11:48:34 +0200 Subject: [Dovecot] dovecot auth against AD on samba4 Message-ID: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> Hi group I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and I am thinking about installing it because of it's native outlook support capabilities. The ZEG appliance wouldn't be an option for me because I use a virtual server from a provider where I can't install my own vm or even an iso. When I go thru the documentation there is a part with installing OpenChange based on samba4. As far as I understood the OpenChange authentication is against the samba4 AD. Actually there is no support in syncing the AD against an OpenLdap Server and I would have to change the OpenLdap port because the AD is listening on port 389. To change the port wouldn't be a big deal, but what i was thinking about to run the dovecot auth also against the samba 4 AD. I searched around on the internet but didn't find a doc yet how to do that. Does anyone here could provide me with a link or a how-to ? Thanks very much in advance. Regards, Carsten Laun-De Lellis Hauptstrasse 13 D-67705 Trippstadt Phone: +49 6306 992140 Fax: +49 6306 992142 Mobile: +49 151 27530865 email: carsten.delellis at delellis.net From listen at mjh.name Wed Oct 24 14:28:11 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Wed, 24 Oct 2012 13:28:11 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox Message-ID: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Hello all, I have a problem with an incosistent mdbox: Oct 24 10:43:23 two dovecot: imap-login: Login: user=<...>, method=PLAIN, rip=..., lip=..., mpid=4977, TLS Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-24 10:43:23] bytes=115/53726 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: Inconsistency in map index (467,31960 != 467,36768) Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: rebuilding indexes Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Panic: file mdbox-storage-rebuild.c: line 773 (rebuild_update_refcounts): assertion failed: (map_uid < msgs[i]->map_uid) Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x380ca) [0x7f99cf35b0ca] -> /usr/lib/dovecot/libdovecot.so.0(+0x3810e) [0x7f99cf35b10e] -> /usr/li b/dovecot/libdovecot.so.0(i_fatal+0) [0x7f99cf334a67] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x10a5) [0x7f99cf5f42d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_s torage_rebuild+0x24) [0x7f99cf5f4414] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x53565) [0x7f99cf5f4565] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7f99cf5c8caa] -> /usr/lib/dovec ot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7f99cf5c8cf3] -> dovecot/imap(client_destroy+0x109) [0x7f99cfaa69e9] -> dovecot/imap(client_input+0xaa) [0x7f99cfaa6dba] -> /usr/lib/dovecot/libdovecot.so.0(i o_loop_call_io+0x48) [0x7f99cf366c98] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f99cf367d27] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f99cf366c28] -> /usr/lib/dovecot /libdovecot.so.0(master_service_run+0x13) [0x7f99cf354e33] -> dovecot/imap(main+0x304) [0x7f99cfa9e554] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f99cef8576d] -> dovecot/imap(+0x95e5) [0 x7f99cfa9e5e5] Oct 24 10:45:19 two dovecot: master: Error: service(imap): child 4977 killed with signal 6 (core dumps disabled) I use: Dovecot 2.0.19-0ubuntu1 Ubuntu 12.04, x86-64, Kernel 3.2.0-32-generic local XFS filesystem for the mdbox The problem appeared out of nowhere. Many messages been continously delivered to this mailbox on this installation since May 2012, and the mdbox was only accessed with deliver and imap/pop3 from dovecot. About four hours after the problem initially appeared, I did a hard reset of the system because it was unresponsive. The error message is exactly the same before and after the hard reset. The problem is triggered by both IMAP access and dovecot deliver access. The whole mdbox is 6.6 GiB large and I guess that it contains about 300k-600k messages. It's an archive of public mailing lists, so I could give access to the files. Can anybody say something about this? May the mdbox be repaired? Regards, Milan Holz?pfel -- Milan Holz?pfel -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doveconf.txt URL: From rs at sys4.de Wed Oct 24 14:43:19 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 13:43:19 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <5087D457.6040205@sys4.de> Am 24.10.2012 13:28, schrieb Milan Holz?pfel: > The whole mdbox is 6.6 GiB large and I guess that it contains about > 300k-600k messages. It's an archive of public mailing lists, so I could > give access to the files. > > Can anybody say something about this? May the mdbox be repaired? perhaps this helps http://wiki2.dovecot.org/Tools/Doveadm/ForceResync however upgrading to dovecot latest might be a good idea Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stan at hardwarefreak.com Wed Oct 24 17:01:24 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 09:01:24 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <5087F4B4.2060107@hardwarefreak.com> On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: > I have a problem with an incosistent mdbox: ... > four hours after the problem initially appeared, I did a hard reset of > the system because it was unresponsive. ... > Can anybody say something about this? May the mdbox be repaired? If the box is truly unresponsive, i.e. hard locked, then the corrupted indexes are only a symptom of the underlying problem, which is unrelated to Dovecot, UNLESS, the lack of responsiveness was due to massive disk access, which will occur when rebuilding indexes on a 6.6GB mailbox. You need to know the difference so we have accurate information to troubleshoot with. If the there's a kernel or hardware problem, you should see related errors in dmesg. Please share those. Neither Timo nor anyone here can fix your index problem if the cause lie elsewhere. You must fix the root problem first. -- Stan From CMarcus at Media-Brokers.com Wed Oct 24 17:45:01 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 24 Oct 2012 10:45:01 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087F4B4.2060107@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> Message-ID: <5087FEED.7060007@Media-Brokers.com> On 2012-10-24 10:01 AM, Stan Hoeppner wrote: > If the box is truly unresponsive, i.e. hard locked, then the corrupted > indexes are only a symptom of the underlying problem, which is unrelated > to Dovecot, UNLESS, the lack of responsiveness was due to massive disk > access, which will occur when rebuilding indexes on a 6.6GB mailbox. > You need to know the difference so we have accurate information to > troubleshoot with. Hmmm... I wonder would it be possible for dovecot to automatically lower the 'niceness' for index rebuilds (on systems that support such) to avoid causing such distress? -- Best regards, Charles From weber at papaya-cms.com Wed Oct 24 17:46:39 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Wed, 24 Oct 2012 16:46:39 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. Message-ID: <5087FF4F.8050103@papaya-cms.com> Hi, I've got some trouble here.. i created some sieve rules, but the debug log says that there is a invalid mailbox name error: msgid=<*>: failed to store into mailbox '/home/shared/.automail.Bugtracker/': Invalid mailbox name. ~/.dovecot.sieve if address :is "to" "mantis-admin@<*>" { fileinto "/home/shared/.automail.Bugtracker/"; } here's my doveconf -n output # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-32-generic x86_64 Ubuntu 12.04.1 LTS base_dir = /var/run/dovecot/ disable_plaintext_auth = no hostname = * mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = yes location = maildir:/home/shared:CONTROL=~/.Maildir/control/Shared:INDEX=~/.Maildir/index/Shared prefix = shared/ separator = / subscriptions = yes type = public } namespace { inbox = yes location = prefix = separator = / type = private } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@* protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } ssl_ca = was automatically rejected:%n%r } protocol imap { mail_plugins = " zlib, acl, imap_zlib" } any idea? Mit freundlichen Gruessen / best regards papaya Software GmbH i.A. Alexander Weber -- papaya Software GmbH | Im MediaPark 5 | 50670 Koeln | Germany Tel./Ph.: +49-221-5743-8070 | Fax: +49-221-5743-8099 mailto:weber at papaya-cms.com | http://www.papaya-cms.com/ -- Geschaeftsfuehrer: Andreas Jacobi, Andr? Schnitzler, Daniel Sch?fer Sitz& Registergericht: Koeln | HRB 60030 | USt.-Id.-Nr.: DE 255642963 -- From sandro.tosi at dada.eu Wed Oct 24 17:48:44 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Wed, 24 Oct 2012 16:48:44 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50808A57.8040201@rename-it.nl> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> <507BBE00.9010007@dada.eu> <50808A57.8040201@rename-it.nl> Message-ID: <5087FFCC.5080504@dada.eu> Hello Stephan, sorry for this late reply. On 10/19/2012 01:01 AM, Stephan Bosch wrote: > On 10/15/2012 9:40 AM, Sandro Tosi wrote: >> Hi Stephan, >> thanks a lot for your reply. >> >> On 10/11/2012 10:35 PM, Stephan Bosch wrote: >>> On 10/10/2012 11:23 AM, Sandro Tosi wrote: >>>> Hello, >>>> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >>>> as this will be provided with an upcoming Debian package) with MySQL >>>> dict lookups with the mail setup we're designing. >>>> >>>> Our (main) goals are: >>>> >>>> 1. store the filters on the database >>> That is possible with some limitations. >> >> Are the ones below the only limitatios (ie one script per user) or are >> there any other worth knowing? > > You cannot currently use ManageSieve when the active script is located > in a dict database. > > And 'one script per user' is not an fully accurate description. It is > technically possible to access multiple different scripts from the dict > database. It is however not possible to use dict support combination > with multiscript support ( > http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Executing_Multiple_Scripts_Sequentially) > to execute multiple scripts in a sequence. Multiscript currently only > works for Sieve scripts that are located in the filesystem. > >> In our situation, what would you suggest? We're now thinking of >> keeping the scripts list on a separate table, and merge the "user >> selected ones" in a single script to write in the filters table. Is >> that what would you suggest? Is there a better solution? > > You can use the include extension > (https://tools.ietf.org/html/draft-ietf-sieve-include-05) to access > scripts in a dict database from a main active script to combine them. I > believe you could even dynamically construct that main script in SQL > using some string manipulation in the query, but that is a bit ugly. > > Could you send me an overview of your configuration, including your > database layout? Provided that I have some time in the next week, I > could investigate building a simple working configuration for the sake > of example. I will follow this up privately (you know, we can't disclose too much) but JFTR we decided to follow a half-and-half solution: - we keep on the backend database all the scripts the customer could activate in separate rows - from them, we merge into a single sieve script file all the filter the customer has decided to activate. This way we still record the script separately in the db, so once we'll be able to feed pigeonhole with multiple lines, it's already there, and then merging into a single file is the most straightforward and simple solution to make what we need to work. Thanks for the support, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From Bill at KnoxvilleChristian.org Wed Oct 24 18:47:07 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 11:47:07 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50878BB6.2090309@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> Message-ID: <50880D7B.4090407@KnoxvilleChristian.org> On 10/24/2012 2:33 AM, Robert Schetterer wrote: > Am 24.10.2012 03:32, schrieb Bill Shirley: >> What is your mailbox_command in main.cf? I just use: >> mailbox_command = /usr/bin/spamc -u "$USER" -e >> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >> >> I don't need anything in master.cf. But you should be using -u ${user} >> for spamc. > long time ago i tested this with dovecot lda postfix master.cf > with a total virtual setup > > dovecot unix - n n - - pipe > flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e > /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} > > but i strongly do not recommand this !!! > > use spamass-milter, amavis etc with dovecot lmtp > as described on many sites > > > Best Regards > MfG Robert Schetterer > Can you get per-user Spamassassin configs this way? Why user=vmail:vmail? Is this for virtual domains? I didn't think we were talking about them. Instead of strongly recommending against this, why not elaborate on the problems with using spamc in the mailbox_command? Bill From bob at computerisms.ca Wed Oct 24 19:04:39 2012 From: bob at computerisms.ca (Bob Miller) Date: Wed, 24 Oct 2012 09:04:39 -0700 Subject: [Dovecot] dovecot auth against AD on samba4 In-Reply-To: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> References: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> Message-ID: <1351094679.2143.474.camel@worklian> I don't have it in production yet because there are other things I am still trying to add to samba4, but my test server has dovecot authenticating against samba4. Without openchange or any other non-native mechanism. Dovecot supports authenticating against ldap, the settings are in your auth-ldap.conf file. Samba4/Active Directory is just another ldap implementation. between the config files and the wiki, I believe all the documentation you need is there... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Wed, 2012-10-24 at 11:48 +0200, Carsten Laun-De Lellis wrote: > Hi group > > I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, > dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and > I am thinking about installing it because of it's native outlook support > capabilities. > > The ZEG appliance wouldn't be an option for me because I use a virtual > server from a provider where I can't install my own vm or even an iso. > > When I go thru the documentation there is a part with installing > OpenChange based on samba4. As far as I understood the OpenChange > authentication is against the samba4 AD. Actually there is no support in > syncing the AD against an OpenLdap Server and I would have to change the > OpenLdap port because the AD is listening on port 389. To change the port > wouldn't be a big deal, but what i was thinking about to run the dovecot > auth also against the samba 4 AD. > > I searched around on the internet but didn't find a doc yet how to do that. > > Does anyone here could provide me with a link or a how-to ? > > Thanks very much in advance. > > Regards, > > Carsten Laun-De Lellis > > Hauptstrasse 13 > D-67705 Trippstadt > > Phone: +49 6306 992140 > Fax: +49 6306 992142 > Mobile: +49 151 27530865 > email: carsten.delellis at delellis.net > > > From rs at sys4.de Wed Oct 24 19:09:12 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 18:09:12 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50880D7B.4090407@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> Message-ID: <508812A8.8000603@sys4.de> Am 24.10.2012 17:47, schrieb Bill Shirley: > > On 10/24/2012 2:33 AM, Robert Schetterer wrote: >> Am 24.10.2012 03:32, schrieb Bill Shirley: >>> What is your mailbox_command in main.cf? I just use: >>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >>> >>> I don't need anything in master.cf. But you should be using -u ${user} >>> for spamc. >> long time ago i tested this with dovecot lda postfix master.cf >> with a total virtual setup >> >> dovecot unix - n n - - pipe >> flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e >> /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} >> >> but i strongly do not recommand this !!! >> >> use spamass-milter, amavis etc with dovecot lmtp >> as described on many sites >> >> >> Best Regards >> MfG Robert Schetterer >> > > Can you get per-user Spamassassin configs this way? > > Why user=vmail:vmail? Is this for virtual domains? I didn't think we > were talking about them. > > Instead of strongly recommending against this, why not elaborate on the > problems with using spamc in the mailbox_command? > > Bill > Hi Bill, you missed my "i tested this with dovecot lda" in hope you may adapt the syntax to your needs by your own here are the recommanded setups http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix http://wiki.dovecot.org/LDA/Postfix --snip mailbox_command = /usr/bin/spamc -e /usr/lib/dovecot/deliver --snipend by the way using dovecot lmtp and i.e amavis or spamass-milter/clamav-milter might be better choice in many ways Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From troy at troyvit.com Wed Oct 24 19:10:38 2012 From: troy at troyvit.com (Troy Vitullo) Date: Wed, 24 Oct 2012 10:10:38 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5087454B.5030704@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> Message-ID: <20121024101038.5f3316f2@hrafn> On Tue, 23 Oct 2012 21:32:59 -0400 Bill Shirley wrote: > On 10/23/2012 9:06 PM, Bill Shirley wrote: > > > > > > What is your mailbox_command in main.cf? I just use: > > mailbox_command = /usr/bin/spamc -u "$USER" -e > > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m > > "$EXTENSION" > > > > I don't need anything in master.cf. But you should be using -u > > ${user} for spamc. > > > > Bill > > > Forgot to ask, are you using Spamassassin's per-user configs? If > you're not, that probably is your problem. It's probably trying to > update bayes tokens and it doesn't have permission. > > I use per-user configs which are nice. One man's spam is another > man's ham. Plus each user can have his/her own whitelist. > > I use these spamd args: -d -c -m10 --user-config > You usually can find the args in /etc/sysconfig. > > Bill Thanks for getting back to me Bill. Actually I'm using per-user prefs and permissions look great all the way down. When I send a test mail with everything turned on the bayes tokens are updated. Things appear to die later in the process. Regarding the mailbox command I was using: mailbox_command = /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" I tried removing the flags from master.cf and changing my command to: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" and then: mailbox_command = /usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} -m "$EXTENSION" and everything in between. No mail made it through, so I kept this in master.cf: dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/lib/dovecot/deliver -d ${recipient} and of course it over-rode my mailbox_command. Mail came thrrough but it contained no spamassassin header. I'm starting to thing that spamc doesn't have the permissions to write its headers to the message. How can I test that theory? spamd runs witht these flags: /usr/sbin/spamd --create-prefs -x --max-children 3 --username spamd --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid It's pretty much the same as yours, I just use the long versions of the args. the spamd user exists: spamd:x:1010:1011::/var/lib/spamassassin:/bin/false I was missing /etc/dovecot/default.sieve, which had to be a big problem, but I recovered it. Here's are its contents: require "fileinto"; if exists "X-Spam-Flag" { if header :contains "X-Spam-Flag" "NO" { } else { discard; stop; } } Anything else I could be missing? I even insanely running spamd as the root user: /usr/sbin/spamd --create-prefs -x --max-children 3 --username root --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid Thanks, Troy From rs at sys4.de Wed Oct 24 19:16:43 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 18:16:43 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087FEED.7060007@Media-Brokers.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> Message-ID: <5088146B.606@sys4.de> Am 24.10.2012 16:45, schrieb Charles Marcus: > On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Hmmm... I wonder would it be possible for dovecot to automatically lower > the 'niceness' for index rebuilds (on systems that support such) to > avoid causing such distress? > i think you missed Stans point ,looking for some hardware problems first Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From Bill at KnoxvilleChristian.org Wed Oct 24 19:28:48 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 12:28:48 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508812A8.8000603@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> Message-ID: <50881740.90207@KnoxvilleChristian.org> On 10/24/2012 12:09 PM, Robert Schetterer wrote: > Am 24.10.2012 17:47, schrieb Bill Shirley: >> On 10/24/2012 2:33 AM, Robert Schetterer wrote: >>> Am 24.10.2012 03:32, schrieb Bill Shirley: >>>> What is your mailbox_command in main.cf? I just use: >>>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >>>> >>>> I don't need anything in master.cf. But you should be using -u ${user} >>>> for spamc. >>> long time ago i tested this with dovecot lda postfix master.cf >>> with a total virtual setup >>> >>> dovecot unix - n n - - pipe >>> flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e >>> /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} >>> >>> but i strongly do not recommand this !!! >>> >>> use spamass-milter, amavis etc with dovecot lmtp >>> as described on many sites >>> >>> >>> Best Regards >>> MfG Robert Schetterer >>> >> Can you get per-user Spamassassin configs this way? >> >> Why user=vmail:vmail? Is this for virtual domains? I didn't think we >> were talking about them. >> >> Instead of strongly recommending against this, why not elaborate on the >> problems with using spamc in the mailbox_command? >> >> Bill >> > Hi Bill, you missed > > my > > "i tested this with dovecot lda" > in hope you may adapt the syntax to your needs by your own > > here are the recommanded setups > > http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix > http://wiki.dovecot.org/LDA/Postfix > > --snip > mailbox_command = /usr/bin/spamc -e /usr/lib/dovecot/deliver > --snipend > > by the way using dovecot lmtp and i.e amavis or spamass-milter/clamav-milter > > might be better choice in many ways > > > Best Regards > MfG Robert Schetterer > I'm saying I have a WORKING setup (local and virtual) where spamc runs and then uses dovecot deliver. spamd uses spamassassin per-user configs. master.cf has (caution, line wraps around in email): vdovecot unix - n n - 5 pipe flags=DRuh user=vmail:vmail argv=/usr/bin/spamc -p 784 -u ${recipient} -e /usr/lib64/dovecot/deliver -d ${user}@${domain} -a {recipient} -f ${sender} -n -m ${extension} main.cf has: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" virtual_transport = vdovecot vdovecot_destination_recipient_limit = 1 I don't understand why you strongly recommend against using the mailbox_command. Is there a security risk here? I've read all the howtos. There are many ways to setup a mail server. That's the beauty of postfix, spamassassin, dovecot, etc; you can make it do what you want. Yes, some setups are bad. I am not the original poster. Hope this clears things up, Bill From rob0 at gmx.co.uk Wed Oct 24 19:32:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 11:32:55 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121023145245.124dd362@hrafn> References: <20121023145245.124dd362@hrafn> Message-ID: <20121024163255.GI3672@harrier.slackbuilds.org> There seems to be much confusion in this thread. I might be able to help clear up some of it, but probably not all, because I agree with Robert about using amavisd-new for filtering and LMTP for delivery. On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > My server uses a system comprised of postfix, dovecot and dspam to > filter and deliver mail. > > Postfix used the following flags in calling spamc and dovecot: > > flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} > -e /usr/lib/dovecot/deliver -d ${recipient} This looks like you might be using pipe(8). If so, refer to the manual, and note that you are invoking this command as user "dovecot" and group "secmail". That is wrong use of the "dovecot" user. You probably should have made and used a dedicated "vmail" user. And according to your own post, q.v., the group "secmail" is definitely wrong. > after an upgrade from Debian lenny to squeeze we were able to get > everything working except spam filtering. Spamassassin is able to > judge whether the mail coming in is spam but everything stops > there. Automated or semi-automated upgrades are often a source of pain. > In mail.err I see: > > pamc[3608]: exec failed: Permission denied I guess that is spamc, and yes, of course. > spamc shows the same thing in syslog: > > exec failed: Permission denied > > postfix delays the email: > > postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > (system resource problem) > > Here are the permissions for deliver: > > -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver The pipe command is not executed as root. Nor is it invoked with the GID "dovecot". You specified group "secmail". Therefore the "other" permissions are what apply. "---" is no read, no write, no execute. > Here are the relevant groups: > > s1:~# grep dovecot /etc/group > secmail:x:119:postfix,spamd,dovecot This is not relevant. The process has EGID secmail, and the fact that dovecot is a member of secmail does not matter. Bottom line here: it seems that you misunderstood what the group permissions meant. > dovecot:x:111: > > here's the dovecot user: > s1:~# grep dovecot /etc/passwd > dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false > > here's dovecot -n: > > # 1.2.15: /etc/dovecot/dovecot.conf You upgraded -- to 1.2.15? Why? snip > Many thanks in advance for any advice you can give. Again, you should check on the wiki about the appropriate use of the "dovecot" user, and also read the wiki about virtual mailboxes. Fix that. Even if you make it work with permissions, you are breaking Dovecot's security model of privilege separation. The "dovecot" user is for Dovecot's internal use only, not for delivering mail and ownership of mailboxes. The poster who was talking about postconf(5) mailbox_command was bringing in a red herring. That is for local(8) delivery, and you evidently are using pipe(8). -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rob0 at gmx.co.uk Wed Oct 24 19:44:48 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 11:44:48 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50881740.90207@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> Message-ID: <20121024164448.GJ3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 12:28:48PM -0400, Bill Shirley wrote: > I don't understand why you strongly recommend against using the > mailbox_command. Is there a security risk here? One issue is that mailbox_command is only used for local(8) delivery. You brought that up for the OP, who is reporting a problem in trying to use pipe(8). mailbox_command is not relevant for pipe. That added more confusion to the issue at hand. I can't speak for Robert, but as I said in the other post I agree with him, so I will say why. You will get better overall performance with amavisd-new and LMTP, rather than invoking a command via pipe for every delivery. No, mailbox_command in itself is not a security risk, except insofar as you could DoS yourself with more deliveries at once than the system is able to handle. Some risk of DoS is present for any kind of content filtering, though. But amavisd-new after-queue reduces that risk. > I've read all the howtos. Eww. I have not. I have made extensive referral to the documentation, however, and that is what I recommend. Many thousands of people who are generating web content do not know much about email. You don't want to turn to them for advice about this! (FWIW, many of the howtos I have looked at are very bad.) > There are many ways to setup a mail server. That's the beauty of > postfix, spamassassin, dovecot, etc; you can make it do what you > want. Yes, some setups are bad. Yes and yes. > I am not the original poster. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From Bill at KnoxvilleChristian.org Wed Oct 24 20:13:42 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:13:42 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024101038.5f3316f2@hrafn> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <20121024101038.5f3316f2@hrafn> Message-ID: <508821C6.4010608@KnoxvilleChristian.org> On 10/24/2012 12:10 PM, Troy Vitullo wrote: > On Tue, 23 Oct 2012 21:32:59 -0400 > Bill Shirley wrote: > >> On 10/23/2012 9:06 PM, Bill Shirley wrote: >>> >>> What is your mailbox_command in main.cf? I just use: >>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m >>> "$EXTENSION" >>> >>> I don't need anything in master.cf. But you should be using -u >>> ${user} for spamc. >>> >>> Bill >>> >> Forgot to ask, are you using Spamassassin's per-user configs? If >> you're not, that probably is your problem. It's probably trying to >> update bayes tokens and it doesn't have permission. >> >> I use per-user configs which are nice. One man's spam is another >> man's ham. Plus each user can have his/her own whitelist. >> >> I use these spamd args: -d -c -m10 --user-config >> You usually can find the args in /etc/sysconfig. >> >> Bill > Thanks for getting back to me Bill. Actually I'm using per-user prefs and permissions look great all the way down. When I send a test mail with everything turned on the bayes tokens are updated. Things appear to die later in the process. > > Regarding the mailbox command I was using: > mailbox_command = /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" > > I tried removing the flags from master.cf and changing my command to: > mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" What was your setting for mailbox_transport (in main.cf) when you did this? mailbox_transport could be overriding mailbox_command. > > and then: > mailbox_command = /usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} -m "$EXTENSION" > > and everything in between. > > No mail made it through, so I kept this in master.cf: > > dovecot unix - n n - - pipe > flags=DRhu user=dovecot:dovecot argv=/usr/lib/dovecot/deliver -d ${recipient} Where are you calling spamc with this? > > and of course it over-rode my mailbox_command. Mail came thrrough but it contained no spamassassin header. > > I'm starting to thing that spamc doesn't have the permissions to write its headers to the message. How can I test that theory? > > spamd runs witht these flags: > /usr/sbin/spamd --create-prefs -x --max-children 3 --username spamd --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid > > It's pretty much the same as yours, I just use the long versions of the args. > > the spamd user exists: > spamd:x:1010:1011::/var/lib/spamassassin:/bin/false Your permissions on /var/lib/spamassassin are probably right, but check them and the subdirectories. > > I was missing /etc/dovecot/default.sieve, which had to be a big problem, but I recovered it. Here's are its contents: > > require "fileinto"; > if exists "X-Spam-Flag" { > if header :contains "X-Spam-Flag" "NO" { > } else { > discard; > stop; > } > } > > Anything else I could be missing? I even insanely running spamd as the root user: > > /usr/sbin/spamd --create-prefs -x --max-children 3 --username root --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid > > Thanks, > > Troy I have two instances of spamd running. One for local users and the other for virtual users (note the port here and in master.cf): [root at elmo includes]# ps aux | grep spamd root 2684 0.1 1.0 173760 88484 ? SN 03:30 0:34 spamd child root 23987 0.0 0.7 147524 61900 ? SNs Oct23 0:05 /usr/bin/spamd -d -c -m10 --user-config root 24004 0.0 0.7 147504 61844 ? SNs Oct23 0:05 /usr/bin/spamd -d -c -m5 -x --virtual-config-dir=/home/vmail/domains/%d/%l/.spamassassin -u vmail --port=784 -H vmail 24014 0.0 0.9 161204 75880 ? SN Oct23 0:05 spamd child vmail 24015 0.0 0.7 147504 59700 ? SN Oct23 0:00 spamd child root 25772 0.0 0.8 155020 69188 ? SN 12:07 0:00 spamd child root 28981 0.0 0.0 16688 940 pts/4 S+ 12:36 0:00 grep --color spamd My vmail user: [root at elmo includes]# grep vmail /etc/{group,passwd} /etc/group:vmail:x:399: /etc/passwd:vmail:x:399:399:Virtual Mail:/home/vmail:/bin/bash My virtual user .spamassassin permissions: [root at elmo includes]# ldp /home/vmail/domains/example.com/bill/.spamassassin drwxr-xr-x 20 root root 4096 May 8 2011 /home drwxr-xr-x 10 vmail vmail 4096 Oct 22 10:59 /home/vmail drwxr-x--- 9 vmail vmail 4096 Oct 21 21:24 /home/vmail/domains drwxr-x--- 6 vmail vmail 4096 Jul 4 2007 /home/vmail/domains/example.com drwxr-x--- 4 vmail vmail 4096 Jul 4 2007 /home/vmail/domains/example.com/bill drwxr-s--- 3 vmail vmail 4096 Jan 30 2012 /home/vmail/domains/example.com/bill/.spamassassin My local user: [root at elmo includes]# ldp /home/bill/.spamassassin drwxr-xr-x 20 root root 4096 May 8 2011 /home drwxr-xr-x 32 bill bill 4096 Oct 22 17:42 /home/bill drwxr-s--- 2 bill bill 4096 Oct 24 12:42 /home/bill/.spamassassin My main.cf: mailbox_transport = mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" virtual_transport = vdovecot vdovecot_destination_recipient_limit = 1 My master.cf: vdovecot unix - n n - 5 pipe flags=DRuh user=vmail:vmail argv=/usr/bin/spamc -p 784 -u ${recipient} -e /usr/lib64/dovecot/deliver -d ${user}@${domain} -a {recipient} -f ${sender} -n -m ${extension} You could try my config substituting your user and directory for mine: I'm using user=vmail:vmail and --virtual-config-dir=/home/vmail/domains/%d/%l/.spamassassin You're using user=dovecot:secmail and --virtual-config-dir=/var/lib/spamassassin/users/%d/%l Currently, your user=dovecot:secmail should probably be user=spamd:spamd in master.cf unless group secmail has write permissions on /var/lib/spamassassin and subdirectories. Hope this helps, Bill From Bill at KnoxvilleChristian.org Wed Oct 24 20:21:58 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:21:58 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024164448.GJ3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> Message-ID: <508823B6.3040208@KnoxvilleChristian.org> On 10/24/2012 12:44 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 12:28:48PM -0400, Bill Shirley wrote: >> I don't understand why you strongly recommend against using the >> mailbox_command. Is there a security risk here? > One issue is that mailbox_command is only used for local(8) delivery. > You brought that up for the OP, who is reporting a problem in trying > to use pipe(8). mailbox_command is not relevant for pipe. That added > more confusion to the issue at hand. It was my understanding that he is implementing local users. > > I can't speak for Robert, but as I said in the other post I agree > with him, so I will say why. You will get better overall performance > with amavisd-new and LMTP, rather than invoking a command via pipe > for every delivery. Admittedly, I have not used amavisd-new or LMTP; they may be better. But will they allow spamassassin per-user prefs? Performance is a plus; another daemon is not. That saying, I'll run another daemon if I get something out of it. Any benchmarks on this? > > No, mailbox_command in itself is not a security risk, except insofar > as you could DoS yourself with more deliveries at once than the > system is able to handle. Some risk of DoS is present for any kind of > content filtering, though. But amavisd-new after-queue reduces that > risk. > >> I've read all the howtos. > Eww. I have not. I have made extensive referral to the documentation, > however, and that is what I recommend. Many thousands of people who > are generating web content do not know much about email. You don't > want to turn to them for advice about this! Probably mis-spoke; I said howtos instead of documentation. Yes, there are many bad howtos out there. > > (FWIW, many of the howtos I have looked at are very bad.) > >> There are many ways to setup a mail server. That's the beauty of >> postfix, spamassassin, dovecot, etc; you can make it do what you >> want. Yes, some setups are bad. > Yes and yes. > >> I am not the original poster. Respectfully, Bill From Bill at KnoxvilleChristian.org Wed Oct 24 20:28:41 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:28:41 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024163255.GI3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> Message-ID: <50882549.3020505@KnoxvilleChristian.org> On 10/24/2012 12:32 PM, /dev/rob0 wrote: > There seems to be much confusion in this thread. I might be able to > help clear up some of it, but probably not all, because I agree with > Robert about using amavisd-new for filtering and LMTP for delivery. > > On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: >> My server uses a system comprised of postfix, dovecot and dspam to >> filter and deliver mail. >> >> Postfix used the following flags in calling spamc and dovecot: >> >> flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} >> -e /usr/lib/dovecot/deliver -d ${recipient} > This looks like you might be using pipe(8). If so, refer to the > manual, and note that you are invoking this command as user "dovecot" > and group "secmail". > > That is wrong use of the "dovecot" user. You probably should have > made and used a dedicated "vmail" user. And according to your own > post, q.v., the group "secmail" is definitely wrong. > >> after an upgrade from Debian lenny to squeeze we were able to get >> everything working except spam filtering. Spamassassin is able to >> judge whether the mail coming in is spam but everything stops >> there. > Automated or semi-automated upgrades are often a source of pain. > >> In mail.err I see: >> >> pamc[3608]: exec failed: Permission denied > I guess that is spamc, and yes, of course. > >> spamc shows the same thing in syslog: >> >> exec failed: Permission denied >> >> postfix delays the email: >> >> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >> (system resource problem) >> >> Here are the permissions for deliver: >> >> -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver > The pipe command is not executed as root. Nor is it invoked with the > GID "dovecot". You specified group "secmail". Therefore the "other" > permissions are what apply. "---" is no read, no write, no execute. > >> Here are the relevant groups: >> >> s1:~# grep dovecot /etc/group >> secmail:x:119:postfix,spamd,dovecot > This is not relevant. The process has EGID secmail, and the fact that > dovecot is a member of secmail does not matter. Bottom line here: it > seems that you misunderstood what the group permissions meant. > >> dovecot:x:111: >> >> here's the dovecot user: >> s1:~# grep dovecot /etc/passwd >> dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false >> >> here's dovecot -n: >> >> # 1.2.15: /etc/dovecot/dovecot.conf > You upgraded -- to 1.2.15? Why? > > snip >> Many thanks in advance for any advice you can give. > Again, you should check on the wiki about the appropriate use of the > "dovecot" user, and also read the wiki about virtual mailboxes. Fix > that. Even if you make it work with permissions, you are breaking > Dovecot's security model of privilege separation. The "dovecot" user > is for Dovecot's internal use only, not for delivering mail and > ownership of mailboxes. > > The poster who was talking about postconf(5) mailbox_command was > bringing in a red herring. That is for local(8) delivery, and you > evidently are using pipe(8). Just a note: the original post did NOT have the word 'virtual' in it. If it did, I missed it and apologize for introducing confusion. Bill From rs at sys4.de Wed Oct 24 20:37:35 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 19:37:35 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50881740.90207@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> Message-ID: <5088275F.1030507@sys4.de> Am 24.10.2012 18:28, schrieb Bill Shirley: > I don't understand why you strongly recommend against using the > mailbox_command. Is there a security risk here? no ,until you dont have made any setup failures... your right there are tons of working possible setups your free to configure as you like, but lmtp with dovecot is state of the art in my eyes, these days in my tests lda combined with spamc had not enough performance for my needs and used to much resources compared to lmtp sometimes it crashed, but as i said ,long time ago however i found total virtual setups much more easy then with local by permissions stuff etc, and milters are much more easy to use and setup, also i.e amavis gives great other choices beside spamassassin stuff but do as you like ,no need to flame Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From rob0 at gmx.co.uk Wed Oct 24 20:39:18 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 12:39:18 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50882549.3020505@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> Message-ID: <20121024173918.GK3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: > On 10/24/2012 12:32 PM, /dev/rob0 wrote: > >There seems to be much confusion in this thread. I might be able > >able to help clear up some of it, but probably not all, because I > >agree with Robert about using amavisd-new for filtering and LMTP > >for delivery. > > > >On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: snip > >>postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > >>delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > >>(system resource problem) > >The poster who was talking about postconf(5) mailbox_command > >was bringing in a red herring. That is for local(8) delivery, > >and you evidently are using pipe(8). > Just a note: the original post did NOT have the word 'virtual' in > it. If it did, I missed it and apologize for introducing confusion. It did not, but it did indeed include the pipe log output shown above, and therefore ^mailbox_.* postconf settings do not apply. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From jeff at bubble.org Wed Oct 24 20:40:25 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Wed, 24 Oct 2012 13:40:25 -0400 Subject: [Dovecot] Snarf plugin Message-ID: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> I've now upgraded dovecot from 2.0.21 to 2.1.10 and the good news is I no longer see dovecot crashing when loading the snarf plugin however snarf still does not do anything except make the inbox disappear. I've come to the conclusion that either snarf does not actually work, possible, but I doubt it, or more likely I have a configuration issue preventing it from working. The system is simple, all email is stored in /var/spool/mail/{username} and I want all the mail moved to ~/mbox when the user logs in via imap, similar to uw-imap. Any guidance would really be appreciated. Thanks, Jeff dovecot -n # 2.1.10: //etc/dovecot/dovecot.conf # OS: Linux 3.6.2-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mbox mail_plugins = snarf zlib mbox_write_locks = fcntl namespace default { inbox = yes location = prefix = separator = / } namespace snarf { hidden = yes list = no location = mbox:/run/dovecot/empty:INBOX=/var/spool/mail/%u:INDEX=MEMORY prefix = /snarf separator = / } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mbox_snarf = ~/mbox snarf = /snarf/INBOX } service imap-login { inet_listener imap { address = localhost } } service pop3-login { inet_listener pop3 { address = localhost } } ssl_cert = , method=PLAIN, rip=::1, lip=::1, mpid=28089, secured, session= Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff:INDEX=MEMORY Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Disconnected: Logged out in=117 out=1504 From rob0 at gmx.co.uk Wed Oct 24 20:49:03 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 12:49:03 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508823B6.3040208@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> Message-ID: <20121024174903.GL3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 01:21:58PM -0400, Bill Shirley wrote: > On 10/24/2012 12:44 PM, /dev/rob0 wrote: > >I can't speak for Robert, but as I said in the other post I > >agree with him, so I will say why. You will get better overall > >performance with amavisd-new and LMTP, rather than invoking a > >command via pipe for every delivery. > Admittedly, I have not used amavisd-new or LMTP; they may be > better. But will they allow spamassassin per-user prefs? Amavisd-new is indeed capable of per-user preferences. > Performance is a plus; another daemon is not. That saying, I'll > run another daemon if I get something out of it. Any benchmarks > on this? A daemon is generally (I'd almost daresay "always") less overhead than the invocation of many single-delivery processes. No benchmarking is needed to support this fact. That said, for many small sites, it does not matter much. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From Bill at KnoxvilleChristian.org Wed Oct 24 20:56:18 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:56:18 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5088275F.1030507@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <5088275F.1030507@sys4.de> Message-ID: <50882BC2.2010702@KnoxvilleChristian.org> On 10/24/2012 1:37 PM, Robert Schetterer wrote: > Am 24.10.2012 18:28, schrieb Bill Shirley: >> I don't understand why you strongly recommend against using the >> mailbox_command. Is there a security risk here? > no ,until you dont have made any setup failures... > > your right there are tons of working possible setups > your free to configure as you like, but lmtp with dovecot is state of > the art in my eyes, these days > > in my tests lda combined with spamc had not enough > performance for my needs and used to much resources compared to lmtp > sometimes it crashed, but as i said ,long time ago > > however i found total virtual setups much more easy then with local > by permissions stuff etc, and milters are much more easy to use and > setup, also i.e amavis gives great other choices beside spamassassin stuff > > but do as you like ,no need to flame > > Best Regards > MfG Robert Schetterer > I don't see a flame anywhere in my posts. The list is for respectfully exchanging information. I thought that was what we were doing. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 21:04:39 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 14:04:39 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024173918.GK3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> Message-ID: <50882DB7.5030202@KnoxvilleChristian.org> On 10/24/2012 1:39 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: >> On 10/24/2012 12:32 PM, /dev/rob0 wrote: >>> There seems to be much confusion in this thread. I might be able >>> able to help clear up some of it, but probably not all, because I >>> agree with Robert about using amavisd-new for filtering and LMTP >>> for delivery. >>> >>> On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > snip >>>> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >>>> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >>>> (system resource problem) >>> The poster who was talking about postconf(5) mailbox_command >>> was bringing in a red herring. That is for local(8) delivery, >>> and you evidently are using pipe(8). >> Just a note: the original post did NOT have the word 'virtual' in >> it. If it did, I missed it and apologize for introducing confusion. > It did not, but it did indeed include the pipe log output shown > above, and therefore ^mailbox_.* postconf settings do not apply. Could be he was going about it the wrong way; mixing the two. Do you know whether he's trying to do virtual or local? My postings describe my implementation. I'm just trying to help him. But I don't think my posts are being received that way. Bill From carsten.delellis at delellis.net Wed Oct 24 21:22:14 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Wed, 24 Oct 2012 20:22:14 +0200 Subject: [Dovecot] dovecot auth against AD on samba4 In-Reply-To: <1351094679.2143.474.camel@worklian> References: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> <1351094679.2143.474.camel@worklian> Message-ID: <296201cdb214$7ef15e50$7cd41af0$@delellis.net> Dear Bob Thankx for your hint. I tried with jxplorer to connect to the AD ldap and I am pretty sure that I will get it up and running like I did with the openldap server. Carsten -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Bob Miller Sent: Mittwoch, 24. Oktober 2012 18:05 To: dovecot at dovecot.org Subject: Re: [Dovecot] dovecot auth against AD on samba4 I don't have it in production yet because there are other things I am still trying to add to samba4, but my test server has dovecot authenticating against samba4. Without openchange or any other non-native mechanism. Dovecot supports authenticating against ldap, the settings are in your auth-ldap.conf file. Samba4/Active Directory is just another ldap implementation. between the config files and the wiki, I believe all the documentation you need is there... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Wed, 2012-10-24 at 11:48 +0200, Carsten Laun-De Lellis wrote: > Hi group > > I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, > dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and > I am thinking about installing it because of it's native outlook support > capabilities. > > The ZEG appliance wouldn't be an option for me because I use a virtual > server from a provider where I can't install my own vm or even an iso. > > When I go thru the documentation there is a part with installing > OpenChange based on samba4. As far as I understood the OpenChange > authentication is against the samba4 AD. Actually there is no support in > syncing the AD against an OpenLdap Server and I would have to change the > OpenLdap port because the AD is listening on port 389. To change the port > wouldn't be a big deal, but what i was thinking about to run the dovecot > auth also against the samba 4 AD. > > I searched around on the internet but didn't find a doc yet how to do that. > > Does anyone here could provide me with a link or a how-to ? > > Thanks very much in advance. > > Regards, > > Carsten Laun-De Lellis > > Hauptstrasse 13 > D-67705 Trippstadt > > Phone: +49 6306 992140 > Fax: +49 6306 992142 > Mobile: +49 151 27530865 > email: carsten.delellis at delellis.net > > > From rs at sys4.de Wed Oct 24 21:24:31 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 20:24:31 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508823B6.3040208@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> Message-ID: <5088325F.7020102@sys4.de> Am 24.10.2012 19:21, schrieb Bill Shirley: > Admittedly, I have not used amavisd-new or LMTP; they may be better. > But will they allow spamassassin per-user prefs? Performance is a plus; > another daemon is not. That saying, I'll run another daemon if I get > something out of it. Any benchmarks on this? this went away from the orig post, it went to general design of a email system, i think rob did explain the possible problems to the orginal poster very fine some people may start with local users as traditional mailsetup depend on this next steps they are going to use lda perhaps trying combined with spamc with local users so there is nothing bad on it, its somehow old school, after all, as said ,there are many broken advices out in www by all setups, and sometimes there are mixed up by local and virtual, so people may fail with permissions of local users , daemons etc sometimes later if more domains should be hosted pure virtual setups are the better way, and making stuff more simple ( but often people fail first in seeing virtual more easy ), lmtp is the best choice for it compared starting a deliver process for each mail, its working as a service So anyone should think about what he needs before starting to setup i.e amavis is a well supported framework since long time, it has tons of features you might wanna have and as well it can be used with per-user prefs if you dont like the complex amavis style ( many functions have many config points ), you could simple use a chain of milter i.e spamass-milter ( also with per-user prefs ), clamav-milter with milter you are able to reject on smtp income stage which is very cool anyway milters also have their pros an contras, read postfix sites about them i didnt tested dspam looks like it chained between lmtp so perhaps also good choice, and could be combined with milters i had other setups with chained spampd/clamsmtp amavis on seperate filter hosts etc all worked fine but as dovecot/postfix development going forward , i redesigned all these depending to have more functions and performance so i recommand, use your working setups as i.e lifetime of your hardware etc, but if building new mailserver choose modern setup ideas and daemon combinations Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From rob0 at gmx.co.uk Wed Oct 24 21:25:52 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 13:25:52 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50882DB7.5030202@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> Message-ID: <20121024182552.GM3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 02:04:39PM -0400, Bill Shirley wrote: > On 10/24/2012 1:39 PM, /dev/rob0 wrote: > >On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: > >>On 10/24/2012 12:32 PM, /dev/rob0 wrote: > >>>On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > >snip > >>>>postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > >>>>delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > >>>>(system resource problem) > >>>The poster who was talking about postconf(5) mailbox_command > >>>was bringing in a red herring. That is for local(8) delivery, > >>>and you evidently are using pipe(8). > >>Just a note: the original post did NOT have the word 'virtual' > >>in it. If it did, I missed it and apologize for introducing > >>confusion. > >It did not, but it did indeed include the pipe log output shown > >above, and therefore ^mailbox_.* postconf settings do not apply. > > Could be he was going about it the wrong way; mixing the two. > Do you know whether he's trying to do virtual or local? There are lots of wrong ways. The most wrongful of the OP's ways I found was the misuse of the dovecot user. The second most wrong, which was the actual problem at hand, was a misunderstanding of how group permissions are applied. Mixing virtual and local in Postfix and Dovecot is no problem at all, and in fact multiple modes of delivery are possible, even within a given address class or even within a domain. All we know here is what the OP posted. You don't usually use pipe for delivery to local (Unix) users. > My postings describe my implementation. For the OP to change to local delivery would require reworking his setup extensively, on the Postfix side, and here we are on the Dovecot list, so I wouldn't go into that here. But sure, there are other (and for many purposes, better) means of doing what he might want to do. > I'm just trying to help him. But I don't think my posts are > being received that way. Regarding Robert's "flame" comment in the other subthread, I agree with you; I saw no flame. And I did not suggest that you were not trying to help. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rs at sys4.de Wed Oct 24 21:32:19 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 20:32:19 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024182552.GM3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> Message-ID: <50883433.8010609@sys4.de> Am 24.10.2012 20:25, schrieb /dev/rob0: > Regarding Robert's "flame" comment in the other subthread, I agree > with you; I saw no flame. And I did not suggest that you were not > trying to help take my sorry, as non native english, perhaps i missused "flame" here Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From CMarcus at Media-Brokers.com Wed Oct 24 21:48:57 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 24 Oct 2012 14:48:57 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5088146B.606@sys4.de> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> <5088146B.606@sys4.de> Message-ID: <50883819.7010005@Media-Brokers.com> On 2012-10-24 12:16 PM, Robert Schetterer wrote: > Am 24.10.2012 16:45, schrieb Charles Marcus: >> On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >>> If the box is truly unresponsive, i.e. hard locked, then the corrupted >>> indexes are only a symptom of the underlying problem, which is unrelated >>> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >>> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >>> You need to know the difference so we have accurate information to >>> troubleshoot with. >> Hmmm... I wonder would it be possible for dovecot to automatically lower >> the 'niceness' for index rebuilds (on systems that support such) to >> avoid causing such distress? > i think you missed Stans point ,looking for some hardware problems first No, I was simply commenting on the one point about heavy load during large index rebuilds - which is why I trimmed the quoted text... maybe I could have trimmed more? -- Best regards, Charles From Bill at KnoxvilleChristian.org Wed Oct 24 21:51:12 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 14:51:12 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024182552.GM3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> Message-ID: <508838A0.6090100@KnoxvilleChristian.org> On 10/24/2012 2:25 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 02:04:39PM -0400, Bill Shirley wrote: >> On 10/24/2012 1:39 PM, /dev/rob0 wrote: >>> On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: >>>> On 10/24/2012 12:32 PM, /dev/rob0 wrote: >>>>> On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: >>> snip >>>>>> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >>>>>> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >>>>>> (system resource problem) >>>>> The poster who was talking about postconf(5) mailbox_command >>>>> was bringing in a red herring. That is for local(8) delivery, >>>>> and you evidently are using pipe(8). >>>> Just a note: the original post did NOT have the word 'virtual' >>>> in it. If it did, I missed it and apologize for introducing >>>> confusion. >>> It did not, but it did indeed include the pipe log output shown >>> above, and therefore ^mailbox_.* postconf settings do not apply. >> Could be he was going about it the wrong way; mixing the two. >> Do you know whether he's trying to do virtual or local? > There are lots of wrong ways. The most wrongful of the OP's ways I > found was the misuse of the dovecot user. The second most wrong, > which was the actual problem at hand, was a misunderstanding of how > group permissions are applied. > > Mixing virtual and local in Postfix and Dovecot is no problem at all, > and in fact multiple modes of delivery are possible, even within a > given address class or even within a domain. > > All we know here is what the OP posted. You don't usually use pipe > for delivery to local (Unix) users. > >> My postings describe my implementation. > For the OP to change to local delivery would require reworking his > setup extensively, on the Postfix side, and here we are on the > Dovecot list, so I wouldn't go into that here. But sure, there are > other (and for many purposes, better) means of doing what he might > want to do. > >> I'm just trying to help him. But I don't think my posts are >> being received that way. > Regarding Robert's "flame" comment in the other subthread, I agree > with you; I saw no flame. And I did not suggest that you were not > trying to help. Thank you for saying this. My intent was to help. I make my living setting up/programming with open source software. I don't want to only 'take'. I want to show my gratitude for is so freely given to me by also giving. I don't program in C so I can't help with that. But I can share configurations/experiences and hopefully that is a contribution. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 23:04:20 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 16:04:20 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5088325F.7020102@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> <5088325F.7020102@sys4.de> Message-ID: <508849C4.9060800@KnoxvilleChristian.org> On 10/24/2012 2:24 PM, Robert Schetterer wrote: > Am 24.10.2012 19:21, schrieb Bill Shirley: >> Admittedly, I have not used amavisd-new or LMTP; they may be better. >> But will they allow spamassassin per-user prefs? Performance is a plus; >> another daemon is not. That saying, I'll run another daemon if I get >> something out of it. Any benchmarks on this? > this went away from the orig post, it went to general design > of a email system, i think rob did explain the possible problems > to the orginal poster very fine > > some people may start with local users as traditional > mailsetup depend on this next steps they are going to use lda > perhaps trying combined with spamc with local users > so there is nothing bad on it, its somehow old school, > after all, as said ,there are many broken advices out in www by all > setups, and sometimes there are mixed up by local and virtual, so people > may fail with permissions of local users , daemons etc > > sometimes later if more domains should be hosted > pure virtual setups are the better way, and making stuff more simple ( > but often people fail first in seeing virtual more easy ), > > lmtp is the best choice for it compared starting a deliver process for > each mail, its working as a service > > So anyone should think about what he needs before starting to setup > > i.e amavis is a well supported framework since long time, it has tons of > features > you might wanna have and as well it can be used with per-user prefs > > if you dont like the complex amavis style ( many functions have many > config points ), you could simple use a chain of milter i.e > spamass-milter ( also with per-user prefs ), clamav-milter > > with milter you are able to reject on smtp income stage > which is very cool > anyway milters also have their pros an contras, read postfix sites about > them > > i didnt tested dspam looks like it chained between lmtp > so perhaps also good choice, and could be combined with milters > > i had other setups with chained spampd/clamsmtp > amavis on seperate filter hosts etc > all worked fine > > but as dovecot/postfix development going forward , i redesigned all > these depending to have more functions and performance > > so i recommand, use your working setups as i.e lifetime of your hardware > etc, but if building new mailserver choose modern setup ideas > and daemon combinations > > > Best Regards > MfG Robert Schetterer > Thank you for a very informative post. I took a quick look at spamass-milter but I can't find any configuration information on how to use spamasssassin's per-user prefs. I thought the only way to support per-user prefs was post queue since you have to know who is getting the email to check their prefs. I am using clamav-milter. Milters are nice. I set my mail server up 15+ years ago, so it's time for me to have a re-think here. At that time there were no milters for postfix (don't remember a Dovecot either). I've try to steer away from re-injects since they affect the mail received numbers. Are we saying Dovecot's LMTP can call spamd? I'm on Dovecot 1.2 at home until I can upgrade. There is no LMTP in Dovecot 1.x, right? I have a few mail servers running Dovecot 2.0 and 2.1 and yes, I want them to perform well. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 23:06:01 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 16:06:01 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50883433.8010609@sys4.de> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> <50883433.8010609@sys4.de> Message-ID: <50884A29.1020901@KnoxvilleChristian.org> On 10/24/2012 2:32 PM, Robert Schetterer wrote: > Am 24.10.2012 20:25, schrieb /dev/rob0: >> Regarding Robert's "flame" comment in the other subthread, I agree >> with you; I saw no flame. And I did not suggest that you were not >> trying to help > take my sorry, as non native english, perhaps i missused "flame" here > > > > Best Regards > MfG Robert Schetterer > No problem. You do very well at speaking English. Bill From roundcube222 at alaadin.org Wed Oct 24 23:04:31 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 24 Oct 2012 23:04:31 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot Message-ID: <4c9608dd97036a48885e68205922d6b2@Coptics.org> Hello, I have a question regarding mailbox locking and hope any one can help me to better understanding the locking of mbox My Postfix lock option is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is fcntl only mbox_write_locks = fcntl mbox_read_locks = fcntl now, when user is download a large mail (20 MB) for example, dovecot locks /var/mail/user with fcntl until the users finish downloading the mail ok here comes my question. While the user is downloading the mail , a mail arrives so postfix make some thing weird 1- Postfix creates /var/mail/user.lock for 5 secs , then postfix defer the mail with reason that the /var/mail/user is locked and then delete the /var/mail/user.lock (after 5 secs) My question is ? why postfix create user.lock although it shouldn't because already dovecot fcntl it ? 1- Dovecot locks /var/mail/user using FCNTL 2- Posttix at the same time tries to FCNTL /var/mail/user .. but it fail since dovecot already fcnl it. 3- Postfix at the same time add dot lock /var/mail/user for 5 secs then remove the lock. the question is how come postfix dot lock /var/mail although it couldn't FCNTL the file in the first place ??????? what i was expecting is 1- Dovecot locks /var/mail/user using FCNTL 2- Postfix tries to FCNTL /var/mail/user 3- POSTFIX WILL NOT CREATE DOTLOCK file unless the FCNTL is released by dovecot!!!! Please advise if postfix will dot lock the file even if it couldn't FCNTL the file in the first place?? Regards From mailadmin at cubixys.com Thu Oct 25 00:07:24 2012 From: mailadmin at cubixys.com (Fasil) Date: Thu, 25 Oct 2012 00:07:24 +0300 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <507F65C5.3090803@brightok.net> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> <507F3C67.5020900@cubixys.com> <507F65C5.3090803@brightok.net> Message-ID: <5088588C.2060005@cubixys.com> Thanks for the input Jack. As I am using debian, the location to edit the ulimit is /etc/default/dovecot. There is no effect even after changing this value. I tried putting the value in the /etc/init.d/dovecot script without success. Fasil. On 10/18/2012 05:13 AM, Jack Bates wrote: > I'm using RHEL6 instead of ubuntu, but check the startup scripts. In > RHEL's case, the following file is sourced, so I updated it instead of > the startup scripts. > > cat /etc/sysconfig/dovecot > # Here you can specify your dovecot command line options. > # > #OPTIONS="" > ulimit -n 4096 > ulimit -u 5120 > > In addition, I had to also up the max allowed processes in the dovecot > config. 2.x and 1.x are different on this. > > http://wiki1.dovecot.org/LoginProcess <-1.x method > > > Jack > > > > On 10/17/2012 6:16 PM, Fasil wrote: >> Thanks Frank. >> Followed the URL and could not find any luck. >> >> Is there a way to change the value of 'max open files' of dovecot. >> I have tried to set the value in /etc/default/dovecot by setting >> ulimit. But the value is not getting applied. >> Could anyone help on this regard. >> >> Fasil. >> >> On 08/16/2012 09:17 AM, Frank Bonnet wrote: >>> hello >>> >>> here some useful informations >>> >>> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ >>> >>> Envoy? de mon iPhone. >>> >>> >>> Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : >>> >>>> Dear all, >>>> >>>> Thank you all for such a wonderful support. Hats off to all :) >>>> >>>> Few times I came across imap login issues where new users will not >>>> be allowed to login. >>>> /var/log/mail.err shows the error below >>>> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in >>>> creating login processes, slowing down for now >>>> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> >>>> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >>>> >>>> # ulimit -Hn >>>> 1024 >>>> >>>> # ulimit -Sn >>>> 1024 >>>> >>>> # cat /proc/sys/fs/file-max >>>> 1238548 >>>> >>>> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >>>> Max open files 1024 1024 files >>>> >>>> Please advice how to get rid off this. >>>> >>>> Fasil. >> >> > From troy at troyvit.com Thu Oct 25 00:34:18 2012 From: troy at troyvit.com (Troy Vitullo) Date: Wed, 24 Oct 2012 15:34:18 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024163255.GI3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> Message-ID: <20121024153418.7a183681@hrafn> On Wed, 24 Oct 2012 11:32:55 -0500 /dev/rob0 wrote: > There seems to be much confusion in this thread. I might be able to > help clear up some of it, but probably not all, because I agree with > Robert about using amavisd-new for filtering and LMTP for delivery. > Thanks for the reality check Rob. I'm circling back with the guy who originally set this up to see if we can get back on the right track. We are using pipe with virtual users, and dovecot doesn't own the mailboxes. If/when we get our collective act together and have more questions I'll respond in more detail. Thanks again, Troy From rs at sys4.de Thu Oct 25 00:40:36 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 23:40:36 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508849C4.9060800@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> <5088325F.7020102@sys4.de> <508849C4.9060800@KnoxvilleChristian.org> Message-ID: <50886054.9090503@sys4.de> Am 24.10.2012 22:04, schrieb Bill Shirley: > > On 10/24/2012 2:24 PM, Robert Schetterer wrote: >> Am 24.10.2012 19:21, schrieb Bill Shirley: >>> Admittedly, I have not used amavisd-new or LMTP; they may be better. >>> But will they allow spamassassin per-user prefs? Performance is a plus; >>> another daemon is not. That saying, I'll run another daemon if I get >>> something out of it. Any benchmarks on this? >> this went away from the orig post, it went to general design >> of a email system, i think rob did explain the possible problems >> to the orginal poster very fine >> >> some people may start with local users as traditional >> mailsetup depend on this next steps they are going to use lda >> perhaps trying combined with spamc with local users >> so there is nothing bad on it, its somehow old school, >> after all, as said ,there are many broken advices out in www by all >> setups, and sometimes there are mixed up by local and virtual, so people >> may fail with permissions of local users , daemons etc >> >> sometimes later if more domains should be hosted >> pure virtual setups are the better way, and making stuff more simple ( >> but often people fail first in seeing virtual more easy ), >> >> lmtp is the best choice for it compared starting a deliver process for >> each mail, its working as a service >> >> So anyone should think about what he needs before starting to setup >> >> i.e amavis is a well supported framework since long time, it has tons of >> features >> you might wanna have and as well it can be used with per-user prefs >> >> if you dont like the complex amavis style ( many functions have many >> config points ), you could simple use a chain of milter i.e >> spamass-milter ( also with per-user prefs ), clamav-milter >> >> with milter you are able to reject on smtp income stage >> which is very cool >> anyway milters also have their pros an contras, read postfix sites about >> them >> >> i didnt tested dspam looks like it chained between lmtp >> so perhaps also good choice, and could be combined with milters >> >> i had other setups with chained spampd/clamsmtp >> amavis on seperate filter hosts etc >> all worked fine >> >> but as dovecot/postfix development going forward , i redesigned all >> these depending to have more functions and performance >> >> so i recommand, use your working setups as i.e lifetime of your hardware >> etc, but if building new mailserver choose modern setup ideas >> and daemon combinations >> >> >> Best Regards >> MfG Robert Schetterer >> > Thank you for a very informative post. I took a quick look at > spamass-milter but I can't find any configuration information on how to > use spamasssassin's per-user prefs. I thought the only way to support > per-user prefs was post queue since you have to know who is getting the > email to check their prefs. you have to study its parameters ( they may differ by version and distro ) http://linux.die.net/man/1/spamass-milter i use it like /usr/sbin/spamass-milter -P /var/spool/postfix/spamass-milter/spamass.pid -f -p /var/spool/postfix/spamass/spamass.sock -f -e -x -I -u vmail -r 15 -i 127.0.0.1 i have my spamassassin setup with mysql for users self settings use i.e.e webmail horde with sam module, or something equal with i.e squirrelmail or roundcube but i managed it before ,also in using local files with maildrop as i said ,its not ideal cause of pre queue design, but reality shows good enough for big isp setup and it may be combined > > I am using clamav-milter. Milters are nice. for antispam using sanesecurity antispam signatures are nice thats "cheaper" then spamassassin > > I set my mail server up 15+ years ago, so it's time for me to have a > re-think here. At that time there were no milters for postfix (don't > remember a Dovecot either). I've try to steer away from re-injects > since they affect the mail received numbers. Are we saying Dovecot's > LMTP can call spamd? i dont tested ,looks like dspam can do it http://wiki2.dovecot.org/HowTo/Virtual%2BPostfix%2BDspam%2BDovecot I'm on Dovecot 1.2 at home until I can upgrade. > There is no LMTP in Dovecot 1.x, right? yes ,you should use 2.1.x > > I have a few mail servers running Dovecot 2.0 and 2.1 and yes, I want > them to perform well. so you may change setup layout > > Bill > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stocton12 at yahoo.com Thu Oct 25 01:13:14 2012 From: stocton12 at yahoo.com (b m) Date: Wed, 24 Oct 2012 15:13:14 -0700 (PDT) Subject: [Dovecot] Public folders and groups Message-ID: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? From stan at hardwarefreak.com Thu Oct 25 06:48:33 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 22:48:33 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087FEED.7060007@Media-Brokers.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> Message-ID: <5088B691.7030100@hardwarefreak.com> On 10/24/2012 9:45 AM, Charles Marcus wrote: > On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Hmmm... I wonder would it be possible for dovecot to automatically lower > the 'niceness' for index rebuilds (on systems that support such) to > avoid causing such distress? Changing the process priority would not help. Indexing a large mailbox is an IO bound, not a compute bound, operation. With Linux, changing from the CFQ to deadline scheduler may help some with low responsiveness. But the only real solution for such a case where iowait is bringing the system to its knees is to acquire storage with far greater IOPS and concurrent IO capability. I.e. a server. -- Stan From stan at hardwarefreak.com Thu Oct 25 06:57:47 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 22:57:47 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <4c9608dd97036a48885e68205922d6b2@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> Message-ID: <5088B8BB.9000109@hardwarefreak.com> On 10/24/2012 3:04 PM, Robert JR wrote: > I have a question regarding mailbox locking and hope any one can help me > to better understanding the locking of mbox > > My Postfix lock option is fcntl dotlock > mailbox_delivery_lock = fcntl, dotlock > virtual_mailbox_lock = fcntl, dotlock > > My Dovecot lock option is fcntl only > mbox_write_locks = fcntl > mbox_read_locks = fcntl Postfix is delivering the mail to dovecot. This is done via the deliver program or lmtp which are pipes, not files. Thus, why is Postfix attempting to write files in the user's mail directory? You write new mail to the mailbox file with either Dovecot or Postfix, not both. Fix that problem and the locking problem disappears. -- Stan From eliezer at ngtech.co.il Thu Oct 25 10:16:28 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 25 Oct 2012 09:16:28 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086F2BB.7010704@rename-it.nl> References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> Message-ID: <5088E74C.9030006@ngtech.co.il> On 10/23/2012 9:40 PM, Stephan Bosch wrote: > Also, could you provide your full configuration as output from `dovecot > -n` ? > > Regards, > > Stephan. > Thanks Stephan, I just upgraded from 2.0.17 to 2.1.9 and ph 0.3.1 (gentoo) and it seems to work as expected and dont leave any traces in the INBOX with the same script. I dont know the old ph version and since it was resolved i'm ok with it. My only problem is that it will put the file in the folder but will not mark the folder with the new file until I actually check the folder manually. it's not that much hustle but if there is a way to solve it I will be more then happy to hear about it. Thanks, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From roundcube222 at alaadin.org Thu Oct 25 10:23:29 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 10:23:29 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <5088B8BB.9000109@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: On 2012-10-25 06:57, Stan Hoeppner wrote: > On 10/24/2012 3:04 PM, Robert JR wrote: > >> I have a question regarding mailbox locking and hope any one can >> help >> me to better understanding the locking of mbox My Postfix lock >> option >> is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock >> virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is >> fcntl >> only mbox_write_locks = fcntl mbox_read_locks = fcntl > > > > Postfix is delivering the mail to dovecot. This is done via the > deliver > program or lmtp which are pipes, not files. Thus, why is Postfix > attempting to write files in the user's mail directory? > > You write new mail to the mailbox file with either Dovecot or > Postfix, > not both. Fix that problem and the locking problem disappears. Stan, sorry but you didnot understand my question at all, dovecot in this case is reading the mailbox file while user downloading the mail and not WRITING. only postfix write when a mail arrives and DOVECOT only read the mail. And even if both write to the file, I have already set the locking option of both to FCNTL so no problem should happen. My question is postfix locking option is = FCNTL, DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, why it dotlock the file 5 secs then remove the dotlock and say resource unaviable? why in the first place it dotlock the file, althought it couldnot FCNTL it in the first place since it is FCNTL by dovecot while reading the in the moment Any help will be greatly appreciated..... Regards Robert. From dg at dguhl.org Thu Oct 25 11:56:00 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 25 Oct 2012 10:56:00 +0200 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025085559.GA7323@laptop-dg.leere.eu> On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: [..] > should happen. My question is postfix locking option is = FCNTL, > DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, > why it dotlock the file 5 secs then remove the dotlock and say > resource unaviable? why in the first place it dotlock the file, > althought it couldnot FCNTL it in the first place since it is FCNTL > by dovecot while reading the in the moment You are on the wrong mailing list, this is no problem of Dovecot. Ask this question on postfix-users (maybe you are requested to show evidence of this behaviour). Dennis From roundcube222 at alaadin.org Thu Oct 25 12:26:10 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 12:26:10 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025085559.GA7323@laptop-dg.leere.eu> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025085559.GA7323@laptop-dg.leere.eu> Message-ID: On 2012-10-25 11:56, Dennis Guhl wrote: > On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > > [..] > >> should happen. My question is postfix locking option is = FCNTL, >> DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, >> why it dotlock the file 5 secs then remove the dotlock and say >> resource >> unaviable? why in the first place it dotlock the file, althought it >> couldnot FCNTL it in the first place since it is FCNTL by dovecot >> while >> reading the in the moment > > You are on the wrong mailing list, this is no problem of Dovecot. > > Ask this question on postfix-users (maybe you are requested to show > evidence of this behaviour). > > Dennis\ Thanks dennis for your reply. But, This is also an dovecot issue, because how dovecot use the FCNTL lock maybe different on how Postfix use it , that's why i had to ask also here at dovecot mailist this question. I think This behavior is understood by any one already have experience on how locking is made to /var/mail/files, I am just trying to understand how/why it happened. My question is postfix has locking option: FCNTL, DOTLOCK, and a file is already FCNTL by dovecot, will postfix Apply the DOTLOCK and continue try to FCNTL , althugh postfix already found that the file is already FCNTL by dovecot. So it shouldnot dotlock the mbx file untill it frist FCNTL the mailbox after dovecot finished. When i asked this question on irc postfix, they said that maybe FCNTL is used in different option with dovecot and this Might be the reason of the issue and recommended to ask here for help .. I guess Timo will be able to respond to this on the spot ... Thanks again. Robert JR. From jg at softjury.de Thu Oct 25 13:35:53 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Thu, 25 Oct 2012 12:35:53 +0200 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> Message-ID: <50891609.9070709@softjury.de> Am 25.10.2012 00:13, schrieb b m: > Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? Here a sentence to this, found in the dovecot wiki. (http://wiki2.dovecot.org/ACL) ACL groups support works by returning a comma-separated acl_groups extra field from userdb, which contains all the groups the user belongs to. It seems to be possible, I had an acl_groups field in my MySQL Database for this, I'am sure it is something like that in an AD too. From r.ordinas at math.univ-paris-diderot.fr Thu Oct 25 16:09:47 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Thu, 25 Oct 2012 15:09:47 +0200 Subject: [Dovecot] Small issue with "submission host" Message-ID: <50893A1B.2060205@math.univ-paris-diderot.fr> Hi everyone, I'm facing a small issue with the lda/lmtp "submission_host" feature in dovecot 2.0.14. When sending mail to MTA (in case of sieve filter forwarding for example), dovecot pass a RCPT TO command just after the EHLO. He's missing the MAIL FROM command. Therefore, my MTA show me a warning like this : "improper command pipelining after EHLO". How can i solve that ? Regards, Raphael From tlx at leuxner.net Thu Oct 25 17:08:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 25 Oct 2012 16:08:52 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <50893A1B.2060205@math.univ-paris-diderot.fr> References: <50893A1B.2060205@math.univ-paris-diderot.fr> Message-ID: <20121025140852.GA15639@nihlus.leuxner.net> On Thu, Oct 25, 2012 at 03:09:47PM +0200, Raphael Ordinas wrote: > When sending mail to MTA (in case of sieve filter forwarding for > example), dovecot pass a RCPT TO command just after the EHLO. He's > missing the MAIL FROM command. > Therefore, my MTA show me a warning like this : "improper command > pipelining after EHLO". Works for me with latest and greatest although I'm not using the 'submission_host' option but pure LMTP Unix socket: [...] service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } Best to show your 'doveconf -n' for more thoughts. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From rob0 at gmx.co.uk Thu Oct 25 17:28:00 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 25 Oct 2012 09:28:00 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025142800.GN3672@harrier.slackbuilds.org> On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > Stan, sorry but you didnot understand my question at all, dovecot > in this case is reading the mailbox file while user downloading the > mail and not WRITING. only postfix write when a mail arrives and > DOVECOT only read the mail. And even if both write to the file, I I can't answer (don't know), but I can tell you that this is not true. Dovecot also writes to the file: updating message read flags and such. > Any help will be greatly appreciated..... Maildir is not for everyone, but it does handle issues like this smoothly. The delivery agent is always able to deliver new mail. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From roundcube222 at alaadin.org Thu Oct 25 19:08:25 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 19:08:25 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025142800.GN3672@harrier.slackbuilds.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025142800.GN3672@harrier.slackbuilds.org> Message-ID: <23542f848cc61c879822b03810621256@Coptics.org> On 2012-10-25 17:28, /dev/rob0 wrote: > On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > >> Stan, sorry but you didnot understand my question at all, dovecot in >> this case is reading the mailbox file while user downloading the >> mail >> and not WRITING. only postfix write when a mail arrives and DOVECOT >> only read the mail. And even if both write to the file, I > > I can't answer (don't know), but I can tell you that this is not > true. Dovecot also writes to the file: updating message read flags > and such. > >> Any help will be greatly appreciated..... > > Maildir is not for everyone, but it does handle issues like this > smoothly. The delivery agent is always able to deliver new mail. We can not convert to maildir now as we have alot of users and converting each account will take a huge time .. well.. we can live with /var/mail/mailbox format , I just need to understand the locking issue and hope Timo will be able to answer our question.. Thanks again From stan at hardwarefreak.com Thu Oct 25 21:00:24 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:00:24 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <50897E38.6070304@hardwarefreak.com> On 10/25/2012 2:23 AM, Robert JR wrote: > On 2012-10-25 06:57, Stan Hoeppner wrote: > >> On 10/24/2012 3:04 PM, Robert JR wrote: >> >>> I have a question regarding mailbox locking and hope any one can help >>> me to better understanding the locking of mbox My Postfix lock option >>> is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock >>> virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is fcntl >>> only mbox_write_locks = fcntl mbox_read_locks = fcntl >> >> >> >> Postfix is delivering the mail to dovecot. This is done via the deliver >> program or lmtp which are pipes, not files. Thus, why is Postfix >> attempting to write files in the user's mail directory? >> >> You write new mail to the mailbox file with either Dovecot or Postfix, >> not both. Fix that problem and the locking problem disappears. > > Stan, sorry but you didnot understand my question at all Yes, actually I did, but I missed one part of it because I assumed you had Dovecot setup properly. It doesn't matter if the mbox locks are write or read or both. Locks are the problem, period, because you have two daemons fighting over the same files. The fix is absolutely trivial: Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates the file locking issue. Thus it also increases throughput as lock latency is eliminated. It also enables using all the Dovecot delivery plugins such as Sieve, Quota, anti-spam, etc. I had the same Postfix/Dovecot mbox locking problem many years ago when I first started using Dovecot. After the suggestions from the fine folks on this list I switched to LDA. It not only eliminated locking, it completely changed the character of my Dovecot install, both in performance and capabilities, as well as fixed some message flag problems, etc. If you're not using LDA with Postfix/Dovecot you're insane. ;) Some might say you're insane for using mbox but I feel it's fine for many installations. I use it myself. Our setups are very similar. To switch to LDA... If you're using 1.2.x http://wiki.dovecot.org/LDA/Postfix If you're using 2.x http://wiki2.dovecot.org/LDA/Postfix -- Stan From brintoul at sbcglobal.net Thu Oct 25 21:01:53 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 11:01:53 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process Message-ID: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Hello: I'm using Dovecot to provide IMAP services for Thunderbird clients. The user's mail is stored in Maildir format and the individual emails which the user "receives" are actually downloaded and put into the Maildir "repository" using a Python email client (POP3 client). I am trying to create the dovecot-uidlist file and maintain the "next UID" value within it, but I am having trouble with the UIDVALIDITY and the 128 bit mailbox global UID among other things. How can I "seed" a dovecot-uidlist file with these values..? Sorry if the information is not much to go on, please let me know what more information you might need to help point me in the right direction. Thanks, B. RIntoul From sven at svenhartge.de Thu Oct 25 21:06:17 2012 From: sven at svenhartge.de (Sven Hartge) Date: Thu, 25 Oct 2012 20:06:17 +0200 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <697ejv7veev8@mids.svenhartge.de> Bradley Rintoul wrote: > I'm using Dovecot to provide IMAP services for Thunderbird clients. > The user's mail is stored in Maildir format and the individual emails > which the user "receives" are actually downloaded and put into the > Maildir "repository" using a Python email client (POP3 client). > I am trying to create the dovecot-uidlist file and maintain the "next > UID" value within it, but I am having trouble with the UIDVALIDITY and > the 128 bit mailbox global UID among other things. How can I "seed" a > dovecot-uidlist file with these values..? > Sorry if the information is not much to go on, please let me know what > more information you might need to help point me in the right > direction. How about injecting the mails into the LDA. It will take care of proper delivery without the need for your programm to know the internal workings of dovecot. S? -- Sigmentation fault. Core dumped. From stan at hardwarefreak.com Thu Oct 25 21:23:37 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:23:37 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50897E38.6070304@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> Message-ID: <508983A9.9090605@hardwarefreak.com> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: > Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > the file locking issue. Thus it also increases throughput as lock > latency is eliminated. It also enables using all the Dovecot delivery > plugins such as Sieve, Quota, anti-spam, etc. I forgot to mention one very important feature of Dovecot LDA: New messages delivered by Postfix are indexed by LDA as they are written to the mailbox, flags updated at this time, etc. Thus when a mailbox is opened in an IMAP MUA, new messages are displayed instantly (I don't use POP but it's probably faster as well). With your current setup it can take from a few to many seconds to show new mail, depending on message count. With LDA new message count seems to have no impact on the speed of display. -- Stan From stan at hardwarefreak.com Thu Oct 25 21:37:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:37:50 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <508986FE.6060309@hardwarefreak.com> On 10/25/2012 1:01 PM, Bradley Rintoul wrote: > the individual emails which the user > "receives" are actually downloaded and put into the Maildir "repository" using a > Python email client (POP3 client). Tell us more about this. This doesn't sound like 'normal' email being fetched from an external service provider over a slow link scenario. This sounds more like an application server generating data files that are then POP'd down to the Dovecot server. Assuming that for now... If you're able to run a popd on this application server, why not run a simple smtp MTA and send these files directly to the user email addresses? Injecting the payload is a pretty simply shell command line, or from within a Python/Perl/etc script. Dovecot handles the rest as it arrives. Problem solved. The more you can do with smtp the better off you are and the easier it is. -- Stan From roundcube222 at alaadin.org Thu Oct 25 20:38:26 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 20:38:26 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508983A9.9090605@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> Message-ID: <239f53fb233beb2b45e68638cca24260@Coptics.org> On 2012-10-25 21:23, Stan Hoeppner wrote: > On 10/25/2012 1:00 PM, Stan Hoeppner wrote: > >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place >> of >> the Postfix local/virtual delivery agent. Using Dovecot LDA >> eliminates >> the file locking issue. Thus it also increases throughput as lock >> latency is eliminated. It also enables using all the Dovecot >> delivery >> plugins such as Sieve, Quota, anti-spam, etc. > > I forgot to mention one very important feature of Dovecot LDA: > > New messages delivered by Postfix are indexed by LDA as they are > written > to the mailbox, flags updated at this time, etc. Thus when a mailbox > is > opened in an IMAP MUA, new messages are displayed instantly (I don't > use > POP but it's probably faster as well). With your current setup it can > take from a few to many seconds to show new mail, depending on > message > count. With LDA new message count seems to have no impact on the > speed > of display. Thanks stan very much for your detailed answer, i will read about LDA to know how it works. But i still wonder why this mailbox locking issue and I hope for my referece some one at this form explain to me the issue reason. as since postfix,dovecot has the same lockign settings so why this issue happen ? I have spend 3 days searching for an answer and i couldnot find any.. I think only Timo can answer my question... Thanks again stan From brintoul at sbcglobal.net Thu Oct 25 22:13:34 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 12:13:34 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <697ejv7veev8@mids.svenhartge.de> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> Message-ID: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> This could be good. I'd never looked into the LDA - I will study up on it. Someone else was helping out here and I thought I'd shed some more light on what I'm doing here... Let's say someone has an account with Yahoo, for example. My Python code is fetching email from the user's Yahoo! account and placing it into the Dovecot Maildir storage for a particular user. Now when the user retrieves their mail, they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, of sorts... Thanks for the responses! (Is there an IRC channel?) ________________________________ From: Sven Hartge To: dovecot at dovecot.org Sent: Thu, October 25, 2012 11:07:26 AM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process Bradley Rintoul wrote: > I'm using Dovecot to provide IMAP services for Thunderbird clients. > The user's mail is stored in Maildir format and the individual emails > which the user "receives" are actually downloaded and put into the > Maildir "repository" using a Python email client (POP3 client). > I am trying to create the dovecot-uidlist file and maintain the "next > UID" value within it, but I am having trouble with the UIDVALIDITY and > the 128 bit mailbox global UID among other things. How can I "seed" a > dovecot-uidlist file with these values..? > Sorry if the information is not much to go on, please let me know what > more information you might need to help point me in the right > direction. How about injecting the mails into the LDA. It will take care of proper delivery without the need for your programm to know the internal workings of dovecot. S? -- Sigmentation fault. Core dumped. From slitt at troubleshooters.com Thu Oct 25 22:31:38 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 15:31:38 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <697ejv7veev8@mids.svenhartge.de> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> Message-ID: <20121025153138.39c6b363@mydesk> On Thu, 25 Oct 2012 20:06:17 +0200, Sven Hartge said: > Bradley Rintoul wrote: > > > I'm using Dovecot to provide IMAP services for Thunderbird clients. > > The user's mail is stored in Maildir format and the individual > > emails which the user "receives" are actually downloaded and put > > into the Maildir "repository" using a Python email client (POP3 > > client). > > > I am trying to create the dovecot-uidlist file and maintain the > > "next UID" value within it, but I am having trouble with the > > UIDVALIDITY and the 128 bit mailbox global UID among other things. > > How can I "seed" a dovecot-uidlist file with these values..? > > > Sorry if the information is not much to go on, please let me know > > what more information you might need to help point me in the right > > direction. > > How about injecting the mails into the LDA. It will take care of > proper delivery without the need for your programm to know the > internal workings of dovecot. What is an LDA? SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From slitt at troubleshooters.com Thu Oct 25 22:35:06 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 15:35:06 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <20121025153138.39c6b363@mydesk> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <20121025153138.39c6b363@mydesk> Message-ID: <20121025153506.1f959544@mydesk> On Thu, 25 Oct 2012 15:31:38 -0400, Steve Litt said: > On Thu, 25 Oct 2012 20:06:17 +0200, Sven Hartge said: > > How about injecting the mails into the LDA. It will take care of > > proper delivery without the need for your programm to know the > > internal workings of dovecot. > > What is an LDA? Nevermind, somebody had already answered that question. Sorry for asking too quickly. SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From loshkovskyi at yandex.ua Thu Oct 25 13:03:23 2012 From: loshkovskyi at yandex.ua (Loshkovskyi Andrii) Date: Thu, 25 Oct 2012 13:03:23 +0300 Subject: [Dovecot] Dovecot sends BYE while fetching X-GM-MSGID Message-ID: <1698191351159403@web29e.yandex.ru> Hello everyone, While using the following set of commands, I am having the error as below: FETCH 7 (X-GM-MSGID) A15 FETCH 7 (X-GM-MSGID) A15 BAD Error in IMAP command FETCH: Unknown parameter X-GM-MSGID Can I somehow disable such errors so that Dovecot won't send BYE on X-GM-MSGID but just proceed with following emails? -- My configuration is below: [root at server ~]# cat /etc/issue CentOS release 6.3 (Final) [root at server ~]# dovecot --version 2.0.9 [root at server ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.17.1.el6.centos.plus.i686 i686 CentOS release 6.3 (Final) ext4 auth_debug = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes default_client_limit = 10000 default_process_limit = 300 disable_plaintext_auth = no listen = * mail_location = maildir:/var/spool/mail/%d/%n/Maildir mbox_write_locks = fcntl passdb { args = scheme=SSHA512 username_format=%u /etc/dovecot/users driver = passwd-file } service imap-login { service_count = 0 vsz_limit = 128 M } ssl_cert = References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <508995D2.3070705@whyscream.net> On 25/10/12 21:13, Bradley Rintoul wrote: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on what > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... I'm intrigued by this. Why are you using some self-baked(?) python script to fetch the mail in stead of using ready-made components like fetchmail? Unless there's a special reason not to, try using the LDA (and fetchmail/getmail for that matter). This sounds exactly what you want: http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal -- Tom > > Thanks for the responses! > > (Is there an IRC channel?) > > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > From rs at sys4.de Thu Oct 25 22:46:20 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 25 Oct 2012 21:46:20 +0200 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <5089970C.1080900@sys4.de> Am 25.10.2012 21:13, schrieb Bradley Rintoul: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on what > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... > > Thanks for the responses! > > (Is there an IRC channel?) > perhaps you could use parts from here http://wiki.dovecot.org/HowTo/RefilterMail > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft M?nchen Amtsgericht M?nchen HRB 0000 Vorstandsmitglieder Patrick Ben Koetter Axel von der Ohe Marc Schifbauer Vorstandsvorsitzender Patrick Ben Koetter Aufsichtsratsvorsitzender J?rg Heidrich From brintoul at sbcglobal.net Thu Oct 25 22:54:43 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 12:54:43 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <508995D2.3070705@whyscream.net> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> Message-ID: <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> I am brand new to this whole "email" thing. I am looking at this article right now: http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot Thanks for the input! ________________________________ From: Tom Hendrikx To: dovecot at dovecot.org Sent: Thu, October 25, 2012 12:41:24 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process On 25/10/12 21:13, Bradley Rintoul wrote: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on >what > > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their >mail, > > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > > of sorts... I'm intrigued by this. Why are you using some self-baked(?) python script to fetch the mail in stead of using ready-made components like fetchmail? Unless there's a special reason not to, try using the LDA (and fetchmail/getmail for that matter). This sounds exactly what you want: http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal -- Tom > > Thanks for the responses! > > (Is there an IRC channel?) > > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > From rob0 at gmx.co.uk Thu Oct 25 23:10:59 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 25 Oct 2012 15:10:59 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <20121025201059.GO3672@harrier.slackbuilds.org> > From: Tom Hendrikx > I'm intrigued by this. Why are you using some self-baked(?) python > script to fetch the mail in stead of using ready-made components > like fetchmail? > > Unless there's a special reason not to, try using the LDA (and > fetchmail/getmail for that matter). > > This sounds exactly what you want: > http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal > On Thu, Oct 25, 2012 at 12:54:43PM -0700, Bradley Rintoul wrote: > I am brand new to this whole "email" thing. I am looking at this > article right now: > http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot I did not see where you described the ultimate goal. That should have been the starting point of this thread. Describe the problem, not how you think it should be solved, because you are new to this, and your ideas might benefit from some scrutiny. Use plain language. I have not reviewed your howto, but personally I would recommend neither fetchmail (I'd choose getmail) nor procmail (other choices exist, depending on what you are trying to do.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Thu Oct 25 23:18:51 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 15:18:51 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <239f53fb233beb2b45e68638cca24260@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> Message-ID: <50899EAB.2030003@hardwarefreak.com> On 10/25/2012 12:38 PM, Robert JR wrote: > On 2012-10-25 21:23, Stan Hoeppner wrote: > >> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: >> >>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >>> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >>> the file locking issue. Thus it also increases throughput as lock >>> latency is eliminated. It also enables using all the Dovecot delivery >>> plugins such as Sieve, Quota, anti-spam, etc. >> >> I forgot to mention one very important feature of Dovecot LDA: >> >> New messages delivered by Postfix are indexed by LDA as they are written >> to the mailbox, flags updated at this time, etc. Thus when a mailbox is >> opened in an IMAP MUA, new messages are displayed instantly (I don't use >> POP but it's probably faster as well). With your current setup it can >> take from a few to many seconds to show new mail, depending on message >> count. With LDA new message count seems to have no impact on the speed >> of display. > > Thanks stan very much for your detailed answer, i will read about LDA to > know how it works. But i still wonder why this mailbox locking issue and I > hope for my referece some one at this form explain to me the issue reason. > as since postfix,dovecot has the same lockign settings so why this issue > happen ? I have spend 3 days searching for an answer and i couldnot find > any.. I think only Timo can answer my question... I chose to focus on the permanent and proper solution, which is eliminating your lock contention altogether and enabling maximum performance and features. Learning about file locking problems between applications may be a noble endeavor, but at this point it's simply a waste of your time. A gazillion papers have been written about this subject over the years. If you're that interested hunt them down and read them. I'm sure Google can find 1000 of them quickly. You'll be looking for academic papers, not forum posts. > Thanks again stan You're welcome. -- Stan From slitt at troubleshooters.com Thu Oct 25 23:18:52 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 16:18:52 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <20121025161852.5d3c448f@mydesk> On Thu, 25 Oct 2012 12:13:34 -0700 (PDT), Bradley Rintoul said: > This could be good. I'd never looked into the LDA - I will study up > on it. > > Someone else was helping out here and I thought I'd shed some more > light on what I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python > code is fetching email from the user's Yahoo! account and placing it > into the Dovecot Maildir storage for a particular user. Now when > the user retrieves their mail, they are doing so using my Dovecot - > my Dovecot instance is acting as a proxy, of sorts... > > Thanks for the responses! > > (Is there an IRC channel?) Hi Bradley, I'm doing almost the exact same thing, but with fetchmail and procmail. I go out and grab my email from about five different places using fetchmail, which feeds the messages to procmail, with .procmailrc deciding where in the Dovecot maildir tree to put them. Your only need I *haven't* accomplished is having different users get their mail from my Dovecot, and to make sure each users' email goes where they can get it via IMAP connection to your Dovecot. If you can get different IMAP mailboxes for different users, you can put each user in .procmailrc so as to deliver to the correct box. Anyway, Procmail knows exactly how to submit an email to Dovecot, so you don't need to worry about actually placing the file into the tree, or anything like that. You mention you've written some Python code. If the purpose of your Python code is just to retrieve from SMTP servers, you can drop your Python code in favor of Fetchmail and Procmail. If your Python code actually does something with the emails, you can call a subset of your Python code from Procmail, to do its magic on each email. Here's how my fetchmail is running: 29588 ? Ss 0:21 fetchmail -f /home/slitt/.fetchmailrc And here's a partial view of my .fetchmailrc, showing my retrieval from Bluehost and gmail: =================================== set postmaster "slitt" set bouncemail set no spambounce set properties "" set daemon 180 #poll mail.a3b3.com protocol POP3: poll mail.a3b3.com protocol IMAP: user 'slitt at troubleshooters.com' there is 'slitt' here pass wouldnt_you_like_to_know limit 50000000 warnings 3200 expunge 60 ssl #Use ssl encryption sslcommonname "*.bluehost.com" sslcertck mda "/usr/bin/procmail -d %T" fetchall; poll imap.gmail.com protocol IMAP user 'litttest at gmail.com' there is 'slitt' here pass 'I_just_cant_say' #portnumber 993 limit 50000000 warnings 3200 expunge 60 mda "/usr/bin/procmail -d %T" fetchlimit 50 ssl; ==================================== Do you notice the "mda" line on both pulls? That means "use procmail as your mda", which just ships each email to Procmail. Procmail knows exactly how to deliver stuff to Dovecot. The following are the top several lines of my .procmailrc: =================================== DEFAULT=$HOME/mail/Maildir/.INBOX/ MAILDIR=$HOME/mail/Maildir/ LOCKFILE=$HOME/mail/.lock VERBOSE=no LOGFILE=$HOME/procmail/log #GARBAGE=.garbage/ GARBAGE=/dev/null PURGATORY=.garbage/ SUPREMUM=9876543210 #PROCMAIL SUPREMUM NUMBER, SEE http://www.perlcode.org/tutorials/procmail/proctut/proctip2.pod #### HANDLE STUFF FROM littdom at gmail.com and litttest at gmail.com #### :0: * ^Delivered-To:.*littdom at gmail.com .littdom_gmail/ :0: * ^Delivered-To:.*litttest at gmail.com .litttest_gmail/ =================================== A few explanations: First, I couldn't include my actual filters, because they are full of very unflattering comments concerning various trolls, ignos, blabbermouths, and proudly helpless fools. The $MAILDIR environment variable is the rood directory of your Maildir tree. $DEFAULT is the location of the main inbox for that -- I think it's where you put email that doesn't get routed elsewhere by Procmail. $GARBAGE is an environment var I made up as code for where filtered stuff gets sent. It's usually /dev/null because I don't want to see that junk again. However, I can temporarily change it to an actual IMAP directory for troubleshooting. $PURGATORY is junk that I actually want to OK the deletion of. I actually currently have nothing filtered to $PURGATORY, but it's there. $SUPREMUM is a very large number that is used in making OR logic, which is otherwise difficult. I couldn't make the $SUPREMUM env var work, so I had to use a literal, and here's a way I got all my magazines into one mailbox: :0: * 9876543210^0 ^From.*onsale.com * 9876543210^0 ^From.*pcmag.com * 9876543210^0 ^From.*itworld.com * 9876543210^0 ^From.*networkworld.info * 9876543210^0 ^From.*infoworld.com * 9876543210^0 ^From.*whatsnewnow.com * 9876543210^0 ^From.*eweek.com * 9876543210^0 ^From.*computerworld.com .mags/ By the way, BE SURE to note the slash after the directory name. That trailing slash tells Procmail that it's delivering to a Maildir, not to an (ugh) mbox. Anyway, I think you and I are doing very similar things, albeit for very different reasons. My motivation is that I consider all currently available email clients to be junk, and don't want them holding my email, so I hold it in a Dovecot hosted Maildir instead. I'll be interested in how you solve this. Please keep me (and probably everyone on this list) in the loop. Thank you so much! SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From mike at alaadin.org Thu Oct 25 22:21:24 2012 From: mike at alaadin.org (Mike John) Date: Thu, 25 Oct 2012 22:21:24 +0300 Subject: [Dovecot] Changing password for users Message-ID: <7a6a7cc64756b895059f7384b256019b@coptics.org> Hello, I am using dovecot (2.0.9) and using virtual users using passdb { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i make my virtual users change their passwords using web interface ? My users already uses squirrelmail to access their mail. is there a program to add to squirrelmail to add this function to the clients ? or should i user different separate website for password changing ? and what program/tool can help me with this ? Any ideas is greatly appreciated. Mike. From roundcube222 at alaadin.org Thu Oct 25 22:42:32 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 22:42:32 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50899EAB.2030003@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> <50899EAB.2030003@hardwarefreak.com> Message-ID: On 2012-10-25 23:18, Stan Hoeppner wrote: > On 10/25/2012 12:38 PM, Robert JR wrote: > >> On 2012-10-25 21:23, Stan Hoeppner wrote: >> >>> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: >>> >>>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in >>>> place of the Postfix local/virtual delivery agent. Using Dovecot >>>> LDA eliminates the file locking issue. Thus it also increases >>>> throughput as lock latency is eliminated. It also enables using >>>> all >>>> the Dovecot delivery plugins such as Sieve, Quota, anti-spam, etc. >>> I forgot to mention one very important feature of Dovecot LDA: New >>> messages delivered by Postfix are indexed by LDA as they are >>> written >>> to the mailbox, flags updated at this time, etc. Thus when a >>> mailbox >>> is opened in an IMAP MUA, new messages are displayed instantly (I >>> don't use POP but it's probably faster as well). With your current >>> setup it can take from a few to many seconds to show new mail, >>> depending on message count. With LDA new message count seems to >>> have >>> no impact on the speed of display. >> Thanks stan very much for your detailed answer, i will read about >> LDA >> to know how it works. But i still wonder why this mailbox locking >> issue >> and I hope for my referece some one at this form explain to me the >> issue reason. as since postfix,dovecot has the same lockign settings >> so >> why this issue happen ? I have spend 3 days searching for an answer >> and >> i couldnot find any.. I think only Timo can answer my question... > > I chose to focus on the permanent and proper solution, which is > eliminating your lock contention altogether and enabling maximum > performance and features. > > Learning about file locking problems between applications may be a > noble > endeavor, but at this point it's simply a waste of your time. A > gazillion papers have been written about this subject over the years. > If you're that interested hunt them down and read them. I'm sure > Google > can find 1000 of them quickly. You'll be looking for academic papers, > not forum posts. > >> Thanks again stan > > You're welcome. Thanks again Stan, you are very helpfull, i will start learning how to configure LDA, but hopefully i can also have an answer from Timo about why this issue happened.. i am sure he is aware of it and can explain it.. From stocton12 at yahoo.com Thu Oct 25 23:49:52 2012 From: stocton12 at yahoo.com (b m) Date: Thu, 25 Oct 2012 13:49:52 -0700 (PDT) Subject: [Dovecot] Public folders and groups In-Reply-To: <50891609.9070709@softjury.de> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> Message-ID: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> No AD doesn't have such a field, but I could use some unused field to get what I want. Let's say set "Attribute1" to "group1". The problem is how to get that info. I guess I have to edit dovecot-ldap.conf and put in user_attrs something like that ",=acl_groups=Attribute1". Any suggestions? ________________________________ From: Jan Phillip Greimann To: b m ; Dovecot Mailing List Sent: Thursday, October 25, 2012 1:35 PM Subject: Re: [Dovecot] Public folders and groups Am 25.10.2012 00:13, schrieb b m: > Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? Here a sentence to this, found in the dovecot wiki. (http://wiki2.dovecot.org/ACL) ACL groups support works by returning a comma-separated acl_groups extra field from userdb, which contains all the groups the user belongs to. It seems to be possible, I had an acl_groups field in my MySQL Database for this, I'am sure it is something like that in an AD too. From stan at hardwarefreak.com Fri Oct 26 00:01:58 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 16:01:58 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <5089A8C6.1090308@hardwarefreak.com> On 10/25/2012 2:13 PM, Bradley Rintoul wrote: > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... There are already a gazillion email services that do mailbox consolidation--the term for what you're attempting. Why are you reinventing the wheel? Must be a family and the dog project... A quick browse of your Wordpress blog confirms such. It also explains why you wrote code to create maildir files from scratch. Only a programmer assumes that the first path to a solution is to write new code. A system administrator on the other hand, most of us here, Google's for suitable FOSS, then looks in his distro repos for it, and builds from source if a package isn't available. We only script when out of Elmer's and duct tape. ;) -- Stan From ben at morrow.me.uk Fri Oct 26 00:15:09 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 25 Oct 2012 22:15:09 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025211509.GE5388@anubis.morrow.me.uk> At 10AM +0300 on 25/10/12 you (Robert JR) wrote: > > Stan, sorry but you didnot understand my question at all, dovecot in > this case is reading the mailbox file while user downloading the mail > and not WRITING. only postfix write when a mail arrives and DOVECOT > only read the mail. And even if both write to the file, I have already > set the locking option of both to FCNTL so no problem should happen. > My question is postfix locking option is = FCNTL, DOTLOCK , and > dovecot = FCNTL, if postfix find a file already FCNTL, why it dotlock > the file 5 secs then remove the dotlock and say resource unaviable? As Stan said earlier, this is a Postfix question. The rule for dotlocking is that you must create the .lock *before* opening the file, in case whoever has it locked will be replacing the file altogether; but with fcntl locking you must acquire the lock *after* opening the file, since that's the way the syscall works. This means that if Postfix is going to use both forms of lock, it has to acquire a dotlock before it can look for a fcntl lock. In other words: the methods in mailbox_delivery_lock are *not* tried in order, because they can't be. Dotlock is always tried first. You should have compatible locking settings for all your programs accessing your mboxes. If Postfix is using dotlock, Dovecot should be using dotlock as well. If you don't have any local programs (mail clients, for instance) which require dotlocks, you should probably change Postfix to just use fcntl locks. Ben From ben at morrow.me.uk Fri Oct 26 00:24:03 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 25 Oct 2012 22:24:03 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50897E38.6070304@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> Message-ID: <20121025212403.GF5388@anubis.morrow.me.uk> At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > > Yes, actually I did, but I missed one part of it because I assumed you > had Dovecot setup properly. > > It doesn't matter if the mbox locks are write or read or both. Locks > are the problem, period, because you have two daemons fighting over the > same files. The fix is absolutely trivial: > > Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > the file locking issue. Thus it also increases throughput as lock > latency is eliminated. Nonsense. deliver and imap are still separate processes accessing the same mbox, so they still need to use locks. The only difference is that since they are both dovecot programs, they will automatically be using the *same* locking strategies, and things will Just Work. Ben From brintoul at sbcglobal.net Fri Oct 26 01:02:05 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 15:02:05 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <5089A8C6.1090308@hardwarefreak.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <5089A8C6.1090308@hardwarefreak.com> Message-ID: <1351202525.21830.YahooMailRC@web184702.mail.ne1.yahoo.com> I didn't actually write code to create Maildir files from scratch, technically. I used 15-20 lines of Python to do that and to actually fetch the mail in addition... Mailbox consolidation isn't the only thing which this project I'm working on requires, but it's a big part of it. :) Anyway, I really appreciate all the advice and help! ________________________________ From: Stan Hoeppner To: dovecot at dovecot.org Sent: Thu, October 25, 2012 2:02:10 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process On 10/25/2012 2:13 PM, Bradley Rintoul wrote: > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their >mail, > > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > > of sorts... There are already a gazillion email services that do mailbox consolidation--the term for what you're attempting. Why are you reinventing the wheel? Must be a family and the dog project... A quick browse of your Wordpress blog confirms such. It also explains why you wrote code to create maildir files from scratch. Only a programmer assumes that the first path to a solution is to write new code. A system administrator on the other hand, most of us here, Google's for suitable FOSS, then looks in his distro repos for it, and builds from source if a package isn't available. We only script when out of Elmer's and duct tape. ;) -- Stan From brintoul at sbcglobal.net Fri Oct 26 01:04:02 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 15:04:02 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <20121025201059.GO3672@harrier.slackbuilds.org> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> <20121025201059.GO3672@harrier.slackbuilds.org> Message-ID: <1351202642.93491.YahooMailRC@web184703.mail.ne1.yahoo.com> Cool. Thanks for the input. Can you tell me briefly why you'd choose getmail? Thanks again. B. RIntoul ________________________________ From: /dev/rob0 To: dovecot at dovecot.org Sent: Thu, October 25, 2012 1:11:13 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process > From: Tom Hendrikx > I'm intrigued by this. Why are you using some self-baked(?) python > script to fetch the mail in stead of using ready-made components > like fetchmail? > > Unless there's a special reason not to, try using the LDA (and > fetchmail/getmail for that matter). > > This sounds exactly what you want: > http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal > On Thu, Oct 25, 2012 at 12:54:43PM -0700, Bradley Rintoul wrote: > I am brand new to this whole "email" thing. I am looking at this > article right now: > http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot I did not see where you described the ultimate goal. That should have been the starting point of this thread. Describe the problem, not how you think it should be solved, because you are new to this, and your ideas might benefit from some scrutiny. Use plain language. I have not reviewed your howto, but personally I would recommend neither fetchmail (I'd choose getmail) nor procmail (other choices exist, depending on what you are trying to do.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Fri Oct 26 01:33:48 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 17:33:48 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> <50899EAB.2030003@hardwarefreak.com> Message-ID: <5089BE4C.4010709@hardwarefreak.com> On 10/25/2012 2:42 PM, Robert JR wrote: > Thanks again Stan, you are very helpfull, i will start learning how to > configure LDA, but hopefully i can also have an answer from Timo about > why this issue happened.. i am sure he is aware of it and can explain it.. Probably not. You describe a dot locking "problem" with Postfix. If you want information about that you need to ask on the Postfix list, not the Dovecot list. Wietse will answer you properly. Give it a subject "dot lock problem". -- Stan From jeff at bubble.org Fri Oct 26 01:57:47 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Thu, 25 Oct 2012 18:57:47 -0400 Subject: [Dovecot] Snarf plugin In-Reply-To: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> References: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> Message-ID: <5089C3EB.1040301@bubble.org> Just thought I'd follow up on my original post, I got the snarf plugin to work properly with some help from Jonathan at PSU (need to give credit where credit is due) For anybody else looking for the configuration here is the relevant output from dovecot -n # 2.1.10: //etc/dovecot/dovecot.conf # OS: Linux 3.6.2-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) ext4 mail_location = mbox:/home/%u/mail:INBOX=/home/%u/mbox mail_plugins = snarf zlib namespace Snarf { hidden = yes list = no location = mbox:/home/%u/mbox:INBOX=/var/spool/mail/%u:INDEX=MEMORY prefix = ~~Snarfbox/ separator = / } namespace default { inbox = yes location = prefix = separator = / } plugin { snarf = ~~Snarfbox/INBOX } What I found was I was specifying the snarf mbox location as ~/mbox, once I changed to /home/%u/mbox things started working. Thanks, Jeff From jeff at bubble.org Fri Oct 26 02:07:04 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Thu, 25 Oct 2012 19:07:04 -0400 Subject: [Dovecot] Changing password for users In-Reply-To: <7a6a7cc64756b895059f7384b256019b@coptics.org> References: <7a6a7cc64756b895059f7384b256019b@coptics.org> Message-ID: <5089C618.6090605@bubble.org> On 10/25/2012 03:21 PM, Mike John wrote: > Hello, > > I am using dovecot (2.0.9) and using virtual users using > > passdb { > args = /etc/dovecot/dovecotpasswd > driver = passwd-file > } > > How can i make my virtual users change their passwords using web > interface ? > > My users already uses squirrelmail to access their mail. is there a > program to add to squirrelmail to add this function to the clients ? > or should i user different separate website for password changing ? > and what program/tool can help me with this ? > > Any ideas is greatly appreciated. > > Mike. Mike, I don't know about forcing users to change their passwords however with Squirrelmail there are several password change plugins available that use "poppasswd" to actually change the password. Of course poppasswd will probably need to be modified to go against your password data base, in my case it simply uses PAM. The version I use is poppassd version 1.8.5. Oh you probably want to restrict access to the port from the local host only since passwords are transmitted in clear text. Jeff From roundcube222 at alaadin.org Fri Oct 26 01:11:14 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Fri, 26 Oct 2012 01:11:14 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025211509.GE5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> Message-ID: <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> On 2012-10-26 00:15, Ben Morrow wrote: > At 10AM +0300 on 25/10/12 you (Robert JR) wrote: > >> Stan, sorry but you didnot understand my question at all, dovecot in >> this case is reading the mailbox file while user downloading the >> mail >> and not WRITING. only postfix write when a mail arrives and DOVECOT >> only read the mail. And even if both write to the file, I have >> already >> set the locking option of both to FCNTL so no problem should happen. >> My >> question is postfix locking option is = FCNTL, DOTLOCK , and dovecot >> = >> FCNTL, if postfix find a file already FCNTL, why it dotlock the file >> 5 >> secs then remove the dotlock and say resource unaviable? > > As Stan said earlier, this is a Postfix question. The rule for > dotlocking is that you must create the .lock *before* opening the > file, > in case whoever has it locked will be replacing the file altogether; > but > with fcntl locking you must acquire the lock *after* opening the > file, > since that's the way the syscall works. This means that if Postfix is > going to use both forms of lock, it has to acquire a dotlock before > it > can look for a fcntl lock. > > In other words: the methods in mailbox_delivery_lock are *not* tried > in > order, because they can't be. Dotlock is always tried first. > > You should have compatible locking settings for all your programs > accessing your mboxes. If Postfix is using dotlock, Dovecot should be > using dotlock as well. If you don't have any local programs (mail > clients, for instance) which require dotlocks, you should probably > change Postfix to just use fcntl locks. > > Ben Thanks Ben for your valuable support and detailed explanation .. but according to your explanation dovecot documentation contains wrong explanation to the dotlock and that's why i was confused. it is written as follow ** If multiple lock methods are used, which is usually the case since dotlocks aren't typically used for read locking, the order in which the locking is done is important. Consider if two programs were running at the same time, both use dotlock and fcntl locking but in different order: Program A: fcntl locks the mbox Program B at the same time: dotlocks the mbox Program A continues: tries to dotlock the mbox, but since it's already dotlocked by B, it starts waiting Program B continues: tries to fcntl lock the mbox, but since it's already fcntl locked by A, it starts waiting Now both of them are waiting for each others locks. Finally after a couple of minutes they time out and fail the operation. ** So this means that the documentation as mentioned above is wrong and dotlock is always first even if the order is different .. Anyway i have sent this question to postfix maillist and i will wait their reply . Thanks again Robert. From mike at alaadin.org Fri Oct 26 01:17:43 2012 From: mike at alaadin.org (Mike John) Date: Fri, 26 Oct 2012 01:17:43 +0300 Subject: [Dovecot] Changing password for users Message-ID: <02a35ba19c559b258dba0de278e31a4d@coptics.org> > Hello, > > I am using dovecot (2.0.9) and using virtual users using > > passdb { > args = /etc/dovecot/dovecotpasswd > driver = passwd-file > } > > How can i make my virtual users change their passwords using web > interface ? > > My users already uses squirrelmail to access their mail. is there a > program to add to squirrelmail to add this function to the clients ? > or should i user different separate website for password changing ? > and what program/tool can help me with this ? > > Any ideas is greatly appreciated. > > Mike. >Mike, >I don't know about forcing users to change their passwords however > with >Squirrelmail there are several password change plugins available that >use "poppasswd" to actually change the password. >Of course poppasswd will probably need to be modified to go against > your >password data base, in my case it simply uses PAM. The version I use > is >poppassd version 1.8.5. >Oh you probably want to restrict access to the port from the local > host >only since passwords are transmitted in clear text. >Jeff I know about poppassd , but it works only for /etc/passwd , /etc/shadow, but my dovecot virtual users password files are in different location and i do not know how to modify poppassd, any idea how can i do that? and is there another way other than poppassd? From stan at hardwarefreak.com Fri Oct 26 03:25:53 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 19:25:53 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025212403.GF5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> Message-ID: <5089D891.9080207@hardwarefreak.com> On 10/25/2012 4:24 PM, Ben Morrow wrote: > At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >> >> Yes, actually I did, but I missed one part of it because I assumed you >> had Dovecot setup properly. >> >> It doesn't matter if the mbox locks are write or read or both. Locks >> are the problem, period, because you have two daemons fighting over the >> same files. The fix is absolutely trivial: >> >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >> the file locking issue. Thus it also increases throughput as lock >> latency is eliminated. > > Nonsense. deliver and imap are still separate processes accessing the > same mbox, so they still need to use locks. The only difference is that > since they are both dovecot programs, they will automatically be using > the *same* locking strategies, and things will Just Work. "Nonsense" implies what I stated was factually incorrect, which is not the case. There's a difference between factual incorrectness and simply staying out of the weeds. If you want to get into the weeds, and have me call you out for "nonsense", LDA/deliver is not a separate UNIX process. The LDA code runs within the imap process for the given user. This is what allows Dovecot to perform 'simultaneous' reads/writes to an mbox file, avoiding filesystem level locking latency. Using filesystem level locking to control read/write access between processes of own's program would be insane on many levels. -- Stan From simon.buongiorno at gmail.com Fri Oct 26 05:59:13 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Thu, 25 Oct 2012 22:59:13 -0400 Subject: [Dovecot] Changing password for users In-Reply-To: <02a35ba19c559b258dba0de278e31a4d@coptics.org> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> Message-ID: On Oct 25, 2012 7:20 PM, "Mike John" wrote: >> >> Hello, >> >> I am using dovecot (2.0.9) and using virtual users using >> >> passdb { >> args = /etc/dovecot/dovecotpasswd >> driver = passwd-file >> } >> >> How can i make my virtual users change their passwords using web >> interface ? >> >> My users already uses squirrelmail to access their mail. is there a >> program to add to squirrelmail to add this function to the clients ? >> or should i user different separate website for password changing ? >> and what program/tool can help me with this ? >> >> Any ideas is greatly appreciated. >> >> Mike. >> Mike, > > >> I don't know about forcing users to change their passwords however with >> Squirrelmail there are several password change plugins available that >> use "poppasswd" to actually change the password. > > >> Of course poppasswd will probably need to be modified to go against your >> password data base, in my case it simply uses PAM. The version I use is >> poppassd version 1.8.5. > > >> Oh you probably want to restrict access to the port from the local host >> only since passwords are transmitted in clear text. > > >> Jeff > > > I know about poppassd , but it works only for /etc/passwd , /etc/shadow, but my dovecot virtual users password files > are in different location and i do not know how to modify poppassd, any idea how can i do that? and is there another way other than poppassd? Horde has a change password module too. And essentially it's trivial to write your own php page to do it. I'll do it if you want to contract it out. Simon From stan at hardwarefreak.com Fri Oct 26 06:12:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 22:12:34 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508983A9.9090605@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> Message-ID: <5089FFA2.9050105@hardwarefreak.com> On 10/25/2012 1:23 PM, Stan Hoeppner wrote: > I forgot to mention one very important feature of Dovecot LDA: > > New messages delivered by Postfix are indexed by LDA as they are written > to the mailbox, flags updated at this time, etc. Thus when a mailbox is > opened in an IMAP MUA, new messages are displayed instantly (I don't use > POP but it's probably faster as well). With your current setup it can > take from a few to many seconds to show new mail, depending on message > count. With LDA new message count seems to have no impact on the speed > of display. Robert JR, you posted relevant information to the Postfix list that you omitted here, or at least I didn't see it. This may directly affect the advice myself and others gave you. You stated there that you're using Dovecot POP only and not IMAP. Given the nature of POP, using LDA may not help much even if it eliminates the filesystem locking contention between processes. I don't know if Dovecot will append an mbox file while reading the entire file. This Timo will have an answer to. Timo may also state, as he has before on this list, that: 1. there are better, faster, POPers available 2. Dovecot is developed primarily as an IMAP server 3. the POP function has received little development for quite some time -- Stan From ben at morrow.me.uk Fri Oct 26 06:54:59 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 04:54:59 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <5089D891.9080207@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> Message-ID: <20121026035458.GG5388@anubis.morrow.me.uk> At 7PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > On 10/25/2012 4:24 PM, Ben Morrow wrote: > > At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > >> > >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > >> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > >> the file locking issue. Thus it also increases throughput as lock > >> latency is eliminated. > > > > Nonsense. deliver and imap are still separate processes accessing the > > same mbox, so they still need to use locks. The only difference is that > > since they are both dovecot programs, they will automatically be using > > the *same* locking strategies, and things will Just Work. > > "Nonsense" implies what I stated was factually incorrect, which is not > the case. There's a difference between factual incorrectness and simply > staying out of the weeds. What you stated was factually incorrect. > If you want to get into the weeds, and have me call you out for > "nonsense", LDA/deliver is not a separate UNIX process. The LDA code > runs within the imap process for the given user. Nonsense. dovecot-lda runs in its own process, and does not involve the imap process in any way. As such it has to do locking. If I have the following in my dovecot.conf: mbox_read_locks = fcntl mbox_write_locks = dotlock fcntl namespace { location = mbox:~/mbox separator = / type = private list = yes prefix = MBOX/ } and I run ktrace dovecot-lda -f mauzo at localhost -m MBOX/foo < mail then the ktrace contains 44973 dovecot-lda CALL access(0x8021f5f68,0x6) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo" 44973 dovecot-lda RET access 0 [Check the mbox exists and is accessible] 44973 dovecot-lda CALL lstat(0x8020196c0,0x7fffffffcb60) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat -1 errno 2 No such file or directory [Look for a .lock file] 44973 dovecot-lda CALL open(0x8020196c0,0xa02, 0x1b6) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET open 9 44973 dovecot-lda CALL write(0x9,0x802019830,0x19) 44973 dovecot-lda RET write 25/0x19 44973 dovecot-lda CALL clock_gettime(0xd,0x7fffffffcbf0) 44973 dovecot-lda RET clock_gettime 0 44973 dovecot-lda CALL fstat(0x9,0x7fffffffcd90) 44973 dovecot-lda RET fstat 0 44973 dovecot-lda CALL close(0x9) 44973 dovecot-lda RET close 0 44973 dovecot-lda CALL lstat(0x8020196c0,0x7fffffffce60) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat 0 [Create a .lock file, and check it was successful] 44973 dovecot-lda CALL open(0x8021f5f68,0x2,0xfac3c0) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo" 44973 dovecot-lda RET open 9 [Open the mbox file itself] 44973 dovecot-lda CALL fcntl(0x9,F_SETLKW,0xffffffffffffcfd0) 44973 dovecot-lda RET fcntl 0 [Set a fcntl lock on the mbox file] 44973 dovecot-lda CALL pread(0,0x802031000,0x1000,0) 44973 dovecot-lda RET pread 43/0x2b 44973 dovecot-lda CALL write(0x9,0x802228000,0xf5) 44973 dovecot-lda RET write 245/0xf5 44973 dovecot-lda CALL fsync(0x9) 44973 dovecot-lda RET fsync 0 [Read from stdin and write to the mbox file] 44973 dovecot-lda CALL lstat(0x802028440,0x7fffffffd010) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat 0 44973 dovecot-lda CALL unlink(0x802028440) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET unlink 0 [Remove the .lock file] 44973 dovecot-lda CALL fcntl(0x9,F_SETLK,0xffffffffffffd160) 44973 dovecot-lda RET fcntl 0 [Clear the fcntl lock] 44973 dovecot-lda CALL close(0x9) 44973 dovecot-lda RET close 0 [Close the mbox file] > This is what allows > Dovecot to perform 'simultaneous' reads/writes to an mbox file, avoiding > filesystem level locking latency. Using filesystem level locking to > control read/write access between processes of own's program would be > insane on many levels. I'm not sure what you mean by 'processes of [one's own] program' but it's extremely common for a process to have to take locks against another copy of itself. All traditional Unix LDAs and MUAs do this; for instance, procmail will take locks in part so that if another instance of procmail is delivering another mail to the same user at the same time the mbox won't end up corrupted. Ben From ben at morrow.me.uk Fri Oct 26 07:16:28 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 05:16:28 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> Message-ID: <20121026041628.GH5388@anubis.morrow.me.uk> At 1AM +0300 on 26/10/12 you (Robert JR) wrote: > On 2012-10-26 00:15, Ben Morrow wrote: > > > As Stan said earlier, this is a Postfix question. The rule for [Looking back at the thread it wasn't Stan, it was Dennis Guhl. Sorry about that.] > > dotlocking is that you must create the .lock *before* opening the > > file, in case whoever has it locked will be replacing the file > > altogether; but with fcntl locking you must acquire the lock *after* > > opening the file, since that's the way the syscall works. This means > > that if Postfix is going to use both forms of lock, it has to > > acquire a dotlock before it can look for a fcntl lock. > > > > In other words: the methods in mailbox_delivery_lock are *not* tried > > in order, because they can't be. Dotlock is always tried first. > > > > You should have compatible locking settings for all your programs > > accessing your mboxes. If Postfix is using dotlock, Dovecot should be > > using dotlock as well. If you don't have any local programs (mail > > clients, for instance) which require dotlocks, you should probably > > change Postfix to just use fcntl locks. > > Thanks Ben for your valuable support and detailed explanation .. but > according to your explanation dovecot documentation contains wrong > explanation to the dotlock and that's why i was confused. > > it is written as follow > > ** > If multiple lock methods are used, which is usually the case since > dotlocks aren't typically used for read locking, the order in which the > locking is done is important. Consider if two programs were running at > the same time, both use dotlock and fcntl locking but in different > order: > > Program A: fcntl locks the mbox > Program B at the same time: dotlocks the mbox > Program A continues: tries to dotlock the mbox, but since it's already > dotlocked by B, it starts waiting > Program B continues: tries to fcntl lock the mbox, but since it's > already fcntl locked by A, it starts waiting > Now both of them are waiting for each others locks. Finally after a > couple of minutes they time out and fail the operation. > ** > > So this means that the documentation as mentioned above is wrong and > dotlock is always first even if the order is different .. I just checked, and you are right: Dovecot *will* use the locking strategies in the order listed. This is different from Postfix, which *will not*, so the 'Postfix' section of the MboxLocking page on the wiki is incorrect. For compatibility with Postfix, dotlock should always be listed first. Note that you will still frequently see Postfix acquiring a dotlock but then failing to acquire a fcntl lock. Dovecot's mbox_read_locks is usually set to just 'fcntl', which means that when Dovecot has an mbox open read-only it won't take a dotlock. This isn't really a problem, assuming you don't use any programs locally which only take dotlocks; but if that is the case you might as well configure everything to just use fcntl locks, and forget dotlocks altogether. Stan's earlier point is fundamentally correct: if you can treat the Dovecot mailstore as a black box, with mail going in through the LDA and LMTP and mail coming out through POP and IMAP, your life will be much easier. Traditional Unix mailbox locking strategies are *completely* insane, and if all you are doing is delivering mail from Postfix and reading it from Dovecot it would be better to avoid them altogether, and switch to dbox if you can. However, if you have any other programs which touch the mail spool (local or NFS mail clients, deliveries through procmail) this may not be possible. Ben From stan at hardwarefreak.com Fri Oct 26 09:24:00 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 01:24:00 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026035458.GG5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> Message-ID: <508A2C80.6010803@hardwarefreak.com> On 10/25/2012 10:54 PM, Ben Morrow wrote: > At 7PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >> On 10/25/2012 4:24 PM, Ben Morrow wrote: >>> At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >>>> >>>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >>>> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >>>> the file locking issue. Thus it also increases throughput as lock >>>> latency is eliminated. >>> >>> Nonsense. deliver and imap are still separate processes accessing the >>> same mbox, so they still need to use locks. The only difference is that >>> since they are both dovecot programs, they will automatically be using >>> the *same* locking strategies, and things will Just Work. >> >> "Nonsense" implies what I stated was factually incorrect, which is not >> the case. There's a difference between factual incorrectness and simply >> staying out of the weeds. > > What you stated was factually incorrect. > >> If you want to get into the weeds, and have me call you out for >> "nonsense", LDA/deliver is not a separate UNIX process. The LDA code >> runs within the imap process for the given user. > > Nonsense. dovecot-lda runs in its own process, and does not involve the > imap process in any way. As such it has to do locking. You apparently know your tools better than I do. Neither ps nor top show a 'dovecot-lda' or similarly named process on my systems. When I send a test message from gmail through Postfix I only see CPU or memory activity in an imap process. When I close the MUA to end the imap processes and then send a test message I don't see any CPU or memory activity in any dovecot processes, only Postfix processes, including local, and spamd. So is devecot-lda running as a sub-process or thread of Postfix' local process? Or is it part of the 'dovecot' process, and the message goes through so quick that top doesn't show any CPU usage by the 'dovecot' process? > If I have the following in my dovecot.conf: ... ... > I'm not sure what you mean by 'processes of [one's own] program' but I.e. Dovecot has its own set of processes, Postfix has its processes, etc. With "one's one processes" I'd think it makes more sense to use IPC and other tricks to accomplish concurrent access to a file rather than filesystem locking features. > it's extremely common for a process to have to take locks against > another copy of itself. All traditional Unix LDAs and MUAs do this; for > instance, procmail will take locks in part so that if another instance > of procmail is delivering another mail to the same user at the same time > the mbox won't end up corrupted. I guess I've given MDAs w/mbox too much credit, without actually looking at the guts. Scalable databases such Oracle, db2, etc, are far more intelligent about this, and can have many thousands of processes reading and writing the same file concurrently, usually via O_DIRECT, not buffered IO, so they have complete control over IO. This is accomplished with a record lock manager and IPC, preventing more than one process from accessing one record concurrently, but allowing massive read/write concurrency to multiple records in a file. I'd think the same concurrency optimization could be done with Dovecot. However, as Timo has pointed out, so few people use mbox these days that he simply hasn't spent much, if any, time optimizing mbox. Implementing some kind of lock manager and client code just for mbox IO concurrency simply wouldn't be worth the time. Unless he's already done something similar with mdbox. If he has, maybe that could be 'ported' to mbox as well. But again, it's probably not worth the effort given the number of mbox users, and the fact that nobody is complaining about mbox performance. I'm certainly not. It works great here. -- Stan From jg at softjury.de Fri Oct 26 09:49:04 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Fri, 26 Oct 2012 08:49:04 +0200 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <508A3260.3060803@softjury.de> I didn't know ADs well, but...can't you simply add the Field? In LDAP it should be possible, if you use MS AD, i dunno. Am 25.10.2012 22:49, schrieb b m:> No AD doesn't have such a field, but I could use some unused field to > get what I want. Let's say set "Attribute1" to "group1". The problem is > how to get that info. I guess I have to edit dovecot-ldap.conf and put > in user_attrs something like that ",=acl_groups=Attribute1". Any > suggestions? From stan at hardwarefreak.com Fri Oct 26 10:05:31 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 02:05:31 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026041628.GH5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> <20121026041628.GH5388@anubis.morrow.me.uk> Message-ID: <508A363B.3040100@hardwarefreak.com> On 10/25/2012 11:16 PM, Ben Morrow wrote: > At 1AM +0300 on 26/10/12 you (Robert JR) wrote: >> On 2012-10-26 00:15, Ben Morrow wrote: >> >>> As Stan said earlier, this is a Postfix question. The rule for > > [Looking back at the thread it wasn't Stan, it was Dennis Guhl. Sorry > about that.] I prodded him a second time, might have been off-list, and he finally posted there. So call it a team effort. ;) Wietse has already replied, and in typical fashion, asked for "concrete" evidence that Postfix was performing fcntl before dotlock, because he obviously knows better than anyone that Postfix applies a dotlock first, which you already explained here. >>> dotlocking is that you must create the .lock *before* opening the >>> file, in case whoever has it locked will be replacing the file >>> altogether; but with fcntl locking you must acquire the lock *after* >>> opening the file, since that's the way the syscall works. This means >>> that if Postfix is going to use both forms of lock, it has to >>> acquire a dotlock before it can look for a fcntl lock. >>> >>> In other words: the methods in mailbox_delivery_lock are *not* tried >>> in order, because they can't be. Dotlock is always tried first. >>> >>> You should have compatible locking settings for all your programs >>> accessing your mboxes. If Postfix is using dotlock, Dovecot should be >>> using dotlock as well. If you don't have any local programs (mail >>> clients, for instance) which require dotlocks, you should probably >>> change Postfix to just use fcntl locks. > but if that is the case you might as well configure everything to just > use fcntl locks, and forget dotlocks altogether. Yep. Postfix can use either or both. And, surprise, recommends using maildir to avoid mailbox locking entirely. > Stan's earlier point is fundamentally correct: if you can treat the > Dovecot mailstore as a black box, with mail going in through the LDA and > LMTP and mail coming out through POP and IMAP, your life will be much > easier. Traditional Unix mailbox locking strategies are *completely* > insane, and if all you are doing is delivering mail from Postfix and > reading it from Dovecot it would be better to avoid them altogether, and > switch to dbox if you can. However, if you have any other programs which > touch the mail spool (local or NFS mail clients, deliveries through > procmail) this may not be possible. And since this is a POP only server, users' MUAs should be deleting after download, so there shouldn't be much mail in these mbox files at any given time, making migration to maildir or dbox relatively simple. When using Dovecot LDA you'll eliminate the filesystem level locking problems with mbox. However, you may still have read/write contention within Dovecot, such as in your 20MB download as new mail arrives example, especially if the new message has an xx MB attachment. I don't believe Dovecot is going to start appending a new message while it's still reading out the existing 20MB of emails. Depending on how long this takes Dovecot may still issue a 4xx to Postfix, which will put the new message in the deferred queue. With maildir or dbox, reading existing mail and writing new messages occurs concurrently, as each message is a different file. -- Stan From ben at morrow.me.uk Fri Oct 26 11:11:20 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 09:11:20 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508A2C80.6010803@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> <508A2C80.6010803@hardwarefreak.com> Message-ID: <20121026081120.GI5388@anubis.morrow.me.uk> At 1AM -0500 on 26/10/12 you (Stan Hoeppner) wrote: > On 10/25/2012 10:54 PM, Ben Morrow wrote: > > > > dovecot-lda runs in its own process, and does not involve the > > imap process in any way. As such it has to do locking. > > You apparently know your tools better than I do. Neither ps nor top > show a 'dovecot-lda' or similarly named process on my systems. When I > send a test message from gmail through Postfix I only see CPU or memory > activity in an imap process. When I close the MUA to end the imap > processes and then send a test message I don't see any CPU or memory > activity in any dovecot processes, only Postfix processes, including > local, and spamd. So is devecot-lda running as a sub-process or thread > of Postfix' local process? Or is it part of the 'dovecot' process, and > the message goes through so quick that top doesn't show any CPU usage by > the 'dovecot' process? Assuming you have mailbox_command = /.../dovecot-lda -a "${RECIPIENT}" or something equivalent in your Postfix configuration, dovecot-lda runs as a subprocess of local(8) under the uid of the delivered-to user. > > If I have the following in my dovecot.conf: > ... > > ... > > > I'm not sure what you mean by 'processes of [one's own] program' but > > I.e. Dovecot has its own set of processes, Postfix has its processes, > etc. With "one's one processes" I'd think it makes more sense to use > IPC and other tricks to accomplish concurrent access to a file rather > than filesystem locking features. Filesystem locking, at least if NFS is not involved, is not that expensive. Successfully acquiring a flock or fcntl lock takes only a single syscall which doesn't have to touch the disk, and any form of IPC is going to need to do that. (Even something like a shared memory region will need a mutex for synchronisation, and acquiring the mutex has to go through the kernel.) Dotlocking *is* expensive, because acquiring a dotlock is a complicated process requiring lots of syscalls, some of which have to write to disk; and any scheme involving acquiring several locks on the same file is going to be more so, especially if you can end up getting the first lock but finding you can't get the second, so then you have to undo the first and try again. More importantly, the biggest problem with mbox as a mailbox format is that any access at all has to lock the whole mailbox. If the LDA is trying to deliver a new message at the same time as an IMAP user is fetching a completely different message, or if two instances of the LDA are trying to deliver at the same time, they will be competing for the same lock even though they don't really need to be. A file-per-message format like Maildir avoids this, to the point of being mostly lockless, but that brings its own efficiency problems; the point of dbox is to find the compromise between these positions that works best. > > it's extremely common for a process to have to take locks against > > another copy of itself. All traditional Unix LDAs and MUAs do this; for > > instance, procmail will take locks in part so that if another instance > > of procmail is delivering another mail to the same user at the same time > > the mbox won't end up corrupted. > > I guess I've given MDAs w/mbox too much credit, without actually looking > at the guts. I wouldn't look too hard at the details of the various ways there are of locking and parsing mbox files, or the ways in which they can go wrong. It's enough to make anyone swear off email for life :). > Scalable databases such Oracle, db2, etc, are far more > intelligent about this, and can have many thousands of processes reading > and writing the same file concurrently, usually via O_DIRECT, not > buffered IO, so they have complete control over IO. This is > accomplished with a record lock manager and IPC, preventing more than > one process from accessing one record concurrently, but allowing massive > read/write concurrency to multiple records in a file. I'd think the > same concurrency optimization could be done with Dovecot. > > However, as Timo has pointed out, so few people use mbox these days that > he simply hasn't spent much, if any, time optimizing mbox. Implementing > some kind of lock manager and client code just for mbox IO concurrency > simply wouldn't be worth the time. Unless he's already done something > similar with mdbox. If he has, maybe that could be 'ported' to mbox as > well. But again, it's probably not worth the effort given the number of > mbox users, and the fact that nobody is complaining about mbox > performance. I'm certainly not. It works great here. The only reason for using mbox is for compatibility with other systems which use mbox, which means you have to do the locking the same way as they do (assuming you can work out what that is). If you're going to change the locking rules you might as well change the file format at the same time, both to remove the insanity and to make it actually suitable for use as an IMAP mailstore. That's what Timo did with dbox, so if you've got your systems to the point where nothing but Dovecot touches the mail files you should seriously consider switching. Ben From r.ordinas at math.univ-paris-diderot.fr Fri Oct 26 11:51:52 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Fri, 26 Oct 2012 10:51:52 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121025140852.GA15639@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> Message-ID: <508A4F28.80606@math.univ-paris-diderot.fr> Hi, Here's the doveconf -n output : # doveconf -n # 2.0.14: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.1-RELEASE-p5 amd64 auth_mechanisms = plain login auth_username_format = %Lu auth_worker_max_count = 90 default_process_limit = 1024 first_valid_gid = 1500 first_valid_uid = 1500 hostname = mailhost.mydomain.tld last_valid_gid = 1500 last_valid_uid = 1500 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * mail_gid = 1500 mail_location = maildir:~/Maildir mail_plugins = acl quota mail_log notify mail_privileged_group = mail mail_uid = 1500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/usr/local/etc/dovecot-acls:cache_secs=300 autocreate = Sent autocreate1 = Trash autocreate2 = Drafts autocreate3 = Spam autocreate4 = Faux-positif autosubscribe = Sent autosubscribe1 = Trash autosubscribe2 = Drafts autosubscribe3 = Spam autosubscribe4 = Faux-positif autosubscribe5 = INBOX mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_rule = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 quota_warning2 = storage=80%% quota-warning 80 sieve = ~/.dovecot.sieve sieve_before = /usr/local/lib/dovecot/sieve/backup-all.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at mydomain.tld protocols = imap lmtp sieve quota_full_tempfail = yes service anvil { client_limit = 3500 } service auth-worker { user = $default_internal_user } service auth { client_limit = 5500 unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = doveauth } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 172.0.0.1 port = 2525 } } service managesieve-login { inet_listener sieve_deprecated { port = 2000 } process_limit = 1024 } service managesieve { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } shutdown_clients = no ssl = required ssl_ca = On Thu, Oct 25, 2012 at 03:09:47PM +0200, Raphael Ordinas wrote: >> When sending mail to MTA (in case of sieve filter forwarding for >> example), dovecot pass a RCPT TO command just after the EHLO. He's >> missing the MAIL FROM command. >> Therefore, my MTA show me a warning like this : "improper command >> pipelining after EHLO". > Works for me with latest and greatest although I'm not using the > 'submission_host' option but pure LMTP Unix socket: > > [...] > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > > Best to show your 'doveconf -n' for more thoughts. > > Regards > Thomas From ben at morrow.me.uk Fri Oct 26 11:54:56 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 09:54:56 +0100 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <20121026085456.GJ5388@anubis.morrow.me.uk> At 1PM -0700 on 25/10/12 b m wrote: > From: Jan Phillip Greimann >> Am 25.10.2012 00:13, schrieb b m: >> >> > Currently I have dovecot working with Active Directory >> > authentication and public folders with acl. In acl I have the users >> > I want to access the public folders. It'll be easier for me to use >> > one group instead of 50 users but I can't get it to work. From where >> > does dovecot get the "group" attribute for a user? Can it read the >> > groups that a user belongs from AD? >> >> ACL groups support works by returning a comma-separated acl_groups >> extra field from userdb, which contains all the groups the user >> belongs to. >> >> It seems to be possible, I had an acl_groups field in my MySQL >> Database for this, I'am sure it is something like that in an AD too. > > No AD doesn't have such a field, but I could use some unused field to > get what I want. Let's say set "Attribute1" to "group1". The problem > is how to get that info. I guess I have to edit dovecot-ldap.conf and > put in user_attrs something like that ",=acl_groups=Attribute1". Any > suggestions? That's the wrong way around. Assuming you created an 'imapGroups' attribute containing a comma-separated list of IMAP groups, you would want to add 'imapGroups=acl_groups' to user_attrs. Alternatively, if you don't want to duplicate the information in the LDAP directory, you can use post-login scripting to set up the groups list (see http://wiki2.dovecot.org/PostLoginScripting). If you have your system set up with nss_ldap or winbind so that AD users show up as system users with their proper groups, the example on the wiki using the 'groups' command will work. Otherwise, you can pull the information directly from LDAP, something like #!/bin/sh do_ldap () { /usr/local/bin/ldapsearch -h \ "(&(objectClass=$1)($2))" $3 \ | sed -ne"s/^$3: //p" } user_dn="$(do_ldap User "sAMAccountName=$USER" dn)" ACL_GROUPS="$(do_ldap Group "member=$user_dn" cn | paste -sd, -)" export ACL_GROUPS export USERDB_KEYS="$USERDB_KEYS acl_groups" exec "$@" Obviously you will need to adjust the path and connection parameters for ldapsearch to suit your environment; also, I don't use AD, so you may need to adjust the LDAP search. (If you prefer it might be easier to do this in Perl or Python or something rather than shell.) Ben From tlx at leuxner.net Fri Oct 26 12:00:12 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 26 Oct 2012 11:00:12 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <508A4F28.80606@math.univ-paris-diderot.fr> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> Message-ID: <20121026090012.GA31929@nihlus.leuxner.net> On Fri, Oct 26, 2012 at 10:51:52AM +0200, Raphael Ordinas wrote: > service lmtp { > inet_listener lmtp { > address = 172.0.0.1 > port = 2525 > } > } Right, so you are using network sockets with LMTP. Probably does not answer the question why it is not working with the 'submission_host', but is there a reason why the redirects are not reinjected this way? > submission_host = smtp.mydomain.tld Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From busseniu at in.tum.de Fri Oct 26 12:17:44 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Fri, 26 Oct 2012 11:17:44 +0200 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 In-Reply-To: <5085593D.3080403@in.tum.de> References: <5085593D.3080403@in.tum.de> Message-ID: <508A5538.8080604@in.tum.de> Hi, On 22.10.2012 16:33, Christoph Bu?enius wrote: > . list "" INBOX.shared.%.% > > Dovecot 2.1.10 does not list any folders in response to this command. I hope this helps: I bisected this bug and found it was introduced with this changeset: http://hg.dovecot.org/dovecot-2.1/rev/a41f64348d0d changeset: 14453:a41f64348d0d user: Timo Sirainen date: Fri Apr 20 15:18:14 2012 +0300 files: src/lib-storage/list/mailbox-list-fs-iter.c description: layout=fs: Don't assume '/' hierarchy separator when finding mailbox roots. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From tlx at leuxner.net Fri Oct 26 12:31:34 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 26 Oct 2012 11:31:34 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121026090012.GA31929@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> <20121026090012.GA31929@nihlus.leuxner.net> Message-ID: <20121026093134.GB31929@nihlus.leuxner.net> On Fri, Oct 26, 2012 at 11:00:12AM +0200, Thomas Leuxner wrote: > submission_host = smtp.mydomain.tld On second thought, above probably overrides this: # doveconf -a | grep sendmail sendmail_path = /usr/sbin/sendmail ...which may be the culprit. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From r.ordinas at math.univ-paris-diderot.fr Fri Oct 26 12:59:06 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Fri, 26 Oct 2012 11:59:06 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121026090012.GA31929@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> <20121026090012.GA31929@nihlus.leuxner.net> Message-ID: <508A5EEA.2000705@math.univ-paris-diderot.fr> Actually, LMTP inet listener is only used for delivery purpose. I separated the MTA and the MDA on distinct hosts. Incomming mails are received by the MTA which proceed to some check (anti-virus, spams, and aliases) and transport them to the MDA with LMTP. Maybe I misunderstood something, but i don't see why LMTP is involve in a sieve forwarding process (or stuff like non delivery mail return) . According to comments in the "15-lda.conf" file : # Binary to use for sending mails. #sendmail_path = /usr/sbin/sendmail # If non-empty, send mails via this SMTP host[:port] instead of sendmail. submission_host = smtp.mydomain.tld If you don't use the 'submission_host' option, dovecot will forward mail with '/usr/sbin/sendmail' binary which use the forwarders you tell it to use, am i right ? Regards, Raphael Le 26/10/2012 11:00, Thomas Leuxner a ?crit : > On Fri, Oct 26, 2012 at 10:51:52AM +0200, Raphael Ordinas wrote: > >> service lmtp { >> inet_listener lmtp { >> address = 172.0.0.1 >> port = 2525 >> } >> } > Right, so you are using network sockets with LMTP. Probably does not > answer the question why it is not working with the 'submission_host', > but is there a reason why the redirects are not reinjected this way? > >> submission_host = smtp.mydomain.tld > Regards > Thomas From tss at iki.fi Fri Oct 26 13:07:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 26 Oct 2012 13:07:49 +0300 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 In-Reply-To: <508A5538.8080604@in.tum.de> References: <5085593D.3080403@in.tum.de> <508A5538.8080604@in.tum.de> Message-ID: On 26.10.2012, at 12.17, Christoph Bu?enius wrote: > On 22.10.2012 16:33, Christoph Bu?enius wrote: >> . list "" INBOX.shared.%.% >> >> Dovecot 2.1.10 does not list any folders in response to this command. > > I hope this helps: I bisected this bug and found it was introduced with this changeset: > > http://hg.dovecot.org/dovecot-2.1/rev/a41f64348d0d I couldn't reproduce this exactly and I don't see how a41f64348d0d makes any difference .. but I did find another way to reproduce at least a similar bug. Maybe this fixes your problem too? http://hg.dovecot.org/dovecot-2.1/rev/22875bcaa952 From fabio.ferrari at unimore.it Fri Oct 26 13:24:42 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Fri, 26 Oct 2012 12:24:42 +0200 Subject: [Dovecot] Dovecot stops to work - anvil problem Message-ID: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> Hi all, we have a problem about anvil, it seems that when we have a high load the dovecot stops to work. Sometimes it is sufficient to make a dovecot reload, but sometimes we have to restart it. These are the lines related to anvil in the dovecot.log: [root at secchia ~]# grep anvil /var/log/dovecot.log | more Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files Oct 26 11:14:32 imap-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:32 imap-login: Fatal: Couldn't connect to anvil Oct 26 11:14:33 pop3-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:33 pop3-login: Fatal: Couldn't connect to anvil [...] (many lines like these) Oct 26 12:01:10 pop3-login: Fatal: Couldn't connect to anvil Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused And this is the output of the doveconf -n: [root at secchia ~]# doveconf -n # 2.0.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.11.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.8 (Tikanga) xfs auth_cache_size = 1024 auth_cache_ttl = 21600 s auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no hostname = mail.unimore.it info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/cl/mail/vhosts/sms.unimo.it/%Ln/Maildir mail_plugins = $mail_plugins quota mailbox_idle_check_interval = 60 s mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot.masterusers driver = passwd-file master = yes } passdb { args = dovecot driver = pam } plugin { quota = maildir:User quota quota_exceeded_message = Quota exceeded (mailbox is full) quota_rule = *:storage=200MB quota_rule2 = *:messages=100000 quota_rule3 = INBOX.Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = messages=95%% quota-warning 95 %u quota_warning4 = messages=80%% quota-warning 80 %u setting_name = quota } postmaster_address = postmaster at unimore.it quota_full_tempfail = yes service anvil { client_limit = 199999 process_limit = 199999 } service auth { client_limit = 14500 unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } process_limit = 5000 } service imap { process_limit = 5000 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 } } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail user = vmail } user = dovecot } ssl_ca = References: <5085593D.3080403@in.tum.de> <508A5538.8080604@in.tum.de> Message-ID: <508A668D.4070505@in.tum.de> Hello Timo, On 26.10.2012 12:07, Timo Sirainen wrote: > but I did find another way to reproduce at least a similar bug. Maybe this fixes your problem too? http://hg.dovecot.org/dovecot-2.1/rev/22875bcaa952 That does fix the problem, thank you! Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From dale.gallagher at gmail.com Fri Oct 26 14:27:00 2012 From: dale.gallagher at gmail.com (Dale Gallagher) Date: Fri, 26 Oct 2012 13:27:00 +0200 Subject: [Dovecot] dovecot-lda delivery to Maildir/cur as 'seen'? Message-ID: Hi I've added a server-side feature where authenticated customers sending through our SMTP server have their outbound mail copied to their Sent folder (like Gmail does). The delivery script called by qmail calls dovecot-lda to deliver it to the user's Sent folder. The problem now, is that the Sent folder shows the mail as unread, which MUAs flag (and notify, in the case of some). I've searched the docs and mailing list, but can't find an option to tell dovecot-lda to mark the mail being delivered, as seen/read. If I've missed something, please let me know. If not, then I think it might be a good idea to add a feature to dovecot-lda permitting one to specify delivery to the ./cur subfolder of a Maildir, instead of ./new. Thanks From tony.blue.mailinglist at gmx.de Fri Oct 26 14:44:55 2012 From: tony.blue.mailinglist at gmx.de (tony.blue.mailinglist at gmx.de) Date: Fri, 26 Oct 2012 13:44:55 +0200 Subject: [Dovecot] dovecot lda - Permission denied Message-ID: <20121026114455.30440@gmx.net> Hallo, please excuse my bad english. But I am not a native speaker. I changed my cyrus to dovecot (alltogehter: fetchmail - procmail - exim4 - dovecot). But I get (I think from /usr/lib/dovecot/deliver) the following error-message in my syslog: ... Oct 25 23:37:13 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(andy) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) ... Oct 25 23:37:14 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=500(tony) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) ... Dovecot is installed as !include auth-passwdfile.conf.ext. For all users there is a entry in der /etc/dovecot/users. Usaly the user rights are set to 600. I tryed 755, but I get the same errormessage. ... service auth { unix_listener auth-userdb { mode = 0755 user = mailstore group = mailstore } ... If I try "ls /var/run/dovecot/auth-userdb -la" - i get: srwxr-xr-x 1 mailstore mailstore 0 Okt 25 23:36 /var/run/dovecot/auth-userdb How can I solve this problem? Tony From CMarcus at Media-Brokers.com Fri Oct 26 16:37:59 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 26 Oct 2012 09:37:59 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5088B691.7030100@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> <5088B691.7030100@hardwarefreak.com> Message-ID: <508A9237.7080903@Media-Brokers.com> On 2012-10-24 11:48 PM, Stan Hoeppner wrote: > Changing the process priority would not help. Indexing a large mailbox > is an IO bound, not a compute bound, operation. With Linux, changing > from the CFQ to deadline scheduler may help some with low > responsiveness. But the only real solution for such a case where iowait > is bringing the system to its knees is to acquire storage with far > greater IOPS and concurrent IO capability. I.e. a server. Ok, I get it, thanks for elaborating Stan... -- Best regards, Charles From dg at dguhl.org Fri Oct 26 17:01:41 2012 From: dg at dguhl.org (Dennis Guhl) Date: Fri, 26 Oct 2012 16:01:41 +0200 Subject: [Dovecot] dovecot-lda delivery to Maildir/cur as 'seen'? In-Reply-To: References: Message-ID: <20121026140141.GA6769@PC211.ikt.de> On Fri, Oct 26, 2012 at 01:27:00PM +0200, Dale Gallagher wrote: > Hi [..] > The problem now, is that the Sent folder shows the mail as unread, > which MUAs flag (and notify, in the case of some). I've searched the Use Sieve [1] with Imap4flags (RFC 5232) to mark the email as read. Dennis [1] http://wiki2.dovecot.org/Pigeonhole/Sieve [..] From listen at mjh.name Fri Oct 26 21:28:51 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:28:51 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087D457.6040205@sys4.de> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087D457.6040205@sys4.de> Message-ID: <20121026202851.dc0abb3d5e4a4dd5c32d2d6c@mjh.name> On Wed, 24 Oct 2012 13:43:19 +0200 Robert Schetterer wrote: > Am 24.10.2012 13:28, schrieb Milan Holz?pfel: > > The whole mdbox is 6.6 GiB large and I guess that it contains about > > 300k-600k messages. It's an archive of public mailing lists, so I could > > give access to the files. > > > > Can anybody say something about this? May the mdbox be repaired? > > perhaps this helps > > http://wiki2.dovecot.org/Tools/Doveadm/ForceResync > > however upgrading to dovecot latest might be a good idea I tried this command, but all it will do is the "rebuilding indexes" thing that Dovecot's deliver and imapd will also do. (As I mentioned, this fails.) I haven't tried a more recent version of Dovecot so far. Regards, Milan Holz?pfel -- Milan Holz?pfel From listen at mjh.name Fri Oct 26 21:29:15 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:29:15 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087F4B4.2060107@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> Message-ID: <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> On Wed, 24 Oct 2012 09:01:24 -0500 Stan Hoeppner wrote: > On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: > > > I have a problem with an incosistent mdbox: > ... > > four hours after the problem initially appeared, I did a hard reset of > > the system because it was unresponsive. > ... > > Can anybody say something about this? May the mdbox be repaired? > > If the box is truly unresponsive, i.e. hard locked, then the corrupted > indexes are only a symptom of the underlying problem, which is unrelated > to Dovecot, UNLESS, the lack of responsiveness was due to massive disk > access, which will occur when rebuilding indexes on a 6.6GB mailbox. > You need to know the difference so we have accurate information to > troubleshoot with. Thanks for your suggestion. I wasn't looking for a solution for the unresponsiveness, but I failed to make that clear. I was not patient enough to debug the unresponsiveness issue. The box was not hard locked, but any command took very look if it would at all complete. I think that it could be massive swapping, but I wouldn't expect Dovecot to be the cause. After the reboot, Dovecot would happily re-execute the failing index rebuild on each new incoming message, which suggests that Dovecot wasn't the cause for the unresponsiveness. > If the there's a kernel or hardware problem, you should see related > errors in dmesg. Please share those. The kernel had messages like INFO: task cron:2799 blocked for more than 120 seconds. in the dmesg. But again, I didn't mean to ask for a solution to this problem. Regards, Milan Holz?pfel -- Milan Holz?pfel From listen at mjh.name Fri Oct 26 21:30:24 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:30:24 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> On Wed, 24 Oct 2012 13:28:11 +0200 Milan Holz?pfel wrote: > I have a problem with an incosistent mdbox: > [...] > The problem appeared out of nowhere. [...] That's just wrong. Two minutes before the corruption occured for the first time, the machine was booted after power-off without prior shutdown. I didn't notice this until now, sorry for this. The mailbox is on XFS. As far as I remember, XFS in known for leaving NULL bytes at the end of files after a system reset. At least, I found 72 bytes of NULL in a plain text log file on XFS after such an event. Do you think this may be the source of the index corruption? Do you have any other suggestions for recovering the mailbox? Regards, Milan Holz?pfel -- Milan Holz?pfel From fxmulder at gmail.com Fri Oct 26 22:13:33 2012 From: fxmulder at gmail.com (James Devine) Date: Fri, 26 Oct 2012 13:13:33 -0600 Subject: [Dovecot] Overlapping userdb/passdbs Message-ID: I have an ldap server for which each entry includes the email address and the username portion of the email address for authentication. Authentication works by username if the username is unique among all the entries. I need to now add some users which must authenticate even if the username is not unique. I figured one way to do this would be to add a second user/pass db which puts further restrictions on the ldap query to make it unique for those users. This doesn't seem to work however as if the user is found in the first ldap query but the password does not match it does not try the second. I would use the password as part of the query but this setup requires me to allow the client to hash the password. Is there a way to do this? Or maybe I am approaching the problem wrong. From mike at alaadin.org Fri Oct 26 21:47:44 2012 From: mike at alaadin.org (Mike John) Date: Fri, 26 Oct 2012 21:47:44 +0300 Subject: [Dovecot] Changing password for users In-Reply-To: <02a35ba19c559b258dba0de278e31a4d@coptics.org> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> Message-ID: <7827e2e2d9aa524945d00575c3366400@coptics.org> On 2012-10-26 01:17, Mike John wrote: >> Hello, I am using dovecot (2.0.9) and using virtual users using >> passdb >> { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i >> make my virtual users change their passwords using web interface ? >> My >> users already uses squirrelmail to access their mail. is there a >> program to add to squirrelmail to add this function to the clients ? >> or >> should i user different separate website for password changing ? and >> what program/tool can help me with this ? Any ideas is greatly >> appreciated. Mike. Mike, > >> I don't know about forcing users to change their passwords however >> with >> Squirrelmail there are several password change plugins available >> that >> use "poppasswd" to actually c> ssword. Of course poppasswd will > probably need to be modified to go >> against your password data base, in my case it simply uses PAM. The >> version I> sion 1.8.5. Oh you probably want to restrict access to >> the > port from >> the local host only since pas > ansmitted in clear > >> quot > e>Jeff > > I know about poppassd , but it works only for /etc/passwd , > /etc/shadow, but my dovecot virtual users password files > are in different location and i do not know how to modify poppassd, > any > idea how can i do that? and is there another way other than poppassd? i have googled every where, i can not find how to modify poppassd to modify virtual users passwords at /etc/dovecot/passwords , Is there any other way ? i am sure that some one in this mailing list have virtual users and uses modified poppassd or other utils so that his clients can change their password From dave.mehler at gmail.com Fri Oct 26 23:34:46 2012 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 26 Oct 2012 16:34:46 -0400 Subject: [Dovecot] public mailbox not showing up in web client Message-ID: Hello, I'm trying to set up a public mailbox where users can receive notifications out of. I'm not getting any errors from Dovecot 2.1, but nothing is showing up in my user's web clients. In each /home/vmail/public/mailbox folder right now I just have one called testbox I have a dovecot-acl file with: user=testuser1 lr user=user1 lr etc. I'd appreciate any suggestions. Thanks. Dave. # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 hostname = xxx last_valid_gid = 5000 last_valid_uid = 5000 log_path = /var/log/dovecot.error mail_gid = vmail mail_home = /home/vmail/%d/%n/home mail_location = maildir:/home/vmail/%d/%n:LAYOUT=fs mail_plugins = " acl quota zlib" mail_uid = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = yes location = maildir:/home/vmail/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace inbox { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile autocreate = Spam autosubscribe = Spam quota = dict:User quota::proxy::quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } postmaster_address = postmaster at xxx protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = xxx xxx ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_cert = References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> <7827e2e2d9aa524945d00575c3366400@coptics.org> Message-ID: <508AFACD.8050807@whyscream.net> On 26-10-12 20:47, Mike John wrote: > On 2012-10-26 01:17, Mike John wrote: > >>> Hello, I am using dovecot (2.0.9) and using virtual users using passdb >>> { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i >>> make my virtual users change their passwords using web interface ? My >>> users already uses squirrelmail to access their mail. is there a >>> program to add to squirrelmail to add this function to the clients ? or >>> should i user different separate website for password changing ? and >>> what program/tool can help me with this ? Any ideas is greatly >>> appreciated. Mike. Mike, >> >>> I don't know about forcing users to change their passwords however with >>> Squirrelmail there are several password change plugins available that >>> use "poppasswd" to actually c> ssword. Of course poppasswd will >> probably need to be modified to go >>> against your password data base, in my case it simply uses PAM. The >>> version I> sion 1.8.5. Oh you probably want to restrict access to the >> port from >>> the local host only since pas >> ansmitted in clear >> >>> quot >> e>Jeff >> >> I know about poppassd , but it works only for /etc/passwd , >> /etc/shadow, but my dovecot virtual users password files >> are in different location and i do not know how to modify poppassd, any >> idea how can i do that? and is there another way other than poppassd? > > i have googled every where, i can not find how to modify poppassd to > modify virtual users passwords at /etc/dovecot/passwords > , Is there any other way ? i am sure that some one in this mailing list > have virtual users and uses modified poppassd or other utils so that his > clients can change their password Using a database for managing virtual users seems overkill, until you run into issues like this. I have a postgres backend for 20ish users, and I can plugin everything I want. Postfixadmin works geat, and there are many password plugins for squirrelmail/roundcube/etc that work with such a database. Disclaimer: I tried the file-based approach too, but kept building kludges for things that were a lot simpler with a database. In the end, I joined the dark side. -- Tom From jtam.home at gmail.com Sat Oct 27 01:19:31 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 26 Oct 2012 15:19:31 -0700 (PDT) Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: > From: Mike John > >> I know about poppassd , but it works only for /etc/passwd , >> /etc/shadow, but my dovecot virtual users password files >> are in different location and i do not know how to modify poppassd, >> any idea how can i do that? I downloaded and examined it; it's just a wrapper for /usr/bin/passwd, and there doesn't seem an easy way to modify it to use something other than the system password file. Maybe replace "/usr/bin/passwd" with htpasswd? > and is there another way other than poppassd? Write your own PHP script -- it couldn't be more than a few dozen lines of code for a working skeleton. Or Google "php change password htpasswd". Joseph Tam From ben at morrow.me.uk Sat Oct 27 02:09:11 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Sat, 27 Oct 2012 00:09:11 +0100 Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: <20121026230910.GK5388@anubis.morrow.me.uk> At 3PM -0700 on 26/10/12 you (Joseph Tam) wrote: > > > From: Mike John > > > >> I know about poppassd , but it works only for /etc/passwd , > >> /etc/shadow, but my dovecot virtual users password files > >> are in different location and i do not know how to modify poppassd, > >> any idea how can i do that? > > I downloaded and examined it; it's just a wrapper for /usr/bin/passwd, > and there doesn't seem an easy way to modify it to use something other > than the system password file. > > Maybe replace "/usr/bin/passwd" with htpasswd? Try pam_pwdfile with poppwd or some other poppassd that supports PAM. > > and is there another way other than poppassd? > > Write your own PHP script -- it couldn't be more than a few dozen lines > of code for a working skeleton. Or Google "php change password htpasswd". It's not as simple as you seem to think. Quite apart from getting the password-changing itself right (have you considered what happens when two users change their passwords at the same time? when Dovecot tries to read the password file at the same time as you are changing it? when the system crashes when you are halfway through rewriting the password file?), you really shouldn't be running PHP as a user with write access to a password file (even a virtual password file) in any case. Ben From rob0 at gmx.co.uk Sat Oct 27 03:26:46 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Fri, 26 Oct 2012 19:26:46 -0500 Subject: [Dovecot] Changing password for users In-Reply-To: <508AFACD.8050807@whyscream.net> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> <7827e2e2d9aa524945d00575c3366400@coptics.org> <508AFACD.8050807@whyscream.net> Message-ID: <20121027002646.GS3672@harrier.slackbuilds.org> On Fri, Oct 26, 2012 at 11:04:13PM +0200, Tom Hendrikx wrote: > Using a database for managing virtual users seems overkill, > until you run into issues like this. > > I have a postgres backend for 20ish users, and I can plugin > everything I want. Postfixadmin works geat, and there are many > password plugins for squirrelmail/roundcube/etc that work with > such a database. > > Disclaimer: I tried the file-based approach too, but kept > building kludges for things that were a lot simpler with a > database. In the end, I joined the dark side. SQLite gives me the best of both worlds: file-based stability with SQL flexibility and easy backups. There is no Postfixadmin-type solution out there yet, but if you're fine with sqlite3(1) in the console, you won't miss it. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Sat Oct 27 03:49:32 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 19:49:32 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026081120.GI5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> <508A2C80.6010803@hardwarefreak.com> <20121026081120.GI5388@anubis.morrow.me.uk> Message-ID: <508B2F9C.2050706@hardwarefreak.com> You are a well of accessible knowledge Ben. (How have I missed your posts in the past?) On 10/26/2012 3:11 AM, Ben Morrow wrote: > Assuming you have > > mailbox_command = /.../dovecot-lda -a "${RECIPIENT}" I'm setup for system users so it's a simpler, but yes. > or something equivalent in your Postfix configuration, dovecot-lda runs > as a subprocess of local(8) under the uid of the delivered-to user. Of course that makes sense given Postfix is doing the calling. I would have assumed this but my feeble use of tools wasn't showing anything. > Filesystem locking, at least if NFS is not involved, is not that > expensive. Successfully acquiring a flock or fcntl lock takes only a > single syscall which doesn't have to touch the disk, and any form of IPC > is going to need to do that. (Even something like a shared memory region > will need a mutex for synchronisation, and acquiring the mutex has to go > through the kernel.) Thanks for this. I was under the assumption flock/fcntl were more expensive than they are. Probably because all I'd read about them was in relation to NFS (which I don't use, but I read alot like many do). > Dotlocking *is* expensive, because acquiring a dotlock is a complicated > process requiring lots of syscalls, some of which have to write to disk; > and any scheme involving acquiring several locks on the same file is > going to be more so, especially if you can end up getting the first lock > but finding you can't get the second, so then you have to undo the first > and try again. Yeah, I knew dotlocks were the worst due to disk writes, but didn't know the other details. > More importantly, the biggest problem with mbox as a mailbox format is > that any access at all has to lock the whole mailbox. If the LDA is > trying to deliver a new message at the same time as an IMAP user is > fetching a completely different message, or if two instances of the LDA > are trying to deliver at the same time, they will be competing for the > same lock even though they don't really need to be. A file-per-message > format like Maildir avoids this, to the point of being mostly lockless, > but that brings its own efficiency problems; the point of dbox is to > find the compromise between these positions that works best. mbox locking hasn't been problem here as I split the INBOX from the user mailboxes containing IMAP folders (mbox files). We make heavy use of sieve scripts to sort on delivery, so there's not much concurrent access to any one mbox file. The efficiency issue is why I chose mbox over maildir. Users here keep a lot of (list) mail and FTS often. The load on the spindles with maildir is simply too great and would bog down all users. The IOPS benefit of mbox in this scenario outweighs any locking issues. > I wouldn't look too hard at the details of the various ways there are of > locking and parsing mbox files, or the ways in which they can go wrong. > It's enough to make anyone swear off email for life :). Heheh. > The only reason for using mbox is for compatibility with other systems > which use mbox, Not necessarily true. See above. I'm sure I'm not the only one using mbox for this reason. Dovecot is my only app hitting these mbox files. > which means you have to do the locking the same way as > they do (assuming you can work out what that is). If you're going to > change the locking rules you might as well change the file format at the > same time, both to remove the insanity and to make it actually suitable > for use as an IMAP mailstore. That's what Timo did with dbox, so if > you've got your systems to the point where nothing but Dovecot touches > the mail files you should seriously consider switching. If/when I do switch mailbox formats it'll be to mdbox so FTS doesn't drop a big hammer on the spindles. Thanks for the informative discussion Ben. -- Stan From stan at hardwarefreak.com Sat Oct 27 04:45:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 20:45:34 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> Message-ID: <508B3CBE.1000004@hardwarefreak.com> On 10/26/2012 1:29 PM, Milan Holz?pfel wrote: > On Wed, 24 Oct 2012 09:01:24 -0500 > Stan Hoeppner wrote: > >> On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: >> >>> I have a problem with an incosistent mdbox: >> ... >>> four hours after the problem initially appeared, I did a hard reset of >>> the system because it was unresponsive. >> ... >>> Can anybody say something about this? May the mdbox be repaired? >> >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Thanks for your suggestion. I wasn't looking for a solution for the > unresponsiveness, but I failed to make that clear. It's likely all related. If you have already, or will continue to, hard reset the box, you will lose inflight data in the buffer cache, which may very likely corrupt your mdbox files and/or indexes. I'm a bit shocked you'd hard reset a *slow* responding server. Especially one that appears to be unresponsive due to massive disk IO. That's a recipe for disaster... > I was not patient enough to debug the unresponsiveness issue. The box > was not hard locked, but any command took very look if it would at all > complete. I think that it could be massive swapping, but I wouldn't > expect Dovecot to be the cause. This leads me to believe your filesystem root, swap partition, and Dovecot mailbox storage are all on the same disk, or small RAID set. Is this correct? > After the reboot, Dovecot would happily re-execute the failing index > rebuild on each new incoming message, which suggests that Dovecot > wasn't the cause for the unresponsiveness. This operation is a tiny IO pattern compared to the 6.6GB re-indexing operation you mentioned before. So you can't make the simple assumption that "Dovecot wasn't the cause for the unresponsiveness". If fact Dovecot likely instigated the problem, though it likely isn't the "cause". I'll take a stab at that below. >> If the there's a kernel or hardware problem, you should see related >> errors in dmesg. Please share those. > > The kernel had messages like > > INFO: task cron:2799 blocked for more than 120 seconds. Now we're getting some meat on this plate. > in the dmesg. But again, I didn't mean to ask for a solution to this > problem. "blocked for more than 120 seconds" is a kernel warning message, not an error message. We see this quite often on the XFS list. Rarely, this is related to a kernel bug. Most often the cause of this warning is saturated IO. In this case it appears cron blocked for 120s because it couldn't read /var/cron/crontabs/[user] The most likely cause of this is that so many IO requests are piled up in the queue that it took more than 2 minutes for the hardware (disks) to complete them before servicing the cron process' IO requests. Dovecot re-indexing a 6.6GB mailbox, with other IO occurring concurrently, could easily cause this situation if you don't have sufficient spindle IOPS. I.e. this IO pattern will bring a single SATA disk or mirror pair to its knees. If you currently have everything on a single SATA disk or mirror pair, the solution for eliminating the bogging down of the system, and likely the Dovecot issues related to it, is to simply separate your root filesystem, swap, and Dovecot data files onto different physical devices. For instance, moving the root filesystem and swap to a small SSD will prevent the OS unresponsiveness, even if Dovecot is bogged down with IO to the SATA disk. With spinning rust storage, separation of root filesystem, swap, and application data to different storage IO domains is system administration 101 kind of stuff. If you're using SSD this isn't (as) critical as it's pretty hard to saturate the IO limits of an SSD. -- Stan From stan at hardwarefreak.com Sat Oct 27 05:54:21 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 21:54:21 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> Message-ID: <508B4CDD.4070508@hardwarefreak.com> On 10/26/2012 1:30 PM, Milan Holz?pfel wrote: > On Wed, 24 Oct 2012 13:28:11 +0200 > Milan Holz?pfel wrote: > >> I have a problem with an incosistent mdbox: >> [...] >> The problem appeared out of nowhere. [...] > > That's just wrong. Two minutes before the corruption occured for > the first time, the machine was booted after power-off without prior > shutdown. I didn't notice this until now, sorry for this. Ahh, more critical information. Better late than never I guess. > The mailbox is on XFS. As far as I remember, XFS in known for leaving > NULL bytes at the end of files after a system reset. At least, I found > 72 bytes of NULL in a plain text log file on XFS after such an event. > Do you think this may be the source of the index corruption? Very possibly. > Do you have any other suggestions for recovering the mailbox? Other than restoring from a backup, I do not. Others might. But I will offer this suggestion: Never run a server without a properly functioning UPS and shutdown scripts. The system in question isn't a laptop is it? I'm trying to ascertain how many server 'rules' you're breaking before making any more assumptions or giving any more advice. -- Stan From bernics.gabor at penta.hu Sat Oct 27 10:52:29 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Sat, 27 Oct 2012 09:52:29 +0200 Subject: [Dovecot] mail open slowly Message-ID: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> Hello, I have a dovecot IMAP server (relative small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) with 100 Gbyte maildirs. Server works fine but sometimes I can open mails slowly (5-10 sec), typical with new mails. It's intresting when I open an another old mail (with 0 sec wait), after new mail open quickly. Is it dovecot mail indexes or I/O problem? Load and CPU use is small typical 0.10, 10%, I see small IO wait. Debian 6.0, Dovecot 1.2.15, fsync and nmap is disable Best Regards, Gabor From rs at sys4.de Sat Oct 27 12:36:03 2012 From: rs at sys4.de (Robert Schetterer) Date: Sat, 27 Oct 2012 11:36:03 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> Message-ID: <508BAB03.9050709@sys4.de> Am 27.10.2012 09:52, schrieb Bernics G?bor | Penta Uni? Zrt.: > > > Hello, > > I have a dovecot IMAP server (relative small hardware: HP > Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) > with 100 Gbyte maildirs. that seems ok > > Server works fine but sometimes I can open > mails slowly (5-10 sec), typical with new mails. what mailbox type, how much mail in that mailbox how many concurent cons, imap or pop3 ? > > It's intresting when I > open an another old mail (with 0 sec wait), after new mail open > quickly. may client cached > > Is it dovecot mail indexes or I/O problem? perhaps this , perhaps other > > Load and CPU use > is small typical 0.10, 10%, I see small IO wait. > > Debian 6.0, Dovecot > 1.2.15, fsync and nmap is disable show config and logs > > Best Regards, > > Gabor > > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From bernics.gabor at penta.hu Sat Oct 27 14:58:27 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Sat, 27 Oct 2012 13:58:27 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <508BAB03.9050709@sys4.de> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> Message-ID: <5f981c03d54f00df233f82495df72022@penta.hu> Thank you I use dovecot LDA (+sieve) with maildir. conf: http://pastebin.com/9fhYD58g logs: http://pastebin.com/CXct3B6k connections: http://pastebin.com/v24iRz60 "It's intresting when I open an another old mail (with 0 sec wait), after new mail open quickly. may client cached" it's possible. 2012-10-27 11:36 id?pontban Robert Schetterer ezt ?rta: > Am 27.10.2012 09:52, schrieb Bernics G?bor | Penta Uni? Zrt.: > >> Hello, I have a dovecot IMAP server (relative small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) with 100 Gbyte maildirs. > > that seems ok > >> Server works fine but sometimes I can open mails slowly (5-10 sec), typical with new mails. > > what mailbox type, how much mail in that mailbox > how many concurent cons, imap or pop3 ? > Is it dovecot mail indexes or I/O problem? > > perhaps this , perhaps other From rs at sys4.de Sat Oct 27 15:55:04 2012 From: rs at sys4.de (Robert Schetterer) Date: Sat, 27 Oct 2012 14:55:04 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <5f981c03d54f00df233f82495df72022@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> Message-ID: <508BD9A8.8010101@sys4.de> Am 27.10.2012 13:58, schrieb Bernics G?bor | Penta Uni? Zrt.: > > > Thank you > > I use dovecot LDA (+sieve) with maildir. > > conf: in general you should upgrade to 2.1.x with lmtp for better performance use auth cache http://wiki.dovecot.org/Authentication/Caching an example you find in your config > > > http://pastebin.com/9fhYD58g > > logs: > > http://pastebin.com/CXct3B6k i dont see imap here, do more verbose logging > > > connections: > > http://pastebin.com/v24iRz60 to much info for this stage how much traffic concurent in fail stage would be interesting any chance measure iops of the storage ? what mail client ? if tb do debug http://wiki.dovecot.org/Debugging/Thunderbird how much mail does this mailbox have > > "It's intresting when I > open an another old mail (with 0 sec wait), after new mail open quickly. > > > may client cached" > > it's possible. > > 2012-10-27 11:36 id?pontban Robert > Schetterer ezt ?rta: > >> Am 27.10.2012 09:52, schrieb Bernics G?bor | > Penta Uni? Zrt.: >> >>> Hello, I have a dovecot IMAP server (relative > small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with > 2x 7200 SATA disk) with 100 Gbyte maildirs. >> >> that seems ok >> >>> > Server works fine but sometimes I can open mails slowly (5-10 sec), > typical with new mails. >> >> what mailbox type, how much mail in that > mailbox >> how many concurent cons, imap or pop3 ? >> Is it dovecot mail > indexes or I/O problem? >> >> perhaps this , perhaps other > > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stan at hardwarefreak.com Sat Oct 27 21:27:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 27 Oct 2012 13:27:50 -0500 Subject: [Dovecot] mail open slowly In-Reply-To: <5f981c03d54f00df233f82495df72022@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> Message-ID: <508C27A6.30207@hardwarefreak.com> On 10/27/2012 6:58 AM, Bernics G?bor | Penta Uni? Zrt. wrote: > I use dovecot LDA (+sieve) with maildir. > > conf: > > http://pastebin.com/9fhYD58g Next time simply paste "dovecot -n" output into your email. Assuming Dovecot is the only program accessing the maildirs, try: maildir_very_dirty_syncs=yes That may help some. It may not have been a factor in this case, but note that when anyone is doing a full text search on a large mailbox on this hardware with maildir you will see latency, and it is unavoidable. Neither a single 7.2K SATA spindle nor md/RAID1 pair of them, has enough seek capacity to service all the sector requests in a timely fashion. Also, I noticed you disabled fsync. This is a very very bad idea for a mail server. If you lose power, or suffer a kernel/hardware/etc crash, you lose the Linux buffer cache contents. Thus, you may lose emails that haven't been flushed to disk, and possibly get index file corruption if mmap'd pages haven't been flushed. Running with fsync disabled is like having sex with a Bangkok prostitute without a condom while juggling chainsaws while driving drunk at 250kph at night without headlights. fsync does hurt write performance to a degree, especially with maildir storage, but will likely be invisible on a small server with few users/light load. And it will prevent potentially severe problems with file corruption and/or lost emails. -- Stan From stephan at rename-it.nl Sun Oct 28 00:03:20 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 27 Oct 2012 23:03:20 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5088E74C.9030006@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> <5088E74C.9030006@ngtech.co.il> Message-ID: <508C4C18.5010401@rename-it.nl> On 10/25/2012 9:16 AM, Eliezer Croitoru wrote: > My only problem is that it will put the file in the folder but will > not mark the folder with the new file until I actually check the > folder manually. > it's not that much hustle but if there is a way to solve it I will be > more then happy to hear about it. This is most likely a client problem. Have you configured your client to check that folder? Regards, Stephan. From dave at boostpro.com Sat Oct 27 23:00:16 2012 From: dave at boostpro.com (David Abrahams) Date: Sat, 27 Oct 2012 16:00:16 -0400 Subject: [Dovecot] When are search indexes updated? Message-ID: I noticed that occasionally searching in my huge archive mailbox can be really slow, so I tried doveadm index on it and it seemed to do a lot of work, which seemed strange given, for example, that dovecot-lda says it keeps Dovecot index files up-to-date. Then I thought, "maybe these are different files than the search indices." If so, that's not entirely clear from the docs and Wiki. So, questions: * When are search indexes updated? * Are they updated incrementally? * If not, why not? * If so, why would a mailbox's index drift out-of-date, as mine had? BTW, I'm using the clucene search backend. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From stan at hardwarefreak.com Sun Oct 28 04:46:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 27 Oct 2012 21:46:34 -0500 Subject: [Dovecot] When are search indexes updated? In-Reply-To: References: Message-ID: <508C9C8A.8000309@hardwarefreak.com> On 10/27/2012 3:00 PM, David Abrahams wrote: > > I noticed that occasionally searching in my huge archive mailbox can be > really slow, so I tried doveadm index on it and it seemed to do a lot of > work, which seemed strange given, for example, that dovecot-lda says it > keeps Dovecot index files up-to-date. Then I thought, "maybe these are > different files than the search indices." If so, that's not entirely > clear from the docs and Wiki. So, questions: Mailbox and search indexes are separate. Look in your mailbox directory and you'll see them, such as on 1.2.x with mbox: $ la /home/stan/mail/.imap/1-Dovecot total 3.4M drwx------ 2 stan stan 135 Oct 25 21:39 . drwx------ 51 stan stan 4.0K Apr 13 2012 .. -rw------- 1 stan stan 44K Oct 27 13:28 dovecot.index -rw------- 1 stan stan 1.2M Oct 27 21:23 dovecot.index.cache -rw------- 1 stan stan 18K Oct 27 21:23 dovecot.index.log -rw------- 1 stan stan 1.1M May 20 06:32 dovecot.index.search -rw------- 1 stan stan 1.1M May 20 06:32 dovecot.index.search.uids I've not full text searched this folder for quite some time, thus the search indexes are not current, and the next FTS of this mail folder will take much more time than if the FTS indexes were current. > * When are search indexes updated? When the index is stale. > * Are they updated incrementally? > * If not, why not? > * If so, why would a mailbox's index drift out-of-date, as mine had? When a sufficient number of messages are added to an IMAP folder the FTS index becomes stale. This index is not updated in real time. This is why Timo and others recommend cron'ing a script to index folders regularly that are searched regularly. This keeps the indexes up to date and keeps searches fast. If you don't do this or search often, your indexes become stale. Then each time you do an FTS search the first thing that happens is an FTS re-indexing of the mail folder. Only then does it display the search results. > BTW, I'm using the clucene search backend. I've not used Lucene, but I believe the default behavior is similar to the Dovecot 1.2.x FTS indexer. -- Stan From claude.xavier at gmail.com Sun Oct 28 11:28:28 2012 From: claude.xavier at gmail.com (Xavier Claude) Date: Sun, 28 Oct 2012 10:28:28 +0100 Subject: [Dovecot] How to activate antispam plugin Message-ID: <3146470.D7UsahtfX5@coruscant> Hello, I'm using dovecot 2.1.7 from the Debian backports package and I'm trying to get working the antispam plugin with dspam. I have followed the documentation http://wiki2.dovecot.org/Plugins/Antispam but it does not seem to work. The /var/log/dspam folder is empty even after I put mail in the Spam folder. How can I see if the plugin is working and what config option am I missing ? Thank in advance for your help. Here is my config from dovecot -n: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-xenU-6887-i386 i686 Debian 6.0.6 mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } plugin { antispam_backend = dspam antispam_dspam_args = --mode=teft;--deliver=;--user;%u antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = move antispam_spam = Spam antispam_trash = trash;Corbeille;Trash;Deleted Items; Deleted Messages sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } ssl_cert = References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> <5088E74C.9030006@ngtech.co.il> <508C4C18.5010401@rename-it.nl> Message-ID: <508D4F1F.4030603@ngtech.co.il> On 10/27/2012 11:03 PM, Stephan Bosch wrote: > > This is most likely a client problem. Have you configured your client to > check that folder? Yes unless there there is a special thing I dont know yet about in Thunderbird. Thanks, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From afodis.pinon at hotmail.fr Mon Oct 29 11:01:03 2012 From: afodis.pinon at hotmail.fr (Boris PINON) Date: Mon, 29 Oct 2012 10:01:03 +0100 Subject: [Dovecot] Active Directory 2003 user database and passwords with special characters Message-ID: Hello everybody, As explained in the topic, i have troubles with authentication of my users. First of all, sorry for my poor english... I'm running dovecot v1.2.15 on a Debian 6 64bits server up to date. My users database is an Active Directory 2003 (it's important to know that because Active Directory can't retrieve users passwords, you have to bind LDAP with a domain administrator). So, i'm using userdb ldap for authenticated my users and it works ! BUT... When an user having a password with special characters like " ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ", authentication does not work. And in my log file : MY_SERVER | mail/info | dovecot | 2012/08/27 10:42:14 | auth(default): cache(my_user,192.168.7.127): plain(RU0975?*) != 'RU0975??*' As you can see, the character ? has been replaced by ??. My dovecot.conf : protocols = imap imaps pop3 pop3s managesieve shutdown_clients = yes protocol imap { listen = 192.168.7.1:143 ssl_listen = 192.168.7.1:993 mail_plugins = quota imap_quota autocreate imap_client_workarounds = outlook-idle delay-newmail tb-extra-mailbox-sep } protocol pop3 { listen = 192.168.7.1:110 ssl_listen = 192.168.7.1:995 mail_plugins = quota pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_lock_session = no pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol managesieve { listen = 192.168.7.1:4190 login_executable = /usr/lib/dovecot/managesieve-login mail_executable = /usr/lib/dovecot/managesieve managesieve_implementation_string = dovecot } protocol lda { mail_plugins = sieve quota postmaster_address = postmaster at contoso.fr hostname = webmail.contoso.fr sendmail_path = /usr/sbin/sendmail quota_full_tempfail = no auth_socket_path = /var/run/dovecot/auth-master } log_timestamp = "%Y-%m-%d %H:%M:%S " syslog_facility = mail mail_debug = no auth_debug = no auth_debug_passwords = no ssl = required ssl_cert_file = /etc/ssl/certs/webmail.contoso.fr.pem ssl_key_file = /etc/ssl/private/webmail.contoso.fr.key ssl_ca_file = /etc/ssl/certs/VERYSIGN.pem ssl_verify_client_cert = no mail_location = maildir:%h mail_full_filesystem_access = no mail_uid = 500 mail_gid = 8 mail_privileged_group = mail first_valid_uid = 500 last_valid_uid = 500 first_valid_gid = 8 last_valid_gid = 8 login_greeting = Webmail CONTOSO login_process_size = 256 login_process_per_connection = no login_processes_count = 2 login_max_processes_count = 128 login_max_connections = 512 max_mail_processes = 1024 mail_process_size = 256 mail_max_keyword_length = 50 disable_plaintext_auth = yes auth_failure_delay = 2 auth_process_size = 256 auth_username_format = %Lu auth default { mechanisms = plain login auth_cache_size = 2048 passdb ldap { args = /etc/dovecot/dovecot-ldap.conf } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } user = vmail count = 1 socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail group = mail } client { path = /var/run/dovecot/auth-client mode = 0666 user = vmail group = mail } } } dict { } plugin { quota = maildir:User quota quota_warning = bytes=80%% /usr/lib/dovecot/quota-warning 80 quota_warning2 = bytes=95%% /usr/lib/dovecot/quota-warning 95 quota_warning3 = bytes=99%% /usr/lib/dovecot/quota-warning 99 sieve=dovecot.sieve sieve_dir=~/.Sieve sieve_extensions=+imapflags autocreate = Spam autocreate2 = Trash autosubscribe = Spam autosubscribe2 = Trash autosubscribe3 = Sent autosubscribe4 = Drafts } And my dovecot-ldap.conf : # My domain controller uris = ldap://192.168.1.1:3268 dn = CN=ServerOperator,CN=Users,DC=contoso,DC=fr dnpass = MyPassword debug_level = 0 auth_bind = yes ldap_version = 3 base = CN=Users,DC=contoso,DC=fr deref = never scope = subtree user_attrs = mailDirectory=home=/var/spool/mail/%$,mailQuota=quota_rule=*:bytes=%$,=quota_rule2=Trash:storage=100%% user_filter = (&(|(sAMAccountName=%n)(mailAcceptingGeneralID=%u)(mail=%u))(!(|(mailDrop="*|*")(mailDrop="*:*")(mailDrop="*/*")(userParams=noMail)))) pass_filter = (&(sAMAccountName=%n)(!(|(mailDrop="*|*")(mailDrop="*:*")(mailDrop="*/*")(userParams=noMail)))) default_pass_scheme = CRYPT Does anyone else have this problem? If yes, how to solve? Thank you in advance. From tss at iki.fi Mon Oct 29 16:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:18:22 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <20121016231856.GA10851@daniel.localdomain> References: <20121016231856.GA10851@daniel.localdomain> Message-ID: <0CECDB89-90BF-4A2F-97AC-713344F24996@iki.fi> On 17.10.2012, at 2.18, Daniel Parthey wrote: > doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A > shows the following message in the log when iterating the 49th user: > > Oct 17 00:47:17 10.129.3.233 dovecot: doveadm: Error: purge: invalid option -- 'e' > Oct 17 00:47:17 10.129.3.233 dovecot: doveadm(someuser at example-ll.org): Error: doveadm purge: Client sent unknown parameter: ? > > Any ideas on how this error gets triggered? Not sure. There's no valid 'e' option anywhere. I guess one of the non-option parameters begin with "-e" and it think it's an option. But I can't really think of how that would happen with purge either. So it would be helpful to look at what exactly the doveadms are talking to each others. Could you get the network traffic from them? Or strace -s 1000 doveadm purge should show it somewhere too. From tss at iki.fi Mon Oct 29 16:27:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:27:20 +0200 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <20121018060354.GA2528@leningrad.koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> Message-ID: On 18.10.2012, at 9.03, Levent Dane wrote: >> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html > > in Code/dovecot-virtual: > Archive > inthread refs keyword code not deleted I still couldn't reproduce with this. > I tried to take coredump but i didn't compile with debug flags. > http://pastebin.com/CMbiYJeK I think the problem here mainly is that gdb doesn't work very nicely across multiple execs (imap executes doveconf which executes imap again). You can avoid that by getting a core dump the regular way or making the $base_dir/config socket 0666 permissions. > If you can't reproduce this error. Tomorrow, I'll compile with debug flags. A proper gdb backtrace would definitely be the easiest way to solve this. BTW. Is it only STATUS (UNSEEN) that crashes, or also if you simply SELECT the mailbox? From bernics.gabor at penta.hu Mon Oct 29 16:29:22 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Mon, 29 Oct 2012 15:29:22 +0100 Subject: [Dovecot] mail open slowly In-Reply-To: <508C27A6.30207@hardwarefreak.com> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> <508C27A6.30207@hardwarefreak.com> Message-ID: Thanks a lot to everybody First step I will upgrade to dovecot2. I will write my experiences. "Running with fsync disabled is like having sex with a Bangkok prostitute without a condom while juggling chainsaws while driving drunk at 250kph at night without headlights." :-) Gabor 2012-10-27 20:27 id?pontban Stan Hoeppner ezt ?rta: > On 10/27/2012 6:58 AM, Bernics G?bor | Penta Uni? Zrt. wrote: > >> I use dovecot LDA (+sieve) with maildir. conf: http://pastebin.com/9fhYD58g [1] > > Next time simply paste "dovecot -n" output into your email. > > Assuming Dovecot is the only program accessing the maildirs, try: > > maildir_very_dirty_syncs=yes > > That may help some. > > It may not have been a factor in this case, but note that when anyone is > doing a full text search on a large mailbox on this hardware with > maildir you will see latency, and it is unavoidable. Neither a single > 7.2K SATA spindle nor md/RAID1 pair of them, has enough seek capacity to > service all the sector requests in a timely fashion. > > Also, I noticed you disabled fsync. This is a very very bad idea for a > mail server. If you lose power, or suffer a kernel/hardware/etc crash, > you lose the Linux buffer cache contents. Thus, you may lose emails > that haven't been flushed to disk, and possibly get index file > corruption if mmap'd pages haven't been flushed. > > Running with fsync disabled is like having sex with a Bangkok prostitute > without a condom while juggling chainsaws while driving drunk at 250kph > at night without headlights. > > fsync does hurt write performance to a degree, especially with maildir > storage, but will likely be invisible on a small server with few > users/light load. And it will prevent potentially severe problems with > file corruption and/or lost emails. Links: ------ [1] http://pastebin.com/9fhYD58g From tss at iki.fi Mon Oct 29 16:38:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:38:58 +0200 Subject: [Dovecot] Dovecot quota postgres dictionary problems In-Reply-To: <1350632450161-38234.post@n4.nabble.com> References: <1350632450161-38234.post@n4.nabble.com> Message-ID: <794F0BC6-CF9D-4BE7-A0C9-FC0D93E72166@iki.fi> On 19.10.2012, at 10.40, tmihalicek wrote: > I have a strange errors in .err log file, but the postgres seem to be filling > with quota changes, i will also put configs in > > Oct 19 09:23:52 mailstore-node-01 dovecot: imap(test at example.net): Error: > read(/var/run/dovecot/dict) failed: Timeout after 30 seconds > Oct 19 09:24:22 mailstore-node-01 dovecot: imap(test at example.net): Error: > read(/var/run/dovecot/dict) failed: Timeout after 30 seconds dict process is taking too long to give results back. Is PostgreSQL too heavily loaded? > Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Panic: > file dict-client.c: line 270 (client_dict_finish_transaction): assertion > failed: (dict->async_commits > 0) http://hg.dovecot.org/dovecot-2.1/rev/67e9cb0b06ec should fix this crash. From tss at iki.fi Mon Oct 29 16:42:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:42:57 +0200 Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Message-ID: On 19.10.2012, at 23.43, E.B. wrote: > I'm having some problems getting LDA to work without > userdb lookups and have a few related questions. This system has all > users in MySQL, each user with unique UID/GID, no local users at all. > Installation is from apt-get. > > > 1) If LDA is invoked without > lookups, is it correct to assume that the "service auth" and "service > auth-worker" can be completely removed from dovecot master > configuration? (I have tried commenting them out and logging into IMAP, > which seems to work, not sure if anyone else needs the auth service) If you remove them the defaults are simply used. > 2) > If LDA is invoked without lookups, will I be unable to use Dovecot > quota plugin? Does it need to have a user lookup to get quota info? > (haven't added quota support, need to take this one step at a time) You can give quota info also via either environment variables or via -o plugin/quota_rule=xx parameter. > 3) The interesting part -- I am invoking LDA from Maildrop. See: > http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So > when invoked, Maildrop has already dropped to the destination UID/GID > and the needed paths are available in the environment. However, using > as many permutations of calling LDA as I can think of (based on http://wiki2.dovecot.org/LDA ), I always get this: > > (command line usage error. Command output: lda: Fatal: Couldn't lookup our username (uid=2500) ) Set USER environment. > 3.5) > Related question, my users have separate homedir and maildir, both > paths are looked up by Maildrop. I think I need to call LDA with > "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? As long as the home and mail directories point to the same ones as they are when logging in via IMAP/POP3. From tss at iki.fi Mon Oct 29 16:45:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:45:06 +0200 Subject: [Dovecot] Configuring Dovecot & Snarf plugin for the first time In-Reply-To: <50835541.8000808@bubble.org> References: <50835541.8000808@bubble.org> Message-ID: On 21.10.2012, at 4.52, Jeffrey Ross wrote: > However whenever I enable the snarf plugin using the example on the wiki page my email is not loaded and when I remove my configuration for snarf my email re-appears. Based upon what I can tell the snarf plugin is either not loading (but I see it listed in the logs) or simply not working (which is probably because its not configured properly). .. > plugin { > snarf = = /snarf/INBOX > } Looks like you have one too many "="? From tss at iki.fi Mon Oct 29 16:47:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:47:25 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: References: Message-ID: <189B7E53-0495-4D2E-A845-6CEE1304898D@iki.fi> On 18.10.2012, at 11.05, Jan-Frode Myklebust wrote: > I enabled the trash plugin yesterday, adding "trash" to mail_plugins, > and configuring the plugin setting "trash = > /etc/dovecot/dovecot-trash.conf.ext". > > > But I still see users with lots of files in INBOX.Trash getting > bounced because of quota exceeded: .. > # 2.0.14: /etc/dovecot/dovecot.conf There are several fixes to Trash plugin in v2.1. I think it's simply somewhat broken in v2.0. From tss at iki.fi Mon Oct 29 16:49:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:49:58 +0200 Subject: [Dovecot] Auth caching and password changes In-Reply-To: <508526C2.8030403@um.es> References: <508149CC.9070004@um.es> <508526C2.8030403@um.es> Message-ID: <1B474730-A7EF-4607-9A1E-4DD215518E6B@iki.fi> On 22.10.2012, at 13.58, Angel L. Mateo wrote: > My question now is there any way to configure authentication so a mechanism is only use when connections coming from a set of IPs? local/remote {} blocks were supposed to provide this. They don't currently work for auth process settings though. From tss at iki.fi Mon Oct 29 17:04:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:04:25 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <5087FF4F.8050103@papaya-cms.com> References: <5087FF4F.8050103@papaya-cms.com> Message-ID: <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> On 24.10.2012, at 17.46, Alexander Weber wrote: > if address :is "to" "mantis-admin@<*>" > { > fileinto "/home/shared/.automail.Bugtracker/"; > } Use mailbox name, not filesystem path: fileinto "shared/automail/Bugtracker"; From tss at iki.fi Mon Oct 29 17:08:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:08:42 +0200 Subject: [Dovecot] Dovecot sends BYE while fetching X-GM-MSGID In-Reply-To: <1698191351159403@web29e.yandex.ru> References: <1698191351159403@web29e.yandex.ru> Message-ID: On 25.10.2012, at 13.03, Loshkovskyi Andrii wrote: > While using the following set of commands, I am having the error as below: > > FETCH 7 (X-GM-MSGID) > A15 FETCH 7 (X-GM-MSGID) > A15 BAD Error in IMAP command FETCH: Unknown parameter X-GM-MSGID > > Can I somehow disable such errors so that Dovecot won't send BYE on X-GM-MSGID but just proceed with following emails? BYE or BAD? It shouldn't send BYE unless you send 20 consecutive BAD commands. From weber at papaya-cms.com Mon Oct 29 17:11:25 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Mon, 29 Oct 2012 16:11:25 +0100 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> Message-ID: <508E9C9D.8060601@papaya-cms.com> Am 29.10.2012 16:04, schrieb Timo Sirainen: > On 24.10.2012, at 17.46, Alexander Weber wrote: > >> if address :is "to" "mantis-admin@<*>" >> { >> fileinto "/home/shared/.automail.Bugtracker/"; >> } > > Use mailbox name, not filesystem path: > > fileinto "shared/automail/Bugtracker"; > error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. sieve: info: started log at Oct 29 16:10:03. error: msgid=<*>: failed to store into mailbox 'shared/automail/Bugtracker/': Invalid mailbox name. nope, didn't work :/ From tss at iki.fi Mon Oct 29 17:18:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:18:42 +0200 Subject: [Dovecot] Dovecot stops to work - anvil problem In-Reply-To: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> References: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> Message-ID: <51972B14-6973-4510-870D-956F858FC76B@iki.fi> On 26.10.2012, at 13.24, FABIO FERRARI wrote: > Hi all, > > we have a problem about anvil, it seems that when we have a high load the > dovecot stops to work. Sometimes it is sufficient to make a dovecot > reload, but sometimes we have to restart it. > > Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files This is the problem. > And these are the limit settings in the OS: > * soft nofile 131072 > * hard nofile 131072 > > Have someone had the same problem? The OS limits are ok. But you need to make sure that the dovecot processes have enough fds in ulimit. You can check the limits with: cat /proc//limits The "Max open files" soft limit is what you're most likely hitting. Use "ulimit -n 10000" or something before running dovecot binary. And make sure that it changes the limit in the proc. Many init scripts change the ulimit internally. From tss at iki.fi Mon Oct 29 17:18:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:18:46 +0200 Subject: [Dovecot] Can Dovecot authenticate against an external email server? In-Reply-To: <5087035B.7060208@perkel.com> References: <5087035B.7060208@perkel.com> Message-ID: On 23.10.2012, at 23.51, Marc Perkel wrote: > Just wondering if anyone has done this. > > I have a spam filtering service where I am now storing spam for users I filter for. It's a filter and forward service so I don't control the recipient's email server. > > What I would like to do somehow is have the user enter their email address and password and then look up their POP/IMAP server from a database and try to authenticate from it. If sucessful then the user will be able to access their stored spam using Dovecot and Squirrelmail. http://wiki2.dovecot.org/PasswordDatabase/IMAP From tss at iki.fi Mon Oct 29 17:20:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:20:15 +0200 Subject: [Dovecot] dovecot lda - Permission denied In-Reply-To: <20121026114455.30440@gmx.net> References: <20121026114455.30440@gmx.net> Message-ID: <157A66BA-69AB-45AE-927C-21F827B1736B@iki.fi> On 26.10.2012, at 14.44, tony.blue.mailinglist at gmx.de wrote: > Oct 25 23:37:13 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(andy) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) > ... > Oct 25 23:37:14 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=500(tony) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) > ... > > Dovecot is installed as !include auth-passwdfile.conf.ext. For all users there is a entry in der /etc/dovecot/users. > > Usaly the user rights are set to 600. I tryed 755, but I get the same errormessage. 0755 is basically the same as 0600 for sockets, since you disabled writes for others. Use 0777 to give everyone permissions. From tss at iki.fi Mon Oct 29 17:22:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:22:22 +0200 Subject: [Dovecot] Overlapping userdb/passdbs In-Reply-To: References: Message-ID: <98C16420-1D9E-4F37-86D7-9FB91438B843@iki.fi> On 26.10.2012, at 22.13, James Devine wrote: > I have an ldap server for which each entry includes the email address and > the username portion of the email address for authentication. > Authentication works by username if the username is unique among all the > entries. I need to now add some users which must authenticate even if the > username is not unique. I figured one way to do this would be to add a > second user/pass db which puts further restrictions on the ldap query to > make it unique for those users. This doesn't seem to work however as if > the user is found in the first ldap query but the password does not match > it does not try the second. I would use the password as part of the query > but this setup requires me to allow the client to hash the password. Is > there a way to do this? Or maybe I am approaching the problem wrong. You'd need to update this patch: http://dovecot.org/patches/2.0/auth-multi-password-2.0.diff It worked for v1.1 and maybe for v1.2. I never included it mainly because I never had time to check if it had any security issues. From jk at jkart.de Mon Oct 29 17:23:29 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 16:23:29 +0100 Subject: [Dovecot] Out of memory/Managesieve Message-ID: <508E9F71.8050208@jkart.de> Hello, I have here a problem with managesieve. With the login about webmail (roundcube) comes here in the log: --snip dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) --snap I've increased of 2048M and the same above. then with 4096 M happens the following --snip dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) --snap How can one solve then this? Any ideas greatly appreciated. Thanks. -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Backup interessiert niemanden - Auf Restore kommt es an! From tss at iki.fi Mon Oct 29 17:26:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:26:49 +0200 Subject: [Dovecot] When are search indexes updated? In-Reply-To: <508C9C8A.8000309@hardwarefreak.com> References: <508C9C8A.8000309@hardwarefreak.com> Message-ID: <6DC094E4-2D06-4146-A4C2-1717614E30E4@iki.fi> On 28.10.2012, at 4.46, Stan Hoeppner wrote: >> * When are search indexes updated? > > When the index is stale. > >> * Are they updated incrementally? >> * If not, why not? >> * If so, why would a mailbox's index drift out-of-date, as mine had? > > When a sufficient number of messages are added to an IMAP folder the FTS > index becomes stale. This index is not updated in real time. This is > why Timo and others recommend cron'ing a script to index folders > regularly that are searched regularly. This keeps the indexes up to > date and keeps searches fast. If you don't do this or search often, > your indexes become stale. Then each time you do an FTS search the > first thing that happens is an FTS re-indexing of the mail folder. Only > then does it display the search results. Otherwise correct, but "re-indexing" is the wrong word. No already indexed mails are reindexed. Only new mails are added to the index. From tss at iki.fi Mon Oct 29 17:31:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:31:42 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <394FADB5-4E6C-4179-BF30-244390964FA4@iki.fi> On 24.10.2012, at 14.28, Milan Holz?pfel wrote: > Oct 24 10:43:23 two dovecot: imap-login: Login: user=<...>, method=PLAIN, rip=..., lip=..., mpid=4977, TLS > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-24 10:43:23] bytes=115/53726 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: Inconsistency in map index (467,31960 != 467,36768) > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: rebuilding indexes The above problems aren't too bad, since Dovecot fixes itself. > Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Panic: file mdbox-storage-rebuild.c: line 773 (rebuild_update_refcounts): assertion failed: (map_uid < msgs[i]->map_uid) Now this is a bug. > Dovecot 2.0.19-0ubuntu1 But the bug may have already been fixed in v2.1. > The whole mdbox is 6.6 GiB large and I guess that it contains about > 300k-600k messages. It's an archive of public mailing lists, so I could > give access to the files. I'd try first with a recent 2.1 version and if that doesn't fix the crash the easiest way for me to fix it would be to get the files. If you put the files through http://dovecot.org/tools/mdbox-obfuscate.pl they should compress pretty nicely. From tss at iki.fi Mon Oct 29 17:34:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:34:28 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <508E9C9D.8060601@papaya-cms.com> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> <508E9C9D.8060601@papaya-cms.com> Message-ID: <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> On 29.10.2012, at 17.11, Alexander Weber wrote: >> fileinto "shared/automail/Bugtracker"; > > error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. "shared/automail/Bugtracker" isn't the same as "shared/.automail.Bugtracker/". Anyway that specific error message comes from the trailing '/'. From jk at jkart.de Mon Oct 29 17:34:45 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 16:34:45 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508E9F71.8050208@jkart.de> References: <508E9F71.8050208@jkart.de> Message-ID: <508EA215.6000303@jkart.de> am 29.10.12 16:23 schrieb Jim Knuth : > Hello, > > I have here a problem with managesieve. With the login about > webmail (roundcube) comes here in the log: > > --snip > dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): > Out of memory > dovecot: managesieve-login: Fatal: master: service(managesieve-login): > child 10157 returned error 83 (Out of memory (service > managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) > --snap > I've increased of 2048M and the same above. > then with 4096 M happens the following > > --snip > dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: > Bad file descriptor > dovecot: managesieve-login: Fatal: master: service(managesieve-login): > child 9777 killed with signal 6 (core dumps disabled) > --snap > > How can one solve then this? > Any ideas greatly appreciated. Thanks. > Sorry, I've forgotten. OS Debian stable and Dovecot 2.1. -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Nicht Absicht unterstellen, wenn auch Dummheit ausreicht! From tss at iki.fi Mon Oct 29 17:41:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:41:06 +0200 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508E9F71.8050208@jkart.de> References: <508E9F71.8050208@jkart.de> Message-ID: On 29.10.2012, at 17.23, Jim Knuth wrote: > I have here a problem with managesieve. With the login about > webmail (roundcube) comes here in the log: You can always easily reproduce this? Can you get the network traffic logs between Roundcube and Dovecot and reproduce it by sending those same commands manually? > --snip > dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory > dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) Looks like there's a bug somewhere.. Doesn't it log a "raw backtrace"? > --snap > I've increased of 2048M and the same above. > then with 4096 M happens the following > > --snip > dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor > dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) > --snap Probably related to the first error. It would be helpful to get gdb backtraces from both of them, although from the first one you couldn't without patching + recompiling Dovecot. But from the second one I think you can get a core dump with: service managesieve-login { executable = managesieve-login -D } Then you can do something like: gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core bt full From weber at papaya-cms.com Mon Oct 29 17:54:51 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Mon, 29 Oct 2012 16:54:51 +0100 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> <508E9C9D.8060601@papaya-cms.com> <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> Message-ID: <508EA6CB.9010600@papaya-cms.com> Am 29.10.2012 16:34, schrieb Timo Sirainen: > On 29.10.2012, at 17.11, Alexander Weber wrote: > >>> fileinto "shared/automail/Bugtracker"; >> >> error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. > > "shared/automail/Bugtracker" isn't the same as "shared/.automail.Bugtracker/". > > Anyway that specific error message comes from the trailing '/'. > I've tried every combination of this shared subfolder ( pretty wired :S ) but nevermind - i try procmail with sieve after procmail. thanks anyway :) From tss at iki.fi Mon Oct 29 18:53:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 18:53:00 +0200 Subject: [Dovecot] Save/restore IMAP session state Message-ID: <1351529580.13571.93.camel@hurina> Some future Dovecot version will have "imap-idle" processes where IDLEing IMAP connections get moved, so the system wouldn't waste so much memory for all the IDLEing imap processes. A week ago I thought I'd see how easy it would be to implement this. I got a basic proof of concept working as a "X-STATE" command. Save the state: a x-state * STATE AQDLW45QdwAAAAMAAABuAQAAAAAAAFAcffYAPHnpFctbjlDbYQAAcEmzCwAA a OK State exported. Restore the state: b x-state AQDLW45QdwAAAAMAAABuAQAAAAAAAFAcffYAPHnpFctbjlDbYQAAcEmzCwAA b OK State imported. This could also be used to implement quick session state restoring for webmails (as suggested by Michael Slusarz). For getting the imap-idle process there would have to be code that: * triggers the session saving when process is IDLEing * figures out what filesystem paths the imap-idle should be looking at (i.e. paths to selected mailbox's dovecot.index.log file and maybe for e.g. maildir new/) * send the session state string, paths and imap connection fd to imap-idle process via UNIX socket * implement the actual imap-idle process * implement a way for imap-idle process to send back the state and connection fd to restore the imap process The patch is ugly and still missing many things. Anyway I thought I'd include it here just in case someone was really eager to continue implementing it. :) I'm not sure when I'll have time for it. A full patch would probably have to have some session_save()/session_restore() functions in lib-storage API. But a quick and dirty way is possible to implement for v2.1 as well, as long as some IMAP extensions aren't used (most importantly rfc5267). -------------- next part -------------- A non-text attachment was scrubbed... Name: imap-state.diff Type: text/x-patch Size: 11305 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 18:57:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 18:57:32 +0200 Subject: [Dovecot] Save/restore IMAP session state In-Reply-To: <1351529580.13571.93.camel@hurina> References: <1351529580.13571.93.camel@hurina> Message-ID: <567ADA17-F5E6-48DF-9E9D-601267C568FE@iki.fi> On 29.10.2012, at 18.53, Timo Sirainen wrote: > The patch is ugly and still missing many things. Anyway I thought I'd > include it here just in case someone was really eager to continue > implementing it. :) I'm not sure when I'll have time for it. Oh, and of course I forgot one file out of the patch. Here's an updated one. -------------- next part -------------- A non-text attachment was scrubbed... Name: imap-state2.diff Type: application/octet-stream Size: 12948 bytes Desc: not available URL: From guallar at easternrad.com Mon Oct 29 19:57:37 2012 From: guallar at easternrad.com (Josep L. Guallar-Esteve) Date: Mon, 29 Oct 2012 13:57:37 -0400 Subject: [Dovecot] INBOX permissios woes Message-ID: <9cc05811b75ed0f7235dd86d0e5c1dfd@easternrad.com> Hello, I have a dovecot system that uses winbind authentication against Active Directory. I set it up by following the directions in the wiki. That works great. When a new user receives an email, the inbox is created with permissions 600 (rw- --- --) and ownership user:mail , even though I did chmod 02770 /var/spool/mail. And then, when dovecot tries to access the inbox, it throws the error: Oct 29 13:47:59 imap-login: Info: Login: user=, method=PLAIN, rip=10.0.0.6, lip=10.0.0.26, mpid=29047, secured Oct 29 13:47:59 imap(user1): Error: stat(/var/mail/user1) failed: Permission denied Oct 29 13:47:59 imap(user1): Error: stat(/var/mail/user1) failed: Permission denied Accessing users' Sent, Trash, creating new folders.... all that works fine. I've been looking at the documentation, reading the wiki, searching on google, asking on IRC. If you have any hint or documentation that I've must have overlooked, please let me know. Here's my dovecot information: [josep at testmail ]$ dovecot --version 2.0.9 [josep at testmail ]$ dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.11.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain ntlm login auth_use_winbind = yes auth_username_format = %Lu base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log listen = * log_path = /var/log/dovecot.log mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail mbox_write_locks = fcntl passdb { driver = pam } protocols = imap service auth { unix_listener auth-userdb { mode = 0600 } } ssl_cert = References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> Message-ID: <20121029181700.GA4240@leningrad.koli.be> On 10/29, Timo Sirainen wrote: >On 18.10.2012, at 9.03, Levent Dane wrote: > >>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >> >> in Code/dovecot-virtual: >> Archive >> inthread refs keyword code not deleted > >I still couldn't reproduce with this. I think the problem is mail-search.c is corrupting the index files. >> I tried to take coredump but i didn't compile with debug flags. >> http://pastebin.com/CMbiYJeK > >I think the problem here mainly is that gdb doesn't work very nicely across multiple execs (imap executes doveconf which executes imap again). You can avoid that by getting a core dump the regular way or making the $base_dir/config socket 0666 permissions. I compiled with -ggdb flag. I'm getting this informations /var/log/messages: http://pastebin.com/bpkvp4Ak and from gdb: http://pastebin.com/HY0mVYBS I'm using mutt for imap access. When I pressed '%' key which runs function, the dovecot got seqfault. >> If you can't reproduce this error. Tomorrow, I'll compile with debug flags. > >A proper gdb backtrace would definitely be the easiest way to solve this. > >BTW. Is it only STATUS (UNSEEN) that crashes, or also if you simply SELECT the mailbox? I tried simple SELECT and it still crashed. As I said, the problem is dovecot.index files. Somehow, mail-search.c corrupts this file. -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 From tss at iki.fi Mon Oct 29 20:23:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 20:23:14 +0200 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <20121029181700.GA4240@leningrad.koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> <20121029181700.GA4240@leningrad.koli.be> Message-ID: <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> On 29.10.2012, at 20.17, Levent Dane wrote: > On 10/29, Timo Sirainen wrote: >> On 18.10.2012, at 9.03, Levent Dane wrote: >> >>>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >>> >>> in Code/dovecot-virtual: >>> Archive >>> inthread refs keyword code not deleted >> >> I still couldn't reproduce with this. > > I think the problem is mail-search.c is corrupting the index files. Not that itself, but yeah looks like if virtual plugin assert-crashes in mail-search.c it leaves the indexes so that the next access will segfault. > I compiled with -ggdb flag. I'm getting this informations > /var/log/messages: http://pastebin.com/bpkvp4Ak > and from gdb: http://pastebin.com/HY0mVYBS Better backtrace than last time, but still no debug information in the backtrace. Maybe that got stripped somewhere between compiling and installing? You can check with "file ..../imap" to see if it's there. Also backtrace from both the mail-search.c assert crash and the segfault would be useful. From calestyo at scientia.net Mon Oct 29 22:31:48 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 21:31:48 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? Message-ID: <1351542708.3435.25.camel@fermat.scientia.net> Hi. For mbox, http://wiki2.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata lists a numer of mail headers: - X-IMAPbase - X-IMAP - X-UID - Status - X-Status - X-Keywords - Content-Length that are treated specially by dovecot. It also suggests, that these should be stripped by the LDA (I guess in order that someone sending you such mail cannot set the status or keywords, or even "attack you" by setting a bogus Content-Length). I wondered, when using maildir, are there any headers that dovecote would treat specially, too? And which I therefore should strip? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 22:39:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 22:39:51 +0200 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? Message-ID: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. From tss at iki.fi Mon Oct 29 22:40:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 22:40:46 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351542708.3435.25.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> Message-ID: <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> On 29.10.2012, at 22.31, Christoph Anton Mitterer wrote: > For mbox, > http://wiki2.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata lists a > numer of mail headers: > - X-IMAPbase > - X-IMAP > - X-UID > - Status > - X-Status > - X-Keywords > - Content-Length > that are treated specially by dovecot. > > It also suggests, that these should be stripped by the LDA (I guess in > order that someone sending you such mail cannot set the status or > keywords, or even "attack you" by setting a bogus Content-Length). Right. > I wondered, when using maildir, are there any headers that dovecote > would treat specially, too? > And which I therefore should strip? No. Maildir metadata is stored elsewhere. From calestyo at scientia.net Mon Oct 29 22:54:09 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 21:54:09 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste Message-ID: <1351544049.3435.47.camel@fermat.scientia.net> Hi. I recently mentioned in several posts, that I'd tended to use mbox rather than maildir, because you don't loose so much space (due to always allocating full blocks per maildir file and thus per mail). I made some tests of my archive, which consists of some 3,4 million mails at a total of 42GB). Most of these mails are probably normal sized, but there are also some with bigger attachments. For those who are interested here are the results: I used a 53687091200 B image file (via loop device) and tested ext4 only. btrfs is IMHO not yet ready, I have had often issues with XFS (corruptions), reiser4 is more or less dead and reiser3 is said to have issues (see e.g. its wikipedia article, even though it has that mode for small files which would fit nicely). As you see the number of mails increased a bit, cause I tested over several days... but this is only a very small increase so it shouldn't change the numbers a lot. 1) Original mbox archives (right now in Evolution) mbox exact space: 38122676224 (does not include meta-data) mbox guess space: 44625670144 (includes Evolution meta-data which is several GBs) mbox num mails: 3412999 (occurances of From_ lines) In the following: - image file, 1B-blocks, Used_begin, Used_end, Available_begin, Available_end result out of df -B 1 - mdir exact used space is the sum of du -B 1 for each regular file (i.e. each mdir file) - mdir guess used space du -B 1 on the root dir of the filesystem - mdir num mails: find . type -f | wc -l on the root dir of the filesystem 2) EXT4 with 4096 blocks: image file: 53687091200 1B-blocks: 52844687360 Used_begin: 188555264 Used_end: 45198778368 Available_begin: 49971777536 Available_end: 2444972032 mdir exact used space: 44810866688 mdir guess used space: 45010243584 mdir num mails: 3423296 delta: 6.688190464 G delta / mail: 1953 B 3) EXT4 with 2048 blocks: image file: 53687091200 1B-blocks: 50324295680 Used_begin: 82857984 Used_end: 41598846976 Available_begin: 47557083136 Available_end: 6041094144 mdir exact used space: 41323991040 mdir guess used space: 41516007424 mdir num mails: 3425033 delta: 3.201314816 G delta / mail: 934 B 4) EXT4 with 1024 blocks: image file: 53687091200 1B-blocks: 50314834944 Used_begin: 38287360 Used_end: 39909360640 Available_begin: 47592193024 Available_end: 7721119744 mdir exact used space: 39683908608 mdir guess used space: 39871086592 mdir num mails: 3425033 delta: 1.561232384 G delta / mail: 455 B As you can see, the delta per mail is rather close to the statistically expected values of 2048B, 1024B and 512B. In the end I probably changed my opinion. ~7GB of wasted block space for all my mails is actually quite a lot, but in days of cheap disk space it's acceptable. And with mbox one has IMHO the major disadvantage that mailservers (including dovecot) store some meta-data _in_ it (i.e. in the mails themselves) , which I don't like a lot. I still think about reports that mbox is much faster with full text search (which sounds reasonable)... but therefore one needs probably and database backend anyway. HTH, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 23:00:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:00:56 +0200 Subject: [Dovecot] v2.2.alpha1 released Message-ID: <1351544456.13571.102.camel@hurina> http://dovecot.org/releases/2.2/alpha/dovecot-2.2.alpha1.tar.gz http://dovecot.org/releases/2.2/alpha/dovecot-2.2.alpha1.tar.gz.sig I wanted to start stabilizing v2.2 release some months ago already, but I somehow got stuck adding more and more features. Now it looks like all of the necessary API changes are done, so everything I'm planning on near future can still be added to v2.2 without major changes. The redesigned dsync hasn't had much testing yet, so avoid running it with important mails. Would be nice if people started testing and reporting any bugs. I was going to create some kind of a test suite for testing all the possible syncing combinations and also doing some random stress testing, but I haven't had time for that yet. The new dsync supports doing very fast syncs by saving the resulting state and giving it as command line parameter to the next sync. So for example doveadm sync -s "" > new-state saves the state and doveadm sync -s `cat new-state` continues from the saved state. The replicator code doesn't yet support this. * When creating home directories, the permissions are copied from the parent directory if it has setgid-bit set. For full details, see http://wiki2.dovecot.org/SharedMailboxes/Permissions * "doveadm auth" command was renamed to "doveadm auth test" * IMAP: ID command now advertises server name as Dovecot by default. It was already trivial to guess this from command replies. + Implemented IMAP MOVE and BINARY extensions + Implemented IMAP CATENATE, URLAUTH and URLAUTH=BINARY extensions (by Stephan Bosch). + Implemented IMAP NOTIFY extension. Requires mailbox_list_index=yes to be enabled. + Redesigned and rewritten dsync. The new design makes the syncing faster, more reliable and more featureful. The new dsync protocol isn't backwards compatible with old dsync versions (but is designed to be forwards compatible with future versions). + All mailbox formats now support per-user message flags for shared mailboxes by using a private index. It can be enabled by adding :INDEXPVT= to mail location. This should be used instead of :INDEX also for Maildir/mbox to improve performance. + Improved mailbox list indexes. They should be usable now, although still disabled by default. + Added LAYOUT=index. The mailbox directories are created using their GUIDs in the filesystem, while the actual GUID <-> name mapping exists only in the index. + LMTP proxy: Implemented XCLIENT extension for passing remote IP address through proxy. From pw at wk-serv.de Mon Oct 29 23:05:42 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 29 Oct 2012 22:05:42 +0100 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <5083D963.3000700@wk-serv.de> References: <5083D963.3000700@wk-serv.de> Message-ID: <508EEFA6.1020506@wk-serv.de> Hi, no one here who is able to reply to my questions? Regards Patrick Patrick Westenberg schrieb: > Hi everyone, > > short version: > Is there no built in failover mechanism for the director service to > handle a backend failure? > > Long version: > I have a frontend server running the director service and two backends. > Due to maintenance I had to shut down one of the backends which caused > connection errors for the users being directed to this backend. > > I was very surprised as I expected the director to redirect these users > to the remaining backend. > > Am I wrong or is the director not working as expected? > > Regards > Patrick > > > > # 2.1.6: /usr/local/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 > auth_mechanisms = plain login > director_mail_servers = 172.17.1.1 172.17.1.2 > director_servers = 172.17.1.3 172.17.1.4 > lmtp_proxy = yes > log_path = /var/log/dovecot.log > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > protocols = imap pop3 lmtp sieve > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-userdb { > user = dovecot > } > } > service director { > fifo_listener login/proxy-notify { > mode = 0666 > } > inet_listener { > address = 172.17.1.3 > port = 9090 > } > unix_listener director-userdb { > mode = 0600 > } > unix_listener login/director { > mode = 0666 > } > } > service imap-login { > executable = imap-login director > } > service lmtp { > inet_listener lmtp { > address = 172.17.1.3 > port = 24 > } > } > service managesieve-login { > executable = managesieve-login director > inet_listener sieve { > port = 4190 > } > } > service pop3-login { > executable = pop3-login director > } > ssl_cert = ssl_key = protocol !smtp { > passdb { > args = proxy=y nopassword=y starttls=any-cert > driver = static > } > } > protocol smtp { > passdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > } > protocol lmtp { > auth_socket_path = director-userdb > } From tss at iki.fi Mon Oct 29 23:06:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:06:53 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351544049.3435.47.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> Message-ID: <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> On 29.10.2012, at 22.54, Christoph Anton Mitterer wrote: > I recently mentioned in several posts, that I'd tended to use mbox > rather than maildir, because you don't loose so much space (due to > always allocating full blocks per maildir file and thus per mail). .. > In the end I probably changed my opinion. > ~7GB of wasted block space for all my mails is actually quite a lot, but > in days of cheap disk space it's acceptable. > And with mbox one has IMHO the major disadvantage that mailservers > (including dovecot) store some meta-data _in_ it (i.e. in the mails > themselves) , which I don't like a lot. > I still think about reports that mbox is much faster with full text > search (which sounds reasonable)... but therefore one needs probably and > database backend anyway. There is of course mdbox also, which gives the best of both mbox and maildir (and some of its own new annoyances). From calestyo at scientia.net Mon Oct 29 23:09:11 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:09:11 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> Message-ID: <1351544951.3435.61.camel@fermat.scientia.net> Hi Timo. On Mon, 2012-10-29 at 22:40 +0200, Timo Sirainen wrote: > > I wondered, when using maildir, are there any headers that dovecote > > would treat specially, too? > > And which I therefore should strip? > > No. Maildir metadata is stored elsewhere. Great... and I expect that this ("no headers from the maildir files are interpreted") applies also, when one "imports" mails the first time. With import I don't mean via IMAP, but plainly moving e.g. a maildir++ tree under dovecots mail location. Then dovecot usually starts to generate all it's metadata,... and I expect that things like status and keywords are left simply unset... and things like UID and UIDVALIDITY are freshly initialised and not tried to be converted from the maildir files, right? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 23:11:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:11:15 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <508EEFA6.1020506@wk-serv.de> References: <5083D963.3000700@wk-serv.de> <508EEFA6.1020506@wk-serv.de> Message-ID: <6DFB1CD2-5FE6-405A-B2A8-545938A11F98@iki.fi> People already replied and pointed to poolmon. There is no built-in failure handling, because it's not possible to implement it in a way that works well for everyone. Although I think poolmon could also itself use a bit of tweaking. For example if all hosts became very heavily loaded, poolmon would now probably drop all of them immediately if one if its check connections failed. On 29.10.2012, at 23.05, Patrick Westenberg wrote: > Hi, > > no one here who is able to reply to my questions? > > Regards > Patrick > > > > Patrick Westenberg schrieb: >> Hi everyone, >> >> short version: >> Is there no built in failover mechanism for the director service to >> handle a backend failure? >> >> Long version: >> I have a frontend server running the director service and two backends. >> Due to maintenance I had to shut down one of the backends which caused >> connection errors for the users being directed to this backend. >> >> I was very surprised as I expected the director to redirect these users >> to the remaining backend. >> >> Am I wrong or is the director not working as expected? >> >> Regards >> Patrick >> >> >> >> # 2.1.6: /usr/local/etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 >> auth_mechanisms = plain login >> director_mail_servers = 172.17.1.1 172.17.1.2 >> director_servers = 172.17.1.3 172.17.1.4 >> lmtp_proxy = yes >> log_path = /var/log/dovecot.log >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> protocols = imap pop3 lmtp sieve >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0666 >> user = postfix >> } >> unix_listener auth-userdb { >> user = dovecot >> } >> } >> service director { >> fifo_listener login/proxy-notify { >> mode = 0666 >> } >> inet_listener { >> address = 172.17.1.3 >> port = 9090 >> } >> unix_listener director-userdb { >> mode = 0600 >> } >> unix_listener login/director { >> mode = 0666 >> } >> } >> service imap-login { >> executable = imap-login director >> } >> service lmtp { >> inet_listener lmtp { >> address = 172.17.1.3 >> port = 24 >> } >> } >> service managesieve-login { >> executable = managesieve-login director >> inet_listener sieve { >> port = 4190 >> } >> } >> service pop3-login { >> executable = pop3-login director >> } >> ssl_cert = > ssl_key = > protocol !smtp { >> passdb { >> args = proxy=y nopassword=y starttls=any-cert >> driver = static >> } >> } >> protocol smtp { >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> } >> protocol lmtp { >> auth_socket_path = director-userdb >> } > From tss at iki.fi Mon Oct 29 23:13:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:13:36 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351544951.3435.61.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> Message-ID: <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> On 29.10.2012, at 23.09, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 22:40 +0200, Timo Sirainen wrote: >>> I wondered, when using maildir, are there any headers that dovecote >>> would treat specially, too? >>> And which I therefore should strip? >> >> No. Maildir metadata is stored elsewhere. > > Great... and I expect that this ("no headers from the maildir files are > interpreted") applies also, when one "imports" mails the first time. > > With import I don't mean via IMAP, but plainly moving e.g. a maildir++ > tree under dovecots mail location. Yeah. > Then dovecot usually starts to generate all it's metadata,... and I > expect that things like status and keywords are left simply unset... and > things like UID and UIDVALIDITY are freshly initialised and not tried to > be converted from the maildir files, right? Flags are stored in the maildir filenames, so they're always preserved. Keywords, UIDs, UIDVALIDITY etc is preserved if you copy the dovecot-* files with it (which is a good idea to do). From calestyo at scientia.net Mon Oct 29 23:15:30 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:15:30 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> Message-ID: <1351545330.3435.66.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:06 +0200, Timo Sirainen wrote: > There is of course mdbox also, which gives the best of both mbox and maildir (and some of its own new annoyances). Thanks, Timo,... I forgot to mention that. For me _personally_ two things speak against using it: a) To be honest, "you must not lose the dbox index files, they can't be regenerated without data loss"[0] made me a bit scared ;-) b) ext* has no integrity checking (by hash sums) so I used to create my own that puts SHA512 hashes into the inodes of files (as USER_XATTRS). This of course, works only when you have a storage format where files don't change anymore once written,... which can't work with formats having multiple mails per file. Thanks, Chris. btw: What are the actual advantages of sdbox over maildir? [0] http://wiki2.dovecot.org/MailboxFormat/dbox -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Mon Oct 29 23:20:27 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:20:27 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> Message-ID: <1351545627.3435.71.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:13 +0200, Timo Sirainen wrote: > > Great... and I expect that this ("no headers from the maildir files > are > > interpreted") applies also, when one "imports" mails the first time. > > > > With import I don't mean via IMAP, but plainly moving e.g. a maildir > ++ > > tree under dovecots mail location. > > Yeah. So that means: From a "security" point of view, when using maildir (!) there's no need to remove such headers, cause dovcote ignores them (on maildir) always. Right?! I just wondered because when I looked through my mail archive (currently as mentioned, under Evolution)... many emails already had X-UID and X-IMAP* headers.... (set by the remote side, not by Evolution) ... and in no case these should be able to mess around in my dovecot :) > UIDs, UIDVALIDITY etc is preserved if you copy the dovecot-* files > with it (which is a good idea to do). I'll have a question on that too, but ask it under a separate mail in a few minutes,.. cause it doesn't fit this thread anymore ;) Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From limon at koli.be Mon Oct 29 23:26:48 2012 From: limon at koli.be (Levent Dane) Date: Mon, 29 Oct 2012 16:26:48 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> <20121029181700.GA4240@leningrad.koli.be> <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> Message-ID: <20121029212648.GA4292@leningrad.koli.be> On 10/29, Timo Sirainen wrote: >On 29.10.2012, at 20.17, Levent Dane wrote: > >> On 10/29, Timo Sirainen wrote: >>> On 18.10.2012, at 9.03, Levent Dane wrote: >>> >>>>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >>>> >>>> in Code/dovecot-virtual: >>>> Archive >>>> inthread refs keyword code not deleted >>> >>> I still couldn't reproduce with this. >> >> I think the problem is mail-search.c is corrupting the index files. > >Not that itself, but yeah looks like if virtual plugin assert-crashes in mail-search.c it leaves the indexes so that the next access will segfault. > >> I compiled with -ggdb flag. I'm getting this informations >> /var/log/messages: http://pastebin.com/bpkvp4Ak >> and from gdb: http://pastebin.com/HY0mVYBS > >Better backtrace than last time, but still no debug information in the backtrace. Maybe that got stripped somewhere between compiling and installing? You can check with "file ..../imap" to see if it's there. > >Also backtrace from both the mail-search.c assert crash and the segfault would be useful. I think I get correct backtrace. I attached this mail and uploaded pastebin. http://pastebin.com/L41e6AXY -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 -------------- next part -------------- Oct 29 16:21:40 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) Oct 29 16:21:40 widder dovecot: imap(limon): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x451b1) [0xb76911b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4521f) [0xb769121f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7660d4e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x3b845) [0xb770a845] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_search_result_update_flags+0xe3) [0xb77320d3] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_sync_search_results_update+0x69) [0xb77394f9] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mailbox_sync_deinit+0x1f5) [0xb7738855] -> /usr/lib/dovecot/lib20_fts_plugin.so(+0xa0a6) [0xb74970a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x3a) [0xb770f7fa] -> /usr/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0xbf2) [0xb7487ac2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x39) [0xb770f769] -> dovecot/imap(imap_sync_init+0x54) [0x8060294] -> dovecot/imap() [0x8052262] -> dovecot/imap(cmd_idle+0xc3) [0x80523f3] -> dovecot/imap(command_exec+0x3d) [0x80591cd] -> dovecot/imap() [0x805815f] -> dovecot/imap() [0x8058230] -> dovecot/imap(client_handle_input+0x12d) [0x805847d] -> dovecot/imap(client_input+0x5f) [0x8058daf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x42) [0xb769ff92] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd3) [0xb76a0f43] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0xb769fa30] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2d) [0xb768880d] -> dovecot/imap(main+0x2b1) [0x8061c71] -> /lib/libc.so.6(__libc_start_main+0xe7) [0xb74de573] -> dovecot/imap() [0x804fa51] Oct 29 16:21:40 widder dovecot: imap(limon): Fatal: master: service(imap): child 8060 killed with signal 6 (core dumped) -------------- next part -------------- #0 0xf57fe416 in __kernel_vsyscall () No symbol table info available. #1 0xb74f1a1a in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb74f3014 in abort () from /lib/libc.so.6 No symbol table info available. #3 0xb76911c5 in default_fatal_finish (type=, status=) at failures.c:191 backtrace = 0x8df75a8 "/usr/lib/dovecot/libdovecot.so.0(+0x451b1) [0xb76911b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4521f) [0xb769121f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7660d4e] -> /usr/lib/dovecot/libdo"... #4 0xb769121f in i_internal_fatal_handler (ctx=0xbfaff584, format=0xb7767320 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfaff5a4 "?v\267Z") at failures.c:649 status = 0 #5 0xb7660d4e in i_panic (format=0xb7767320 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = 0xbfaff5a4 "?v\267Z" #6 0xb770a845 in mail_search_args_init_sub (args=, arg=0x8e6cee8, change_uidsets=false, search_saved_uidset=0x0) at mail-search.c:90 thread_args = keywords = {0x8e6cf40 "lists", 0x0} __FUNCTION__ = "mail_search_args_init_sub" #7 0xb77320d3 in search_result_update_search (changed_uids_arr=0x8e87030, result=0x8e58ed0) at index-search-result.c:69 search_ctx = changed_uids = 0x8e46b30 next_uid = 29224 ret = t = mail = changed_count = 1 changed_idx = 0 #8 index_search_result_update_flags (result=0x8e58ed0, uids=0x8e87030) at index-search-result.c:131 search_arg = {next = 0x8e6cee8, type = SEARCH_UIDSET, value = {subargs = 0x0, seqset = {arr = { buffer = 0x8e740e8, element_size = 8}, v = 0x8e740e8, v_modifiable = 0x8e740e8}, str = 0x0, time = 0, size = 0, flags = 0, search_flags = 0, date_type = 0, thread_type = MAIL_THREAD_NONE, keywords = 0x0, modseq = 0x0, search_args = 0x0, search_result = 0x0, mailbox_glob = 0x0}, context = 0x0, hdr_field_name = 0x0, match_not = 0, match_always = 0, nonmatch_always = 0, fuzzy = 0, result = 0} ret = 0 __FUNCTION__ = "index_search_result_update_flags" #9 0xb77394f9 in search_result_update (result=0x8e58ed0, ctx=0x8e87010) at index-sync-search.c:75 No locals. #10 index_sync_search_results_update (ctx=0x8e87010) at index-sync-search.c:88 results = 0x9060740 i = count = 3 #11 0xb7738855 in index_mailbox_sync_deinit (_ctx=0x8e87010, status_r=0xbfaffa3c) at index-sync.c:386 ctx = 0x8e87010 sync_rec = {seq1 = 3077094660, seq2 = 148987872, type = 0} delayed_expunges = false ret = 0 #12 0xb74970a6 in fts_sync_deinit (ctx=0x8e87010, status_r=0xbfaffa3c) at fts-storage.c:584 box = 0x9060580 fbox = 0x9060898 flist = 0x8e16060 ret = 0 #13 0xb770f7fa in mailbox_sync_deinit (_ctx=0xbfaffa40, status_r=0xbfaffa3c) at mail-storage.c:1347 ctx = box = 0x9060580 errormsg = error = ret = #14 0xb7487ac2 in virtual_sync_backend_box_sync (sync_flags=, bbox=0x8e632a8, ctx=0x8e8dda8) at virtual-sync.c:973 uidmap = sync_rec = {seq1 = 22114, seq2 = 22114, type = MAILBOX_SYNC_TYPE_FLAGS} idx1 = vuid = sync_ctx = 0x0 sync_status = {sync_delayed_expunges = 0} idx2 = vseq = 149273152 #15 virtual_sync_backend_box (bbox=0x8e632a8, ctx=0x8e8dda8) at virtual-sync.c:1067 sync_flags = status = {messages = 3077174859, recent = 148890672, unseen = 152726112, uidvalidity = 3215980904, uidnext = 3077353460, first_unseen_seq = 149046960, first_recent_uid = 149450720, last_cached_seq = 543664, highest_modseq = 13217152038154990465, keywords = 0x8df63a0, permanent_flags = 3077174635, nonpermanent_modseqs = 0, permanent_keywords = 0, allow_new_keywords = 1} ret = #16 virtual_sync_backend_boxes (ctx=0x8e8dda8) at virtual-sync.c:1399 bboxes = 0x9079798 i = count = 1 #17 virtual_sync (flags=0, mbox=0x8e62e18) at virtual-sync.c:1496 ctx = 0x8e8dda8 index_sync_flags = ret = #18 virtual_storage_sync_init (box=0x8e62e18, flags=0) at virtual-sync.c:1516 mbox = 0x8e62e18 sync_ctx = ret = #19 0xb770f769 in mailbox_sync_init (box=0x8e62e18, flags=0) at mail-storage.c:1324 _data_stack_cur_id = 4 ctx = #20 0x08060294 in imap_sync_init (client=0x8e17628, box=0x8e62e18, imap_flags=0, flags=0) at imap-sync.c:142 ctx = 0x8e5ba40 __FUNCTION__ = "imap_sync_init" #21 0x08052262 in idle_sync_now (box=, ctx=0x8e17eb8) at cmd-idle.c:145 __FUNCTION__ = "idle_sync_now" #22 0x080523f3 in cmd_idle (cmd=0x8e17e30) at cmd-idle.c:276 client = 0x8e17628 ctx = 0x8e17eb8 #23 0x080591cd in command_exec (cmd=0x8e17e30) at imap-commands.c:148 hook = 0x8dff260 ret = #24 0x0805815f in client_command_input (cmd=0x8e17e30) at imap-client.c:682 client = 0x8e17628 command = __FUNCTION__ = "client_command_input" #25 0x08058230 in client_command_input (cmd=0x8e17e30) at imap-client.c:733 client = 0x8e17628 command = __FUNCTION__ = "client_command_input" #26 0x0805847d in client_handle_next_command (remove_io_r=, client=0x8e17628) at imap-client.c:774 size = 12 #27 client_handle_input (client=0x8e17628) at imap-client.c:786 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #28 0x08058daf in client_input (client=0x8e17628) at imap-client.c:825 cmd = output = 0x8e16bfc bytes = 12 __FUNCTION__ = "client_input" #29 0xb769ff92 in io_loop_call_io (io=0x8f49090) at ioloop.c:379 ioloop = 0x8dfe400 t_id = 2 #30 0xb76a0f43 in io_loop_handler_run (ioloop=0x8dfe400) at ioloop-epoll.c:213 ctx = 0x8dfe5e0 events = event = 0x8dfe620 list = 0x8e16c90 io = tv = {tv_sec = 1791, tv_usec = 756031} events_count = 148991120 msecs = 1 ret = 1 i = j = call = #31 0xb769fa30 in io_loop_run (ioloop=0x8dfe400) at ioloop.c:398 No locals. #32 0xb768880d in master_service_run (service=0x8dfe330, callback=0x80612f0 ) at master-service.c:543 No locals. #33 0x08061c71 in main (argc=1, argv=0x8dfe1c0) at main.c:389 set_roots = {0x80645e0, 0x0} login_set = {auth_socket_path = 0x8df6060 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x8061720 , failure_callback = 0x8061430 } service_flags = storage_service_flags = username = c = From tss at iki.fi Mon Oct 29 23:39:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:39:33 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351545627.3435.71.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> Message-ID: <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> On 29.10.2012, at 23.20, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:13 +0200, Timo Sirainen wrote: >>> Great... and I expect that this ("no headers from the maildir files >> are >>> interpreted") applies also, when one "imports" mails the first time. >>> >>> With import I don't mean via IMAP, but plainly moving e.g. a maildir >> ++ >>> tree under dovecots mail location. >> >> Yeah. > So that means: From a "security" point of view, when using maildir (!) > there's no need to remove such headers, cause dovcote ignores them (on > maildir) always. Right?! Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). > I just wondered because when I looked through my mail archive (currently > as mentioned, under Evolution)... many emails already had X-UID and > X-IMAP* headers.... (set by the remote side, not by Evolution) ... and > in no case these should be able to mess around in my dovecot :) If you migrated from mbox format it could have brought those headers to maildir. They're anyway not used for anything by Dovecot. From tss at iki.fi Mon Oct 29 23:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:42:28 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351545330.3435.66.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> Message-ID: <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: > btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. From jk at jkart.de Mon Oct 29 23:43:08 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 22:43:08 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> Message-ID: <508EF86C.5070202@jkart.de> am 29.10.12 16:41 schrieb Timo Sirainen : > On 29.10.2012, at 17.23, Jim Knuth wrote: > >> I have here a problem with managesieve. With the login about >> webmail (roundcube) comes here in the log: > > You can always easily reproduce this? Can you get the network traffic logs between Roundcube and Dovecot and reproduce it by sending those same commands manually? > >> --snip >> dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) > > Looks like there's a bug somewhere.. Doesn't it log a "raw backtrace"? > >> --snap >> I've increased of 2048M and the same above. >> then with 4096 M happens the following >> >> --snip >> dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) >> --snap > > Probably related to the first error. It would be helpful to get gdb backtraces from both of them, although from the first one you couldn't without patching + recompiling Dovecot. But from the second one I think you can get a core dump with: > > service managesieve-login { > executable = managesieve-login -D > } > > Then you can do something like: > > gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/managesieve-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/managesieve-login...done. (no debugging symbols found)...done. /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. (gdb) > bt full bt full No stack. (gdb) > -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning. [Churchill] From tss at iki.fi Mon Oct 29 23:46:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:46:05 +0200 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508EF86C.5070202@jkart.de> References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> Message-ID: On 29.10.2012, at 23.43, Jim Knuth wrote: > ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core > /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. You'll of course need to have the core file first. Instead of: >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) It should say (core dumped). Besides the executable change, you'll need to run "ulimit -c unlimited" just before dovecot binary. From calestyo at scientia.net Mon Oct 29 23:52:54 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:52:54 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> Message-ID: <1351547574.3435.74.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:39 +0200, Timo Sirainen wrote: > Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). Great... I think it would worth adding all this to: http://wiki2.dovecot.org/MailboxFormat/Maildir Is the wiki open for public editing (after registering an account)? Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Mon Oct 29 23:54:42 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:54:42 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> Message-ID: <1351547682.3435.76.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: > > btw: What are the actual advantages of sdbox over maildir? > > * Not moving files from new/ to cur/ directory > * Not renaming files when changing message flags > * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) > > Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. Oh that's quite some advantage... And I guess the interior of the files is the same? I.e. just the plain mail without any changes or quoting? For sdbox, does that part with "loosing the indexes means game over" ;) , too? Thanks, Chris -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jk at jkart.de Mon Oct 29 23:58:42 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 22:58:42 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> Message-ID: <508EFC12.4000509@jkart.de> am 29.10.12 22:46 schrieb Timo Sirainen : > On 29.10.2012, at 23.43, Jim Knuth wrote: > >> ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core >> /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. > > You'll of course need to have the core file first. Instead of: > >>> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) > > It should say (core dumped). Besides the executable change, you'll need to run "ulimit -c unlimited" just before dovecot binary. > If I run "ulimit -c unlimited" no problems more with Managesieve Login over Roundcube: Oct 29 22:50:46 srv1 dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=9354, secured, session= Oct 29 22:50:46 srv1 dovecot: managesieve(web1p1): Disconnected: Logged out bytes=120/177 Oct 29 22:53:16 srv1 dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=9418, secured, session= Oct 29 22:53:16 srv1 dovecot: managesieve(web1p1): Disconnected: Logged out bytes=44/145 But the same: srv1:~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/managesieve-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/managesieve-login...done. (no debugging symbols found)...done. /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. (gdb) bt full No stack. (gdb) q -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Ein Tag an dem Du nicht l?chelst, ist ein verlorener Tag. (Charly Chaplin) From tss at iki.fi Tue Oct 30 00:05:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 00:05:42 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351547682.3435.76.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <1351547682.3435.76.camel@fermat.scientia.net> Message-ID: On 29.10.2012, at 23.54, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: >>> btw: What are the actual advantages of sdbox over maildir? >> >> * Not moving files from new/ to cur/ directory >> * Not renaming files when changing message flags >> * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) >> >> Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. > > Oh that's quite some advantage... > > And I guess the interior of the files is the same? I.e. just the plain > mail without any changes or quoting? Yes, but it's in dbox format so it contains also some extra metadata (not in the mail headers). > For sdbox, does that part with "loosing the indexes means game > over" ;) , too? You'll lost message flags then. Both sdbox and mdbox keep dovecot.index.backup files and repairing tries very hard to preserve everything from the indexes it sees, so I don't think it's a big concern as long as the system behaves properly. From tss at iki.fi Tue Oct 30 00:08:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 00:08:28 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351547574.3435.74.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> Message-ID: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> On 29.10.2012, at 23.52, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:39 +0200, Timo Sirainen wrote: >> Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). > > Great... I think it would worth adding all this to: > http://wiki2.dovecot.org/MailboxFormat/Maildir Well, that isn't really maildir-specific. It's pop3 specific that is done with all mailbox formats. pop3_reuse_xuidl setting's comments should probably warn about the possibility of receiving unwanted X-UIDL headers in new mails. > Is the wiki open for public editing (after registering an account)? Yes. You don't even need to register. From calestyo at scientia.net Tue Oct 30 00:16:55 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 23:16:55 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> Message-ID: <1351549015.3435.80.camel@fermat.scientia.net> On Tue, 2012-10-30 at 00:08 +0200, Timo Sirainen wrote: > > Great... I think it would worth adding all this to: > > http://wiki2.dovecot.org/MailboxFormat/Maildir > > Well, that isn't really maildir-specific. It's pop3 specific that is > done with all mailbox formats. pop3_reuse_xuidl setting's comments > should probably warn about the possibility of receiving unwanted > X-UIDL headers in new mails. No I meant _everything_.. i.e. that dovecote never interprets these message headers when using maildir... unless for that one case when using POP3 on maildir and when pop3_reuse_xuidl is set to yes. I'll make some chances and post you the diff links here, so you can check this and correct if something's wrong. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From janfrode at tanso.net Tue Oct 30 00:26:29 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 29 Oct 2012 23:26:29 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> +1 Better to be lenient, than to confuse users by accepting some but not other messages. I believe most larger mail providers has a max message size of around 64MB or less, so allowing the final message to exceed quota by about that sounds reasonable to me. -jf From calestyo at scientia.net Tue Oct 30 00:31:20 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 23:31:20 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> Message-ID: <1351549880.3435.81.camel@fermat.scientia.net> I think it should be configurable by how much (either a fixed space or relative to the quota) the last mail may be larger than the quota.... but then... +1 as well :) Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Tue Oct 30 01:13:45 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 00:13:45 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> Message-ID: <1351552425.3435.83.camel@fermat.scientia.net> Please have a look at: http://master.wiki2.dovecot.org/MailboxFormat/mbox?action=diff&rev2=17&rev1=16 http://master.wiki2.dovecot.org/MailboxFormat/Maildir?action=diff&rev2=45&rev1=44 whether it's correct. Oh and... I'd assume that everything I've added for maildir also applies to the dbox formats? If so, I'd add the text there, too. Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From bob at computerisms.ca Tue Oct 30 01:23:16 2012 From: bob at computerisms.ca (Bob Miller) Date: Mon, 29 Oct 2012 16:23:16 -0700 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <1351552996.2097.57.camel@worklian> +1 to one last mail, though it would be nice if the over percentage could be configurable... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. > From sven at svenhartge.de Tue Oct 30 01:36:08 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 30 Oct 2012 00:36:08 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail > that is over 1MB, Dovecot rejects the mail. But smaller mails aren't > rejected probably for days. So user might not even realize that they > didn't receive one of the mails. Also having a user "almost over > quota" is a rather strange state I think. > So what do you think about v2.2 allowing delivery of one last mail > even if it brings the user over quota? Except add a limit that if the > message size is as much as the user's entire quota limit it wouldn't > be added (or 50% or ..?). Also IMAP wouldn't allow this, since user > would get an error anyway. I could make this also optional, but if > nobody really wants to keep the old behavior there's really no point > in adding the option. Yes, please add this new option. If possible with configurable limit. I'd rather have a user go directly over quota with one final mail than have a situation where half the mails get delivered and the other half is rejected. >From a 1st level support stand point this new behavior is easier to explain than the way it is now. By looking into my new crytal ball I can see the following happening: A user with 300KBytes under his quota gets a mail with 500KBytes in size. This of course bounces. He is then called by the sender who complains about the full mailbox. The user then sends himself a test mail (Subject: Test, Body: Test) which is delivered, because it is rather small and fits inside the few bytes left. The user then is confused. (And I have to use some of my precious time to explain to the user the inner workings of the mail system. ;)) So I'd very much appreciate such an option. Gr??e, Sven. -- Sigmentation fault. Core dumped. From noel.butler at ausics.net Tue Oct 30 01:43:30 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 30 Oct 2012 09:43:30 +1000 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <1351554210.7884.5.camel@tardis> On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? +1 only if configurable, and with an additional configurable quota percentage value option for those that do enable the function. In 99.9% of cases I could never see a service provider wanting this, but some small private businesses perhaps might see a benefit in it. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From sven at svenhartge.de Tue Oct 30 01:48:32 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 30 Oct 2012 00:48:32 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1351554210.7884.5.camel@tardis> Message-ID: Noel Butler wrote: > On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: >> So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? > +1 only if configurable, and with an additional configurable quota > percentage value option for those that do enable the function. > In 99.9% of cases I could never see a service provider wanting this, > but some small private businesses perhaps might see a benefit in it. If your user quota is 1GiB (which is not big, if you look at todays user quotas even at freemail providers) and the max mail size 30MiB, then a users max mailbox size would then be 1054MiB. Not an unreasonable price to pay for an easier to understand error condition, IMHO. Gr??e, Sven. -- Sigmentation fault. Core dumped. From calestyo at scientia.net Tue Oct 30 02:16:05 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 01:16:05 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <1351547682.3435.76.camel@fermat.scientia.net> Message-ID: <1351556165.3435.88.camel@fermat.scientia.net> On Tue, 2012-10-30 at 00:05 +0200, Timo Sirainen wrote: > > And I guess the interior of the files is the same? I.e. just the plain > > mail without any changes or quoting? > Yes, but it's in dbox format so it contains also some extra metadata (not in the mail headers). Yeah of course... but the important point here is the "not in the mail headers" part :) So I've added the following changes, please double check :) http://master.wiki2.dovecot.org/MailboxFormat/dbox?action=diff&rev2=30&rev1=29 > > For sdbox, does that part with "loosing the indexes means game > > over" ;) , too? > You'll lost message flags then. Both sdbox and mdbox keep > dovecot.index.backup files and repairing tries very hard to preserve > everything from the indexes it sees, so I don't think it's a big > concern as long as the system behaves properly. Yeah... sounds not too bad... :) Off topic: Have you ever thought about adding a "real" DB backend? Nothing against dbox... :) ... and I have no performance comparison of dbox with what could be done with a DBMS... but the advantage of the later would be that you get all fancy features from database systems for free... like fast indexing, online replication, etc. p.. One might even reuse something like AOX for this. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From noel.butler at ausics.net Tue Oct 30 02:27:58 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 30 Oct 2012 10:27:58 +1000 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1351554210.7884.5.camel@tardis> Message-ID: <1351556878.7884.20.camel@tardis> On Tue, 2012-10-30 at 00:48 +0100, Sven Hartge wrote: > Noel Butler wrote: > > On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > > >> So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? > > > +1 only if configurable, and with an additional configurable quota > > percentage value option for those that do enable the function. > > > In 99.9% of cases I could never see a service provider wanting this, > > but some small private businesses perhaps might see a benefit in it. > > If your user quota is 1GiB (which is not big, if you look at todays user > quotas even at freemail providers) and the max mail size 30MiB, then a > users max mailbox size would then be 1054MiB. > > Not an unreasonable price to pay for an easier to understand error > condition, IMHO. > Sven , That's nice when it's one or ten, but you need to look at the big picture, what about 300K users, all doing the same. Also, as to mail sizes, in decades gone by with dialup it was 5mb, now days with DSL, Cable, FTTN etc, many that I know of use 50mb mail sizes because that takes mere seconds now days. Don't forget, in some countries, hardware is still incredibly (criminally) overpriced, a 600G drive from HP in the U.S. is about 350 odd last time I looked, probably lot cheaper now, in this country (AU), the same drive today is still around 800, and that was when our dollar was 1.07 to the U.S. 1.00, even with taxes and customs and transport, some so and so's are still making an absolute massive killing in profits. Of course the more appropriate way would be like most of us do now, send the warning messages, if the users can not be bothered to keep an eye on their quota or act when they get mailbox almost/now full warnings, why is it our problem :) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From calestyo at scientia.net Tue Oct 30 02:42:25 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 01:42:25 +0100 Subject: [Dovecot] mbox2mdir... what about UIDs/etc? (was: how to best import Evolution/Thunderbird mail into dovecot?) In-Reply-To: References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1351557745.3435.106.camel@fermat.scientia.net> Hi again :) In the meantime I made some checks[0] on how much storage one looses by using maildir (compared to mbox)... and decided that it's much but I can live with it. This of course doesn't solve my problems that I have a possibly a mix of different mbox subformats, a mix of different mail status formats (Thunderbird and Evolution)... and some 17k mails that suffered from From_ line corruption (due to Evolution, getmail and postfix either incorrectly quoting them or even intentionally using mboxo)... so I'll still need some scripting in the end. Which I'll base upon mb2md[1] respectively it's Dovecot-izsed version[2]. I diffed the two, and it seems the only differences are that the later handles the following in addition: 1) keywords (via X-IMAP, X-IMAPbase and X-Keywords) 2) UIDs, UIDVALITIDYs and UIDLASTs (via the X-IMAP, X-IMAPbase and X-UID mail headers of the mboxes 3) ,S= and ,W= tags (Guess that's it right?) Now I have some questions: to 1) I never used keywords on mails myself so far,... so if any X-Keywords headers exist, these were sent from remote. So I guess I _really want_ to ignore them (and not let remote people set my local keywords), right? to 2) I haven't had time yet to read into the IMAP4 RFC (though I'll need to do so soon),... but AFAIU the UIDs, UIDVALITIDYs and UIDLASTs are used for the server/clients to identify which message they talk about and avoid unnecessary reloading and to assure statuses are set on the right message, etc. All mails that I migrate were only used locally by one client. So I guess I can fully ignore any UID/UIDVALITIDY/UIDLAST preservation, right? So in principle I can use plain mb2md (without the dovecot mods)... and simply convert all my mboxes to maildir, put them in the dovecot mail (having the mails in the ../new dirs) location and start dovecot, right? Now will dovecot itself assign fresh consecutive UIDs to all maildir files? Or will I get into troubles? to 3) If dovecot can make use of these,.. I'm happy with having them set, but analogous to (2): If I use plain mb2md (without the dovecot mods)... and simply convert all my mboxes to maildir, put them in the dovecot mail (having the mails in the ../new dirs) location and start dovecot.... Can I make dovecot to calculate these fields by itself when it loads? Thanks, Chris. [0] http://dovecot.org/pipermail/dovecot/2012-October/069130.html [1] http://batleth.sapienti-sat.org/projects/mb2md/ [2] http://dovecot.org/tools/mb2md.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jtam.home at gmail.com Tue Oct 30 04:09:12 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 29 Oct 2012 19:09:12 -0700 (PDT) Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: Ben Morrow wrote: >> Maybe replace "/usr/bin/passwd" with htpasswd? > > Try pam_pwdfile with poppwd or some other poppassd that supports PAM. That's it! I was trying to remember the name of this PAM module. >>> and is there another way other than poppassd? >> >> Write your own PHP script -- it couldn't be more than a few dozen lines >> of code for a working skeleton. Or Google "php change password htpasswd". > > It's not as simple as you seem to think. Quite apart from getting the > password-changing itself right (have you considered what happens when > two users change their passwords at the same time? when Dovecot tries to > read the password file at the same time as you are changing it? when the > system crashes when you are halfway through rewriting the password > file?), you really shouldn't be running PHP as a user with write access > to a password file (even a virtual password file) in any case. I did consider it, and you're right, it is tricky to get it absolutely right. If robusteness and security was of utmost importance, I would abandon PHP too. I was scaling the solution to the OP's technical ability and apparent size of their operation -- if poppwd passes muster, this wouldn't be too far off. Joseph Tam From tony.blue.mailinglist at gmx.de Tue Oct 30 07:33:22 2012 From: tony.blue.mailinglist at gmx.de (tony.blue.mailinglist at gmx.de) Date: Tue, 30 Oct 2012 06:33:22 +0100 Subject: [Dovecot] dovecot-lda not correct folder Message-ID: <508F66A2.7010809@gmx.de> Hello, i use dovecot with maildir. The maildir-folder looks like this: vmail/mail/user1/cur vmail/mail/user1/.Sent vmail/mail/user1/.optionalfolder Procmail should put some definded mails in the "optionalfolder" My uses configuration does not do this: ... DELIVERMAIL="/usr/lib/dovecot/dovecot-lda" IMAP="$DELIVERMAIL -e -d $LOGNAME -m INBOX" ZUSATZORDNER="$DELIVERMAIL -e -d $LOGNAME -m .optionalfolder" ... dovecot-lda puts the mails for the optionalfolder always in the .cur (INBOX). What?s the correct dovecot-lda parameter to put the mails in the optionalfolder? Thank you! Tony From slusarz at curecanti.org Tue Oct 30 09:19:07 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 30 Oct 2012 01:19:07 -0600 Subject: [Dovecot] Save/restore IMAP session state In-Reply-To: <1351529580.13571.93.camel@hurina> References: <1351529580.13571.93.camel@hurina> Message-ID: <20121030011907.Horde.5xjiGoF5lbhQj39rg9FXuZA@bigworm.curecanti.org> Quoting Timo Sirainen : > A week ago I thought I'd see > how easy it would be to implement this. I got a basic proof of concept > working as a "X-STATE" command. [snip] > This could also be used to implement quick session state restoring for > webmails (as suggested by Michael Slusarz). Wow. We must have some sort of crazy mind-meld going on: I have been working on this concept the last few days with the idea of generating some sort of draft proposal to provoke further discussion going forward. I would say great minds think alike, but that would be giving myself too much credit. So this provided the necessary motivation to finish the draft concept. Now that I have (somewhat) figured out the RFC 2629 XML format for doing this kind of thing, hopefully this has is presented in a somewhat coherent format. The draft, which is significantly more comprehensive than your suggestion, can be found here: https://raw.github.com/slusarz/horde-sandbox/master/imap-state-draft/draft-imap-state-00.txt Not sure if this should remain the forum for discussing this concept, or if we should move to private messages (or even to the imap-protocol list). Let me know your thoughts on this (or anyone else with an interest). michael From crohmann at netcologne.de Tue Oct 30 09:53:06 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 30 Oct 2012 08:53:06 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <508F8762.4040109@netcologne.de> On 29.10.2012 21:39, Timo Sirainen wrote: > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. Great idea. This makes being over quota a stable state and makes it easier for users to understand their "problem". Regards Christian From zybi at talex.pl Tue Oct 30 11:17:03 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Tue, 30 Oct 2012 10:17:03 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <508F9B0F.30108@talex.pl> Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. > This will finally make possible to reject RCPT TO: before the message size is known instead of accepting the message and sending a bounce later (bouncing SPAM is not good). -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From Ralf.Hildebrandt at charite.de Tue Oct 30 11:42:36 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 30 Oct 2012 10:42:36 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> Message-ID: <20121030094236.GG25787@charite.de> * Jan-Frode Myklebust : > > > +1 > > Better to be lenient, than to confuse users by accepting some but not other messages. Amen to that! +1 -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From cr at sys4.de Tue Oct 30 12:11:14 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 11:11:14 +0100 Subject: [Dovecot] copymail deleted Message-ID: Hi, I had enabled an option in dovecot. mail_attachment_dir = /var/mail/virtual/copymail/attachments After a while I checked /var/mail/virtual and did some cleanup. I did not remember that copymail was specified in dovecot and erased it. Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: file_istream.stat(/var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6) failed: No such file or directory Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: istream-concat: Failed to get size of stream /var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: read() failed: Invalid argument (FETCH for mailbox INBOX UID 196) Oct 30 10:56:05 mx0 dovecot: imap(hidden): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-30 10:56:05] in=150 out=950 I have Bacula and have restored most of the stuff, but obviously not all files. That is not too important. But I do not know, how to tell dovecot that it may "forget" about files that produce a "No such file or directory" error. Can I do some "rescan/rebuild" in dovecot? Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Tue Oct 30 12:19:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 12:19:31 +0200 Subject: [Dovecot] copymail deleted In-Reply-To: References: Message-ID: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> On 30.10.2012, at 12.11, Christian R??ner wrote: > Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: file_istream.stat(/var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6) failed: No such file or directory > > I have Bacula and have restored most of the stuff, but obviously not all files. That is not too important. But I do not know, how to tell dovecot that it may "forget" about files that produce a "No such file or directory" error. > > Can I do some "rescan/rebuild" in dovecot? Currently you can't in any easy way. The easiest fix for now I think would be to write a script that reads through dbox files, parses the attachment metadata lines and recreates the missing files with the original size (e.g. sparse-0-filled). The dbox parsing can be done easily with: doveadm dump m.1 | grep ^msg.ext-ref The format is: 1*( ) If the options="-" then the byte count is the final size. If options="B" then byte count is the base64-encoded size while the original file has to be base64-decoded size. From ef at math.uni-bonn.de Tue Oct 30 12:42:36 2012 From: ef at math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) Date: Tue, 30 Oct 2012 11:42:36 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <20121030104235.GL15430@trav.math.uni-bonn.de> Sounds like a reasonable idea, but one has to keep in mind that file system quotas never work that way. So that change would make quota=fs behave differently from the rest. So it should at least be configurable, I think. From sheng-wei.lim at proximityjobs.com Tue Oct 30 11:51:55 2012 From: sheng-wei.lim at proximityjobs.com (sheng-wei.lim) Date: Tue, 30 Oct 2012 17:51:55 +0800 Subject: [Dovecot] Problem about SSL for Dovecot. Message-ID: <000001cdb684$34e16f20$9ea44d60$@proximityjobs.com> Hi All, With the below setting (cropped), the ssl certificate(signed by godaddy) don?t seems to work. It will still ask me if I want to accept this certificate. I have use the same certificate for apache host and postfix it works without any prompt. dovecot version : 2.0.19 # OS: Linux 3.2.0-32-generic-pae i686 Ubuntu 12.04.1 LTS Dovecot config: ssl = required ssl_cert = References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> Message-ID: <508FB360.5090704@Media-Brokers.com> On 2012-10-29 5:42 PM, Timo Sirainen wrote: > On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: > >> btw: What are the actual advantages of sdbox over maildir? > * Not moving files from new/ to cur/ directory > * Not renaming files when changing message flags > * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) > > Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. > I had been wanting to ask about this too... So... what are the disadvantages? -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Oct 30 13:03:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 30 Oct 2012 07:03:02 -0400 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351544049.3435.47.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> Message-ID: <508FB3E6.6030304@Media-Brokers.com> On 2012-10-29 4:54 PM, Christoph Anton Mitterer wrote: > In the end I probably changed my opinion. > ~7GB of wasted block space for all my mails is actually quite a lot, but > in days of cheap disk space it's acceptable. > And with mbox one has IMHO the major disadvantage that mailservers > (including dovecot) store some meta-data_in_ it (i.e. in the mails > themselves) , which I don't like a lot. > I still think about reports that mbox is much faster with full text > search (which sounds reasonable)... but therefore one needs probably and > database backend anyway. What makes the most sense for me is to use mbox (or mdbox) for longer term storage that you may be offloading to slower storage systems, and use maildir (or sdbox) for the new mails... Would work great as long as you have a reliable method for archiving older mails out to your slower storage. This is what I plan on doing someday... -- Best regards, Charles From simon.buongiorno at gmail.com Tue Oct 30 13:11:37 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Tue, 30 Oct 2012 07:11:37 -0400 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <20121030094236.GG25787@charite.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> <20121030094236.GG25787@charite.de> Message-ID: On Oct 30, 2012 5:43 AM, "Ralf Hildebrandt" wrote: > > * Jan-Frode Myklebust : > > > > > > +1 > > > > Better to be lenient, than to confuse users by accepting some but not other messages. > > Amen to that! +1 Surely the answer is that as soon as any mail is rejected an over-quota message is injected? That way, the quota remains as it currently is, but the user will a) be aware that he's over or nearly over quota, b) that a mail was rejected for being too big (if you inject the right over-quota message). Simon From calestyo at scientia.net Tue Oct 30 13:30:29 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 12:30:29 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <508FB360.5090704@Media-Brokers.com> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <508FB360.5090704@Media-Brokers.com> Message-ID: <1351596629.7808.5.camel@heisenberg.scientia.net> On Tue, 2012-10-30 at 07:00 -0400, Charles Marcus wrote: > So... what are the disadvantages? I (but I'm no expert) would guess that it's a dovecot-only format. No support from most other tools,... I'd guess you cannot use e.g. maildrop with it, or can you? I personally was always a bit worried, when meta-data is put in the mail... now AFAIU dbox does _not_ do this... and you can cleanly extract each unmodified mail from the dbox fail (single or multi), right? Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From calestyo at scientia.net Tue Oct 30 13:31:40 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 12:31:40 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <508FB3E6.6030304@Media-Brokers.com> References: <1351544049.3435.47.camel@fermat.scientia.net> <508FB3E6.6030304@Media-Brokers.com> Message-ID: <1351596700.7808.6.camel@heisenberg.scientia.net> On Tue, 2012-10-30 at 07:03 -0400, Charles Marcus wrote: > What makes the most sense for me is to use mbox (or mdbox) for longer > term storage that you may be offloading to slower storage systems, and > use maildir (or sdbox) for the new mails... Was also something I thought about... still the more I think about it, the more I hate, that with mbox meta-data is stored in the mails. > Would work great as long as you have a reliable method for archiving > older mails out to your slower storage. I still hope for some DB backend ;) Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From cgregoir99 at yahoo.com Tue Oct 30 14:01:47 2012 From: cgregoir99 at yahoo.com (Christian Gregoire) Date: Tue, 30 Oct 2012 12:01:47 +0000 (GMT) Subject: [Dovecot] POP3 Proxy : user format not accepted Message-ID: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Hello, I want to use Dovecot as a POP3 proxy (http://wiki.dovecot.org/HowTo/ImapProxy). All is working fine on my sample platform, except that I have plenty (several thousands) of users that login using local_part#domain, instead of local_part at domain, which is an old setting on my POP3 server. And in that case, Dovecot returns 'Authentication failed'. Here is my proxy table : mysql> select * from tbl_proxy; +--------------------+-------------+--------------------+ | user ? ? ? ? ? ? ? | host ? ? ? ?| destuser ? ? ? ? ? | +--------------------+-------------+--------------------+ | christian at mydom.fr | 10.10.100.1 | christian at mydom.fr | | christian#mydom.fr | 10.10.100.1 | christian at mydom.fr | +--------------------+-------------+--------------------+ If I login on the Dovecot proxy with the '@' version, everything is fine : root : ~> telnet?10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian at mydom.fr +OK pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) And MySQL logs show the query : 121030 12:55:28 ? ? 3 Query ? ? SELECT NULL AS password, host, destuser, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS proxy, 'Y' AS nopassword FROM tbl_proxy WHERE user = 'christian at mydom.fr' If I login on the Dovecot proxy with the '#' version, it fails : root : ~> telnet 10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian#mydom.fr +OK pass azerty42 -ERR Authentication failed. And nothing shows up the the MySQL logs. If I login directly on the POP server with the same credentials, no problem : root : ~> telnet?10.10.100.1 110 Trying 10.10.100.1... Connected to?10.10.100.1?(10.10.100.1). Escape character is '^]'. +OK Welcome to POP3 Server V 2.06. Authenticate yourself. user christian#mydom.fr +OK Password required for christian#mydom.fr pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) Any idea ? Is the '#' not internally supported in the user login by Dovecot ? Thanks in advance. Christian From tss at iki.fi Tue Oct 30 15:08:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 15:08:37 +0200 Subject: [Dovecot] POP3 Proxy : user format not accepted In-Reply-To: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> References: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Message-ID: On 30.10.2012, at 14.01, Christian Gregoire wrote: > Any idea ? Is the '#' not internally supported in the user login by Dovecot ? See auth_username_chars setting. From chris at dotchristopher.com Tue Oct 30 15:03:55 2012 From: chris at dotchristopher.com (Chris Smith) Date: Tue, 30 Oct 2012 14:03:55 +0100 Subject: [Dovecot] Dovecot does not update acl_shared_dict file Message-ID: <20121030140355.Horde.LhzrQUVMXLlQj9A7c15yx4A@www.dotchristopher.com> Hi all, Firstly, thanks for all your effort with this software. Much appreciated. I am having a slight issues trying to enable reading of other users mailboxes. The docs are a little sparse for those that aren't mailadmin heros, I wonder if anyone could please help me see where I am going wrong. I would like to allow some users to list and read the mailboxes of others. E.g: When User1 logs in, they are presented with a list of their own folders, and those of User2, User3, etc to which they have (e.g. read/list) access I followed the instructions on the wiki (for Version 1.X)- there are two concepts here: 1. Grant some [e.g. list/read] access on a mailbox folder to a particular user 2. Enable the acl_shared_dict to allow dovecot to track (and display to IMAP clients) the folders to which they have access. This will take the form of a BDB file: /var/lib/dovecot/shared-mailboxes.db This file can only be updated by using the SETACL command. *This is where I have the problem*. The SETACL command does nothing. No matter how hard I try, I cannot get dovecot to update this file. The only indication I have that the file exists and is readable by dovecot is the disappearance from the logs of the line: No acl_shared_dict setting - shared mailbox listing is disabled So I can see that dovecot knows the file is there because it does not complain any more. The file is there (because I created it myself, although it is empty: size = 0), and in a moment of weakness I made sure it could be updated (this will be changed back!): ls -l /var/lib/dovecot/shared-mailboxes.db -rwxrwxrwx 1 dovecot dovecot 0 2012-10-30 12:27 /var/lib/dovecot/shared-mailboxes.db But if I try to update the ACL, absolutely nothing happens: [ > command ] [ < reply ] > telnet localhost 143 < * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=CRAM-MD5] Email server > a login [User2] [pass] < a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH ACL RIGHTS=texk] Logged in > a SETACL Inbox [User 1] rl < a OK Setacl complete. At this time, in the dovecot log (with debug enabled): 2012-10-30 13:55:24 IMAP([User2]): Info: Namespace : Using permissions from /home/mailboxes/[domain]/[User1]: mode=0770 gid=-1 2012-10-30 13:55:24 IMAP([User2]): Info: acl vfile: reading file /home/mailboxes/[domain]/[User1]/dovecot-acl 2012-10-30 13:55:24 IMAP([User2]): Info: acl vfile: reading file /home/mailboxes/[domain]/[USer1]/dovecot-acl Can anyone please help me track down what I'm doing wrong here?! It's driving me mad! Thanks for your time if you've read this far!! As requested: dovecot --version 1.2.9 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot info_log_path: /var/log/dovecot.info log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps imap listen: 127.0.0.1:143 ssl_listen: 37.235.54.98 ssl_cert_file: /etc/ssl/dovecot.crt ssl_key_file: /etc/ssl/private/dovecot.key login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_greeting: Email server valid_chroot_dirs: /var/spool/vmail mail_location: maildir:/home/mailboxes/%d/%n mail_debug: yes mbox_write_locks: fcntl dotlock mail_plugins: acl imap_acl lda: postmaster_address: [ valid at ddress ] auth default: mechanisms: plain cram-md5 verbose: yes passdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: passwd-file args: /etc/dovecot/users socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix master: path: /var/spool/postfix/private/auth-master mode: 384 user: postfix group: postfix plugin: acl: vfile acl_shared_dict: file:/var/lib/dovecot/shared-mailboxes.db acl: vfile:/etc/dovecot/acls From cr at sys4.de Tue Oct 30 15:28:22 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 14:28:22 +0100 Subject: [Dovecot] copymail deleted In-Reply-To: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> Message-ID: <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> > The format is: > > 1*( ) > > If the options="-" then the byte count is the final size. If options="B" then byte count is the base64-encoded size while the original file has to be base64-decoded size. Ok, so far I have "grep'ed" this here: msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 1443213 550635 B76 56/f2/56f25e225385902f3fc5185dc3d0103f59b34d14-b134401e794009503a0400002cb72ff6 1994019 477177 B76 c4/36/c436874b56cf3cd105e82f9243c7eac53c467f32-b234401e794009503a0400002cb72ff6 2561522 1075531 B76 77/af/77af1045a783308dbbf2f8a464c5136a0407e720-b334401e794009503a0400002cb72ff6 3715582 1195635 B76 99/33/99339b17a21ce052cd8f47f1d88c6e869cc1650b-b434401e794009503a0400002cb72ff6 4966686 715386 B76 fe/df/fedf23091720d3fa649af3bd6537e66304b8061a-b534401e794009503a0400002cb72ff6 5805913 788086 B76 ab/36/ab36f53a443f1855bc13caaba9e01e9464b2921f-b634401e794009503a0400002cb72ff6 6684258 906273 B76 10/70/1070d21039bc3f305bb948315a01344eefb2a465-b734401e794009503a0400002cb72ff6 7590707 204613 B76 39/44/394402c057791482f79351363f025ae0a7caf1b0-b834401e794009503a0400002cb72ff6 7795492 1349911 B76 41/bd/41bd01b4880065e5136cafbd1d191a1f8a1ead55-b934401e794009503a0400002cb72ff6 9271435 1504539 B76 c6/71/c671c1367e843741a2cc8f083a37231522d37640-ba34401e794009503a0400002cb72ff6 10877759 357555 B76 58/f5/58f582d2644025b843cf991f5cf783d27f9d90c9-bb34401e794009503a0400002cb72ff6 11826037 890683 B76 82/da/82dabbe06f269e7c79417db3b570246a648d2139-bc34401e794009503a0400002cb72ff6 msg.ext-ref = 118947 317624 B76 ad/9b/ad9be52e11433cd0337cda13bf0a458fd0fd948d-df905c0cd33d0950ae7800002cb72ff6 436770 139669 B76 78/15/781526d896a0530a5e76ebce65f2eb690d102dd3-e0905c0cd33d0950ae7800002cb72ff6 576610 457829 B76 61/3a/613a70c8515c572a04211fb0c63828d9c9acfb70-e1905c0cd33d0950ae7800002cb72ff6 1107667 410786 B76 7f/6b/7f6b7ee9b08a73600d98e8583aae343a90e76b96-e2905c0cd33d0950ae7800002cb72ff6 1611186 816686 B76 ff/ff/ffff9362c5356d8bedb17bd56edf0524bd0ae7b3-e3905c0cd33d0950ae7800002cb72ff6 2516232 643918 B76 4f/aa/4faa153fada5ceea79016cf2eadc1d05110f3f2e-e4905c0cd33d0950ae7800002cb72ff6 3291363 1036359 B76 e6/f3/e6f342bf28e8edfd3214666aaa52f0c067bae22b-e5905c0cd33d0950ae7800002cb72ff6 4418344 668813 B76 20/78/2078c98fb9bcadeeaa49bc38dc31548142fc71b1-e6905c0cd33d0950ae7800002cb72ff6 5154786 502218 B76 40/f4/40f4af3ad2077493caa34faabb201531609b50c4-e7905c0cd33d0950ae7800002cb72ff6 5782912 628591 B76 cc/a9/cca98a2a325f1be9a398d62890836cf11f267c4b-e8905c0cd33d0950ae7800002cb72ff6 6518382 526201 B76 17/47/1747a90b58c50c3d01da7f3a6601f7073cd5b163-e9905c0cd33d0950ae7800002cb72ff6 7140759 517776 B76 04/af/04afe7deb8e6ee99153433d2845da417e54cd042-ea905c0cd33d0950ae7800002cb72ff6 7769983 2317979 B76 05/13/0513bcfceff303125f233ad2c01c5ba2ed96c6a2-eb905c0cd33d0950ae7800002cb72ff6 10214312 3097649 B76 35/e4/35e46902b3e6473b9689a92acd71e58fb7165a8f-ec905c0cd33d0950ae7800002cb72ff6 msg.ext-ref = 75027 1291257 B76 b9/dc/b9dcd6899ae65e5c11b122d7bfc3be9fefc21024-5df010068b3f0950c27d00002cb72ff6 1441078 1131344 B76 f6/e6/f6e63f000d6501be472629747448057b122104c1-5ef010068b3f0950c27d00002cb72ff6 2572595 2218094 B76 93/96/9396c5eaeac2615119e55c67fa8f010332ba0fd3-5ff010068b3f0950c27d00002cb72ff6 4790862 2211695 B76 cc/a5/cca5607fb739306f3628a19575dc41432f74a22d-60f010068b3f0950c27d00002cb72ff6 7002730 2614603 B76 66/10/661002c8039997174e34b9ef31d0e693a556eebe-61f010068b3f0950c27d00002cb72ff6 9617506 2760312 B76 8c/65/8c656fe835af26c175337cd318daca8ae8e00369-62f010068b3f0950c27d00002cb72ff6 12377991 2341764 B76 19/c8/19c83e0bf1284e74e49feecaf95506266201551d-63f010068b3f0950c27d00002cb72ff6 15209343 406758 B76 b6/62/b66216837cc48422e22e7a9a22631f840a49ef78-64f010068b3f0950c27d00002cb72ff6 15616301 136877 B76 06/9f/069f5ab86dc9e8e9972f3f5c0dda03c1f3103730-65f010068b3f0950c27d00002cb72ff6 15753350 971075 B76 a7/7c/a77c36690ff0f0f774b82efaf15f93535ba027e9-66f010068b3f0950c27d00002cb72ff6 16849194 1197333 B76 4f/28/4f2881be6d0e8a7f53c0e226c0dbb148b05674c7-67f010068b3f0950c27d00002cb72ff6 18168424 850768 B76 92/72/9272e1ea7ceb79df6222686bf157f957fa9851c1-68f010068b3f0950c27d00002cb72ff6 19019393 135641 B76 60/fd/60fdcd7851c8f0a21f342aaafce9e49a3e00e1aa-69f010068b3f0950c27d00002cb72ff6 19155207 897179 B76 63/59/6359abf4f9e806e3990e0d6590e519924c838fa5-6af010068b3f0950c27d00002cb72ff6 20169966 1022612 B76 f8/65/f8654367f5df050d23565644e83c8c50abb69c39-6bf010068b3f0950c27d00002cb72ff6 But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. Thanks a lot Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Tue Oct 30 15:42:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 15:42:49 +0200 Subject: [Dovecot] copymail deleted In-Reply-To: <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> Message-ID: On 30.10.2012, at 15.28, Christian R??ner wrote: > msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 > But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. They are all base64 yes, the B76 means that all the encoded lines will be 76 chars long. So the file size above needs to be 1282212, divided by 77 (76+LF) = 16652 full lines and 8 bytes over. Base64 encodes 3 byte blocks into 4 byte chars, so the original data has (16652*76+8)/4*3 = 949170 bytes (or 1-2 bytes less, but that makes no difference because it's padded anyway). So if you create /attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 that is 949170 bytes long, and do the same for the rest of the attachments, you should be able to read this mail without errors. You can easily create the files without wasting space with: dd if=/dev/zero of=foo bs=1 seek=949169 count=1 From cr at sys4.de Tue Oct 30 16:44:01 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 15:44:01 +0100 Subject: [Dovecot] copymail deleted In-Reply-To: References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> Message-ID: <09B9ED24-9319-48A7-85D4-0FF7D12F6296@sys4.de> Hi, >> msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 > >> But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. > > They are all base64 yes, the B76 means that all the encoded lines will be 76 chars long. So the file size above needs to be 1282212, divided by 77 (76+LF) = 16652 full lines and 8 bytes over. Base64 encodes 3 byte blocks into 4 byte chars, so the original data has (16652*76+8)/4*3 = 949170 bytes (or 1-2 bytes less, but that makes no difference because it's padded anyway). > > So if you create /attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 that is 949170 bytes long, and do the same for the rest of the attachments, you should be able to read this mail without errors. > > You can easily create the files without wasting space with: > dd if=/dev/zero of=foo bs=1 seek=949169 count=1 Thanks. I have calculated both other files and recreated zero padded files. Now I am going to watch the log file and see, if errors are gone. One last question: If the user now opens a mail, where the attachments are broken and he/she removes the mail, are the created hand-made files be removed automatically? Thanks in advance Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From dmiller at amfes.com Tue Oct 30 17:00:19 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 30 Oct 2012 08:00:19 -0700 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 Message-ID: I'm compiling as I normally do. The config line for Dovecot is: configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap followed by make & make install Then a 'configure' for Pigeonhole, followed by make, yields: libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sieve-dump sieve-dump.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_new_null at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_peer_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_load_verify_locations at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_rsa_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_desc_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_new_index at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_client_CA_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_ext_d2i at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_accept at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_cert at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_server_method at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OBJ_txt2nid at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_write at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_num at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_get_write_guarantee at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_cipher_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_push at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_get_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_type_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_COMP_get_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_by_id at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_write at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `i2d_DHparams at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_ENTRY_get_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_info_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_subject_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_init at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_clear_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_load_error_strings at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_RSA at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_bits at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_INFO_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_set_flags at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_new at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_dup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_index_by_NID at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `d2i_DHparams at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_text_by_NID at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OPENSSL_add_all_algorithms_noconf at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_DSA at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_cleanup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_entry at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_type at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_library_init at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_cipher at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_cipher_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_connect at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_error_string_n at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_CTX_get_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_verify at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_bio_pair at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_client_method at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_length at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_get_cert_store at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_dh_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_ciphers at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_mem_buf at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data_X509_STORE_CTX_idx at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_pop_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_value at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `GENERAL_NAME_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_version at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RAND_bytes at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_new at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_pending at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_bio at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_load_builtin_engines at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_X509 at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_cleanup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_crl at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_read at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_finish at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_read at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_oneline at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_free_strings at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_PKEY_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_compression at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_verify_cert_error_string at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_ctrl at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_last_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RSA_generate_key at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_state_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_X509_INFO_read_bio at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_generate_parameters at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_free at OPENSSL_1.0.0' collect2: ld returned 1 exit status make[3]: *** [sieve-dump] Error 1 make[3]: *** Waiting for unfinished jobs.... -- Daniel From rob0 at gmx.co.uk Tue Oct 30 18:23:18 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 30 Oct 2012 11:23:18 -0500 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <20121030162318.GB3672@harrier.slackbuilds.org> On Mon, Oct 29, 2012 at 10:39:51PM +0200, Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver > mail that is over 1MB, Dovecot rejects the mail. But smaller mails > aren't rejected probably for days. So user might not even realize > that they didn't receive one of the mails. Also having a user > "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail > even if it brings the user over quota? Except add a limit that if > the message size is as much as the user's entire quota limit it > wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, > since user would get an error anyway. I could make this also > optional, but if nobody really wants to keep the old behavior > there's really no point in adding the option. I think the thing to do is to adjust the admin's thinking about it. Yes, if the current mailstore is under quota, by all means, you should accept the next email up to the maximum size the server accepts. No exception, just take it. You control $quota and $maxMsg. Set your quota with that in mind, where $(($quota - 1 + $maxMsg)) total is something you can live with. That said, I have been fortunate to never have to set up a quota. Storage is cheap. An occasional cron job can point out individual users who might be beyond what you'd consider reasonable, and to those users, apply a LART. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From cgregoir99 at yahoo.com Tue Oct 30 22:57:36 2012 From: cgregoir99 at yahoo.com (Christian Gregoire) Date: Tue, 30 Oct 2012 20:57:36 +0000 (GMT) Subject: [Dovecot] POP3 Proxy : user format not accepted In-Reply-To: References: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Message-ID: <1351630656.22457.YahooMailNeo@web172405.mail.ir2.yahoo.com> Indeed, I set its value to empty to allow all characters and it now works. Thanks a lot Timo. ________________________________ De?: Timo Sirainen ??: Christian Gregoire ; Dovecot Mailing List Envoy? le : Mardi 30 octobre 2012 14h08 Objet?: Re: [Dovecot] POP3 Proxy : user format not accepted On 30.10.2012, at 14.01, Christian Gregoire wrote: > Any idea ? Is the '#' not internally supported in the user login by Dovecot ? See auth_username_chars setting. From stephan at rename-it.nl Tue Oct 30 23:46:02 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 30 Oct 2012 22:46:02 +0100 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: References: Message-ID: <50904A9A.8030703@rename-it.nl> On 10/30/2012 4:00 PM, Daniel L. Miller wrote: > I'm compiling as I normally do. The config line for Dovecot is: > > configure --with-ldap --with-ssl --with-bzlib --with-zlib > --with-stemmer --with-lucene --with-ldap > > followed by make & make install > > Then a 'configure' for Pigeonhole, followed by make, yields: What kind of system are you compiling this on? Regards, Stephan. From cliff at clamjuice.org Wed Oct 31 00:29:21 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Tue, 30 Oct 2012 18:29:21 -0400 Subject: [Dovecot] Unable to get Managesieve working Message-ID: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> I have Roundcube webmail (v. 0.8.2) running with the managesieve plugin (v. 5.1). I am able to create sieve rules without any problems in Roundcube, but incoming mail is not being processed with the rule I specify. I suspect there is something that isn't configured correctly in dovecot, but unfortunately I am unable to find a resolution online and my very limited knowledge of dovecot isn't allowing me to solve this problem on my own. I am including my dovecot config below. Please keep my knowledge level in mind when suggesting options and let me know if there is any other information I can provide to help troubleshoot the problem. Thanks for any assistance! # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key ssl_cipher_list: ALL:!LOW:!SSLv2 login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_location: maildir:~/mail:LAYOUT=fs:INBOX=~/mail/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve auth default: mechanisms: plain login passdb: driver: pam passdb: driver: pam userdb: driver: passwd userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: sieve: ~/.dovecot.sieve sieve_dir: ~/mail/sieve From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 01:37:09 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 00:37:09 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> Message-ID: <20121030233709.GA14111@daniel.localdomain> Hi Cliff, Cliff Dunn wrote: > I have Roundcube webmail (v. 0.8.2) running with the managesieve > plugin (v. 5.1). I am able to create sieve rules without any > problems in Roundcube, but incoming mail is not being processed with > the rule I specify. managesieve is the service for managing your sieve rules, it does not process any emails. In order to actually sort mails into folders, you need to add the "sieve" plugin to your mail_plugins list. http://wiki.dovecot.org/LDA/Sieve/Dovecot protocol lda { .. # Support for dynamically loadable plugins. mail_plugins is a space separated # list of plugins to load. mail_plugins = sieve # ... other plugins like quota } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 02:09:57 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 01:09:57 +0100 Subject: [Dovecot] lmtp out of memory - raw backtrace Message-ID: <20121031000957.GA15191@daniel.localdomain> Hi, Our setup: - 4 hosts with director and mailbox instance - delivery via director lmtp into mailbox lmtp - mailbox format: mdbox - storage on NFS - OS: Linux 2.6.32-44-server x86_64 Ubuntu 10.04.4 LTS - Dovecot 2.1.10 - Pigeonhole 0.3.3 We're getting strange "out of memory" lmtp errors/backtrace with dovecot 2.1.10 accompanied by high load caused caused by a lot of lmtp deliveries to one user. First action would be to increase vsz_limit to a higher value, but I just want to make sure there is no bug - before blindly increasing this limit. I think it should not crash and corrupt mdbox, even if memory limit is reached: Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Fatal: pool_system_realloc(16777216): Out of memory Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4271a) [0x7f6dcbae971a] -> /usr/lib/dovecot/libdovecot.so.0(+0x42766) [0x7f6dcbae 9766] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f6dcbabd158] -> /usr/lib/dovecot/libdovecot.so.0(+0x53690) [0x7f6dcbafa690] -> /usr/lib/dovecot/libdovecot.so.0(+0x3e6f5) [0x7f6dcbae56f5] -> /usr/lib/dove cot/libdovecot.so.0(buffer_write+0x7c) [0x7f6dcbae5e7c] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8033) [0x7f6dcbdda033] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_record_map_move_to_private+0x3 c) [0x7f6dcbdda4ec] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_get_atomic_map+0x18) [0x7f6dcbde9c88] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb7cf9) [0x7f6dcbde9cf9] -> /usr/lib/dovecot/lib dovecot-storage.so.0(mail_index_sync_record+0x7e6) [0x7f6dcbdea626] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x23c) [0x7f6dcbdeae4c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_m ap+0xa8) [0x7f6dcbddc2b8] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb42da) [0x7f6dcbde62da] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb458a) [0x7f6dcbde658a] -> /usr/lib/dovecot/libdovecot-storage.so.0( mail_index_sync_begin_to+0x4f) [0x7f6dcbde6ecf] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1e) [0x7f6dcbde6f4e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_atomic_lock+0x5e) [0x 7f6dcbd6115e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_transaction_save_commit_pre+0x46) [0x7f6dcbd64fb6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9d4f3) [0x7f6dcbdcf4f3] -> /usr/lib/dovecot/libdov ecot-storage.so.0(mail_index_transaction_commit_full+0x9f) [0x7f6dcbddd97f] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x8a) [0x7f6dcbdcf18a] -> /usr/lib/dovecot/modules/lib10_quota_plug in.so(+0xba7f) [0x7f6dca4eca7f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x3d) [0x7f6d Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Fatal: master: service(lmtp): child 26698 returned error 83 (Out of memory (service lmtp { vsz_limit=256 MB }, you may need to increase it)) Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1 at example.org): Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1 at example.org): Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 10:32:27 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: Corrupted dbox file /mail/dovecot/example.org/user1/mail/storage/m.3577 (around offset=1844402): EOF reading msg header (got 0/30 bytes) Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Cron/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/postfix/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Dovecot/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:46 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: 2ZCXNsuej1CcUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHO029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: 2ZCXNsuej1CcUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: PFBjN8uej1CdUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHP029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: spYhE82ej1CiUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHQ029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: sOscMs2ej1CrUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHS029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: spYhE82ej1CiUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: PFBjN8uej1CdUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: sOscMs2ej1CrUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: +jXqE+uej1AiVAAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300933.q9U9URHi029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: +jXqE+uej1AiVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: CxDMM+uej1AnVAAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300933.q9U9URHl029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: CxDMM+uej1AnVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Regards Daniel -- https://plus.google.com/103021802792276734820 From cliff at clamjuice.org Wed Oct 31 02:17:56 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Tue, 30 Oct 2012 20:17:56 -0400 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <20121030233709.GA14111@daniel.localdomain> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> Message-ID: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Ok, so when I add the mail_plugins = sieve I get: sudo service dovecot restart Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found from directory /usr/lib/dovecot/modules/imap Error: imap dump-capability process returned 89 Fatal: Invalid configuration in /etc/dovecot/dovecot.conf failed I am assuming something is missing here? On 2012-10-30 19:37, Daniel Parthey wrote: > Hi Cliff, > > Cliff Dunn wrote: >> I have Roundcube webmail (v. 0.8.2) running with the managesieve >> plugin (v. 5.1). I am able to create sieve rules without any >> problems in Roundcube, but incoming mail is not being processed with >> the rule I specify. > > managesieve is the service for managing your sieve rules, > it does not process any emails. > > In order to actually sort mails into folders, you need to add > the "sieve" plugin to your mail_plugins list. > > http://wiki.dovecot.org/LDA/Sieve/Dovecot > > protocol lda { > .. > # Support for dynamically loadable plugins. mail_plugins is a space > separated > # list of plugins to load. > mail_plugins = sieve # ... other plugins like quota > } > > Regards > Daniel From tss at iki.fi Wed Oct 31 02:20:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 31 Oct 2012 02:20:34 +0200 Subject: [Dovecot] lmtp out of memory - raw backtrace In-Reply-To: <20121031000957.GA15191@daniel.localdomain> References: <20121031000957.GA15191@daniel.localdomain> Message-ID: On 31.10.2012, at 2.09, Daniel Parthey wrote: > - Dovecot 2.1.10 > - Pigeonhole 0.3.3 > > We're getting strange "out of memory" lmtp errors/backtrace with dovecot 2.1.10 > accompanied by high load caused caused by a lot of lmtp deliveries to one user. > > First action would be to increase vsz_limit to a higher value, but I just > want to make sure there is no bug - before blindly increasing this limit. Recent changes in hg should help with this for LMTP. I'll probably release 2.1.11 somewhat soon. > I think it should not crash and corrupt mdbox, even if memory limit is reached: It should crash, because it reached the enforced vsz limit. If you don't want it to crash you can disable the limit, at the cost of potentially eating all of your memory. > Oct 30 10:32:27 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: Corrupted dbox file /mail/dovecot/example.org/user1/mail/storage/m.3577 (around offset=1844402): EOF reading msg header (got 0/30 bytes) This does look like something that should be fixed. It's not exactly a corruption so much as finding a partially written mail during rescan (because of the crash), but still it can probably be avoided. From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 02:45:12 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 01:45:12 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Message-ID: <20121031004511.GA15928@daniel.localdomain> Cliff Dunn wrote: > >In order to actually sort mails into folders, you need to add > >the "sieve" plugin to your mail_plugins list. > > > >http://wiki.dovecot.org/LDA/Sieve/Dovecot > > > >protocol lda { > > mail_plugins = sieve > >} > Ok, so when I add the mail_plugins = sieve I get: > sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found > from directory /usr/lib/dovecot/modules/imap > Error: imap dump-capability process returned 89 > Fatal: Invalid configuration in /etc/dovecot/dovecot.conf > failed > > I am assuming something is missing here? I guess you are missing the sieve plugin completely, did you install the dovecot-sieve plugin? Where did you get sieve from and how did you install it? Did you follow all the compile instructions at http://wiki.dovecot.org/LDA/Sieve/Dovecot or did you install some package? What do the following commands say? # find /usr/lib/dovecot | grep sieve # apt-cache policy dovecot-sieve # dpkg -l "dovecot*" Regards Daniel -- https://plus.google.com/103021802792276734820 From dmiller at amfes.com Wed Oct 31 03:11:50 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 30 Oct 2012 18:11:50 -0700 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: <50904A9A.8030703@rename-it.nl> References: <508FEB83.5010209@amfes.com> <50904A9A.8030703@rename-it.nl> Message-ID: On 30.10.2012 14:46, Stephan Bosch wrote: > On 10/30/2012 4:00 PM, Daniel L. Miller wrote: > >> I'm compiling as I normally do. The config line for Dovecot is: configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap followed by make & make install Then a 'configure' for Pigeonhole, followed by make, yields: > > What kind of system are you compiling this on? AMD Opteron 4180, Ubuntu Precision, Linux 3.2.0-25, 64-bit. libssl-dev 1.0.1c-3ubuntu1 libc6 2.15 gcc 4.6.3 -- Daniel From calestyo at scientia.net Wed Oct 31 03:13:03 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 02:13:03 +0100 Subject: [Dovecot] maildir S= and W= Message-ID: <1351645983.24721.0.camel@fermat.scientia.net> Hi. Even new mails delivered by my MDA don't get the ,S= and ,W= fields set... (but when I "upload" a mail via IMAP to dovecot, they are set) Is there some place in dovecot where I need to enable this? Or would it be the MDA that has to calculate and set this already when placing a file in ./new? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5113 bytes Desc: not available URL: From calestyo at scientia.net Wed Oct 31 03:50:23 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 02:50:23 +0100 Subject: [Dovecot] maildir and end-of-line encoding Message-ID: <1351648223.24721.4.camel@fermat.scientia.net> Hi. I just wondered, the following: My MDA may get mails that use LF or CR/LF end of line encodings and deliver them into maildirs. I couldn't find any information about, whether one should or must convert all into one format, cause AFAIK at least on the IMAP side, CR/LF is always used? How does this work on the maildir/backend side of dovcot? Can it work with both and simply automatically convert LF into CR/LF? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5113 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 09:20:43 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 08:20:43 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351645983.24721.0.camel@fermat.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> Message-ID: <5090D14B.9080805@skye.it> Il 31/10/2012 02:13, Christoph Anton Mitterer ha scritto: > Hi. > > Even new mails delivered by my MDA don't get the ,S= and ,W= fields > set... > (but when I "upload" a mail via IMAP to dovecot, they are set) > > > Is there some place in dovecot where I need to enable this? Or would it > be the MDA that has to calculate and set this already when placing a > file in ./new? The MDA should calculate and set this, dovecot always add these informations, as you can see when upload file via IMAP. Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From skdovecot at smail.inf.fh-brs.de Wed Oct 31 11:08:10 2012 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 31 Oct 2012 10:08:10 +0100 (CET) Subject: [Dovecot] backtrace for non-existant %{ldap:attr} on login Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I'm fetching the user and auth data from LDAP, this is the string: pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,mailUidNumber=userdb_uid,mailGidNumber=userdb_gid,mailLocationDovecot=userdb_mail,uid=userdb_user,=userdb_quota_rule=*:bytes=%{ldap:mailQuotaBytes}, =userdb_quota_rule2=Trash:bytes=+%{ldap:mailQuotaBytesTrash} If mailQuotaBytesTrash or mailQuotaBytes is not present, the LOGIN process does not work: 1 login user pwd 1 NO [UNAVAILABLE] Temporary authentication failure. [mailsrv2:2012-10-31 08:56:51] * OK Waiting for authentication process to respond.. If I add those two attributes, the user can login successfully. dovecot-2.1.10/sbin/dovecot --version 2.1.10 (9cdeab12f3e1) The log entries: 2012-10-31 09:56:51 auth: Panic: pool_data_stack_realloc(): stack frame changed 2012-10-31 09:56:51 auth: Error: Raw backtrace: /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x4857a) [0x7f2c0528c57a] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x485c6) [0x7f2c0528c5c6] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f2c0525feaf] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x58f2e) [0x7f2c0529cf2e] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x442f5) [0x7f2c052882f5] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x68) [0x7f2c05288728] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(str_vprintfa+0x6d) [0x7f2c052a796d] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(str_printfa+0x88) [0x7f2c052a7ac8] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x42682e] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(var_expand_with_funcs+0x6fb) [0x7f2c052ac48b] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb](db_ldap_result_iterate_next+0x12f) [0x42734f] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x428974] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x428f5f] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x4282a1] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f2c05298756] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f2c0529978f] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f2c052986f8] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f2c052840d3] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb](main+0x376) [0x41bba6] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f2c04449c8d] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x40cf89] 2012-10-31 09:56:51 auth: Fatal: master: service(auth): child 15865 killed with signal 6 (core dumps disabled) 2012-10-31 09:56:51 imap-login: Warning: Auth connection closed with 1 pending requests (max 0 secs, pid=15869, EOF) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUJDqe2oxLS8a3A9mAQI+YQf/Qd4IIeM35Hmmpl1IMcQwJFK4854G5Dku yK+GsWhE2gxI6KaLO6DSI/kpN79qhQRkHsUAHzoPiZ7kQpZprNaEP/CIPkTzw//i HyC2Odpfa8fWUOqtH5Cp6X5spF6hQa4mmQyzgXguF9bdjZkNu4vQ78wRuQGG4eHi BOkNL0b93DsN3NSIoXDpiAiJn1aleTLe7mYkfsGewjb+AN+FpR4hLbf32yRRn8J0 Fkn8agEijixGMXEBD4ZprTbX9NbUr92YPfRycMnA2A00MUUlv/iKlqXRpMLafRjL bnHL/QE80xHoKKJUR96/RjruciIolAtlyYvhTU9ibRpLYm5Hcd9bZg== =iPvg -----END PGP SIGNATURE----- From fabio.ferrari at unimore.it Wed Oct 31 13:12:41 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Wed, 31 Oct 2012 12:12:41 +0100 Subject: [Dovecot] Dovecot stops to work - anvil problem In-Reply-To: <51972B14-6973-4510-870D-956F858FC76B@iki.fi> References: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> <51972B14-6973-4510-870D-956F858FC76B@iki.fi> Message-ID: <0d40f13a4256adae8f084c385dcd0fd3.squirrel@webmail2.unimore.it> Thank you very much for your help, I cross mi fingers but it seems that this was the problem. Fabio Ferrari > On 26.10.2012, at 13.24, FABIO FERRARI wrote: > >> Hi all, >> >> we have a problem about anvil, it seems that when we have a high load >> the >> dovecot stops to work. Sometimes it is sufficient to make a dovecot >> reload, but sometimes we have to restart it. >> >> Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files > > This is the problem. > >> And these are the limit settings in the OS: >> * soft nofile 131072 >> * hard nofile 131072 >> >> Have someone had the same problem? > > The OS limits are ok. But you need to make sure that the dovecot processes > have enough fds in ulimit. You can check the limits with: > > cat /proc//limits > > The "Max open files" soft limit is what you're most likely hitting. Use > "ulimit -n 10000" or something before running dovecot binary. And make > sure that it changes the limit in the proc. Many init scripts change the > ulimit internally. > > From tss at iki.fi Wed Oct 31 13:26:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 31 Oct 2012 13:26:11 +0200 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: References: Message-ID: On 30.10.2012, at 17.00, Daniel L. Miller wrote: > I'm compiling as I normally do. The config line for Dovecot is: > > configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap > > followed by make & make install > > Then a 'configure' for Pigeonhole, followed by make, yields: > libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sieve-dump sieve-dump.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot > libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot > /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_new_null at OPENSSL_1.0.0' > /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_error at OPENSSL_1.0.0' I think this is a Dovecot bug, fixed by: http://hg.dovecot.org/dovecot-2.1/rev/7d931927e4ac You could also do this by adding -lssl -lcrypto manually to the installed dovecot-config and running configure again for pigeonhole. From calestyo at scientia.net Wed Oct 31 13:41:51 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 12:41:51 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <5090D14B.9080805@skye.it> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> Message-ID: <1351683711.7825.0.camel@heisenberg.scientia.net> On Wed, 2012-10-31 at 08:20 +0100, Alessio Cecchi wrote: > The MDA should calculate and set this, dovecot always add these > informations, as you can see when upload file via IMAP. Ah thanks,... do you know whether it's possible to have them set by maildrop? I couldn't find anything on this. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 14:04:27 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 13:04:27 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351683711.7825.0.camel@heisenberg.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> Message-ID: <509113CB.7020402@skye.it> Il 31/10/2012 12:41, Christoph Anton Mitterer ha scritto: > On Wed, 2012-10-31 at 08:20 +0100, Alessio Cecchi wrote: >> The MDA should calculate and set this, dovecot always add these >> informations, as you can see when upload file via IMAP. > Ah thanks,... do you know whether it's possible to have them set by > maildrop? I couldn't find anything on this. My maildrop (2.4) version, working with qmail and vpopmail, add S= by default. Probably you are running an old version without Maildir++ support: http://www.inter7.com/courierimap/README.maildirquota.html ============== Delivering to a Maildir++ Delivering to a Maildir++ is like delivering to a Maildir, with the following exceptions: Follow the usual Maildir conventions for naming the filename used to store the message, except that append ,S=nnnnn to the name of the file, where nnnnn is the size of the file. This eliminates the need to stat() most messages when calculating the quota. If the size of the message is not known at the beginning, append ,S=nnnnn when renaming the message from tmp to new. ============== Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From cliff at clamjuice.org Wed Oct 31 14:47:10 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Wed, 31 Oct 2012 08:47:10 -0400 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <20121031004511.GA15928@daniel.localdomain> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> <20121031004511.GA15928@daniel.localdomain> Message-ID: <34eb54a518ab2a0cdff5709b9e8bfac6@mail.clamjuice.org> I didn't follow those instructions as I assumed that it would be installed with Dovecot from the Debian repositories. See output of commands below. And thanks again for the help! # find /usr/lib/dovecot | grep sieve /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.la /usr/lib/dovecot/managesieve /usr/lib/dovecot/managesieve-login # apt-cache policy dovecot-sieve N: Unable to locate package dovecot-sieve # dpkg -l "dovecot*" Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-============================-============================-======================================================================== un dovecot (no description available) ii dovecot-common 1:1.2.15-7 secure mail server that supports mbox and maildir mailboxes ii dovecot-imapd 1:1.2.15-7 secure IMAP server that supports mbox and maildir mailboxes On 2012-10-30 20:45, Daniel Parthey wrote: > Cliff Dunn wrote: >> >In order to actually sort mails into folders, you need to add >> >the "sieve" plugin to your mail_plugins list. >> > >> >http://wiki.dovecot.org/LDA/Sieve/Dovecot >> > >> >protocol lda { >> > mail_plugins = sieve >> >} >> Ok, so when I add the mail_plugins = sieve I get: >> sudo service dovecot restart >> Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found >> from directory /usr/lib/dovecot/modules/imap >> Error: imap dump-capability process returned 89 >> Fatal: Invalid configuration in /etc/dovecot/dovecot.conf >> failed >> >> I am assuming something is missing here? > > I guess you are missing the sieve plugin completely, > did you install the dovecot-sieve plugin? > Where did you get sieve from and how did you install it? > > Did you follow all the compile instructions at > http://wiki.dovecot.org/LDA/Sieve/Dovecot or > did you install some package? > > What do the following commands say? > > # find /usr/lib/dovecot | grep sieve > # apt-cache policy dovecot-sieve > # dpkg -l "dovecot*" > > Regards > Daniel From stephan at rename-it.nl Wed Oct 31 15:03:23 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 31 Oct 2012 14:03:23 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Message-ID: <5091219B.6000709@rename-it.nl> Op 10/31/2012 1:17 AM, Cliff Dunn schreef: > Ok, so when I add the mail_plugins = sieve I get: > sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found from > directory /usr/lib/dovecot/modules/imap > Error: imap dump-capability process returned 89 > Fatal: Invalid configuration in /etc/dovecot/dovecot.conf > failed > > I am assuming something is missing here? You should only put the mail_plugins=sieve inside de protocol lda {} section. Adding the Sieve plugin to IMAP makes no sense. Regards, Stephan. From calestyo at scientia.net Wed Oct 31 15:55:32 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 14:55:32 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <509113CB.7020402@skye.it> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> <509113CB.7020402@skye.it> Message-ID: <1351691732.8425.1.camel@heisenberg.scientia.net> On Wed, 2012-10-31 at 13:04 +0100, Alessio Cecchi wrote: > maildrop (2.4) Ah thanks... yeah I had an old version.. > add S= by > default. > http://www.inter7.com/courierimap/README.maildirquota.html AFAIU that... ,W= is however not set, right? :( thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 16:09:51 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 15:09:51 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351691732.8425.1.camel@heisenberg.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> <509113CB.7020402@skye.it> <1351691732.8425.1.camel@heisenberg.scientia.net> Message-ID: <5091312F.1080603@skye.it> Il 31/10/2012 14:55, Christoph Anton Mitterer ha scritto: > On Wed, 2012-10-31 at 13:04 +0100, Alessio Cecchi wrote: >> maildrop (2.4) > Ah thanks... yeah I had an old version.. > > >> add S= by >> default. >> http://www.inter7.com/courierimap/README.maildirquota.html > AFAIU that... ,W= is however not set, right? :( Yes, W= is not set by maildrop, only dovecot-lda set W=, anyway S=size is sufficient for speed up the system. If the message was stored with CR+LF linefeeds, and are the same. More info here: http://wiki2.dovecot.org/MailboxFormat/Maildir Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From jpdalbec at ysu.edu Wed Oct 31 17:15:56 2012 From: jpdalbec at ysu.edu (John Dalbec) Date: Wed, 31 Oct 2012 11:15:56 -0400 Subject: [Dovecot] IMAP proxy between Office 365 client/Sun Messaging Server Message-ID: <509140AC.80602@ysu.edu> I would like to be able to migrate messages from existing end-user accounts on Sun Messaging Server to Office 365 using an administrator login and password. Unfortunately the migration tool for Office 365 doesn't support SASL AUTHENTICATE PLAIN login. The online documentation I've found suggests that I should be able to configure dovecot as an IMAP proxy and have it log in to Sun Messaging Server with AUTHENTICATE PLAIN and encode_base64("user\0admin\0adminpw") in response to a login from the Office 365 migration tool. I'd like to configure dovecot to run only the IMAP proxy if possible. I was thinking of setting all accounts to use the same (strong) password in the proxy. Would anyone be willing to share a sample configuration? Thanks, John Dalbec ellucian Luminis system administrator Youngstown State University From rs at sys4.de Wed Oct 31 17:37:53 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 31 Oct 2012 16:37:53 +0100 Subject: [Dovecot] IMAP proxy between Office 365 client/Sun Messaging Server In-Reply-To: <509140AC.80602@ysu.edu> References: <509140AC.80602@ysu.edu> Message-ID: <509145D1.1070504@sys4.de> Am 31.10.2012 16:15, schrieb John Dalbec: > I would like to be able to migrate messages from existing end-user > accounts on Sun Messaging Server to Office 365 using an administrator > login and password. Unfortunately the migration tool for Office 365 > doesn't support SASL AUTHENTICATE PLAIN login. > > The online documentation I've found suggests that I should be able to > configure dovecot as an IMAP proxy and have it log in to Sun Messaging > Server with AUTHENTICATE PLAIN and encode_base64("user\0admin\0adminpw") > in response to a login from the Office 365 migration tool. I'd like to > configure dovecot to run only the IMAP proxy if possible. I was > thinking of setting all accounts to use the same (strong) password in > the proxy. > > Would anyone be willing to share a sample configuration? > > Thanks, > John Dalbec > ellucian Luminis system administrator > Youngstown State University perhaps look in this http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy http://wiki.dovecot.org/HowTo/ImapProxy Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From micah at riseup.net Wed Oct 31 21:15:57 2012 From: micah at riseup.net (Micah Anderson) Date: Wed, 31 Oct 2012 15:15:57 -0400 Subject: [Dovecot] Error: Internal quota calculation error Message-ID: <87d2zyxxjm.fsf@minnow.riseup.net> Hello, I'm using 2.1.7 with seive and mysql quotas. We had an outage the other day where the database server where quotas are stored was not available for a short period of time. In dovecot land, the following types of errors occured in that scenario: Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: Internal quota calculation error Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: sieve: msgid=<20122132765181x.ABCCE457 at example.com>: failed to store into mailbox 'Trash': Internal error occurred. Refer to server log for more information. [2012-10-26 22:19:01] Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: sieve: script /maildir/e/example/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /maildir/e/example/.dovecot.sieve.log may reveal additional details) I expect that there would be quota calculation errors as dovecot could not reach the database server, but what worried me was the 'failed to store into mailbox' message from sieve. The 'Trash' mailbox in this particular seive script is the correct location for the message to be filed into, but the worrisome message is the 'failed with unsuccessful implicit keep'. I looked through all the message-ids that reported this error and I found that the messages were properly delivered in the end, so this might be some issue interacting between sieve, dovecot and quota and just causing a scary message that can be ignored? thanks for any information you can provide, dovecot is great! micah -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available URL: From tlx at leuxner.net Wed Oct 31 21:46:27 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 31 Oct 2012 20:46:27 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508EFC12.4000509@jkart.de> References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> <508EFC12.4000509@jkart.de> Message-ID: Am 29.10.2012 um 22:58 schrieb Jim Knuth : > /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. Error says it all? So not really helpful. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4364 bytes Desc: not available URL: From jk at jkart.de Wed Oct 31 21:50:11 2012 From: jk at jkart.de (Jim Knuth) Date: Wed, 31 Oct 2012 20:50:11 +0100 Subject: [Dovecot] (Solved) Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> <508EFC12.4000509@jkart.de> Message-ID: <509180F3.7020608@jkart.de> am 31.10.12 20:46 schrieb Thomas Leuxner : > Am 29.10.2012 um 22:58 schrieb Jim Knuth : > >> /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. > > Error says it all? So not really helpful. > yes, I know ;) But no more problems. I dont no why *bg* -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Ich schaue mir meine Filme nie an. Sie sind mir zu brutal. (Charles Bronson) From stan at hardwarefreak.com Mon Oct 1 00:37:17 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 30 Sep 2012 16:37:17 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <5068BB8D.8020302@hardwarefreak.com> On 9/30/2012 8:02 AM, Charles Marcus wrote: > Hi Timo/everyone, > > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? > > The reason I ask is, we are changing ISPs, and I would like to see in > the logs when an external connection is coming from our OLD ISP > connection, and when it is coming through our new one. Traffic monitoring during a switchover of this nature is typically done at the edge router, not inside an individual server application. Is your router able to compile or export a daily traffic report per physical port, or raw data to your network monitoring software, showing packets/connections for TCP/UDP. Most can. This would give you the information you seek, including all traffic for both the new and old ISP, not just IMAP. -- Stan From eugene at raptor.kiev.ua Mon Oct 1 02:07:25 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 02:07:25 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50641AE7.6040201@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> Message-ID: On Thu, 27 Sep 2012 12:22:47 +0300, Davide wrote: > Hi to all, sorry in advance for my poor english, this is the first time > that i wrote to a list if i make mistake .... excuseme. > My problem is this: i have dovecot 2.1.8 installed and functioning from > 2 years one week ago i have installed crm114 for my last spam detection > filter "version 20100106-BlameMichelson (TRE 0.8.0 (BSD))" > My mail system is qmail that through .qmail default correctly tag with > X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-27CA1CFB > X-CRM114-CacheID: sfid-20120927_105129_798028_B0035817 > X-CRM114-Status: GOOD ( 28.64 ) headers ... > "Operation failed over folder 'UNSURE'. Server for account > davide.marchi at mail.cgilfe.it said: [CANNOT] Failed to call crm114 > binary.." Did you see anything in syslog? If not, let's begin with the attached patch. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix -------------- next part -------------- A non-text attachment was scrubbed... Name: crm_debug.patch Type: application/octet-stream Size: 694 bytes Desc: not available URL: From eugene at raptor.kiev.ua Mon Oct 1 02:16:25 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 02:16:25 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: References: <50641AE7.6040201@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich wrote: > Did you see anything in syslog? If not, let's begin with the attached > patch. Ouch... too sleepy. Here's the correct patch. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix -------------- next part -------------- A non-text attachment was scrubbed... Name: crm_debug.patch Type: application/octet-stream Size: 716 bytes Desc: not available URL: From tss at iki.fi Mon Oct 1 05:41:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 01 Oct 2012 05:41:00 +0300 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <1349059260.18782.42.camel@innu> On Sun, 2012-09-30 at 09:02 -0400, Charles Marcus wrote: > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? Dovecot only sees one remote IP address (%r) and one local IP address (% l) for connections. %r is already logged, but you can add %l if that helps. Other than that, I can't really help. From david.ledger at ivdcs.co.uk Mon Oct 1 10:58:30 2012 From: david.ledger at ivdcs.co.uk (David Ledger) Date: Mon, 1 Oct 2012 08:58:30 +0100 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <5068582A.6030507@brightok.net> References: <506842CB.8080501@Media-Brokers.com> <5068582A.6030507@brightok.net> Message-ID: At 09:33 -0500 30/9/12, Jack Bates wrote: >On 9/30/2012 8:02 AM, Charles Marcus wrote: >> Hi Timo/everyone, >> >> Currently we are logging the remote IP, but is there a way to show >>the IP address that the NAT connection is coming from? >> >> The reason I ask is, we are changing ISPs, and I would like to see >>in the logs when an external connection is coming from our OLD ISP >>connection, and when it is coming through our new one. >> >> We have a Watchguard firewall, and I have both External >>connections setup and working, and have just pointed our DNS >>records to the new public IP, and would like to be able to see >>which WAN connection/IP they are coming from. > >You could bind 2 internal IP Addresses to the server and have each >NAT translation go to a different internal IP. > >Jack From my remembrances of the packet layout there is nowhere in the packet for the pre-NAT address to live. The only place the mapping is stored is in the internal tables of the NAT router which has to know where to send the reply packets. David -- David Ledger - Freelance Unix Sysadmin in the UK. david.ledger at ivdcs.co.uk www.ivdcs.co.uk From CMarcus at Media-Brokers.com Mon Oct 1 11:58:15 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 01 Oct 2012 04:58:15 -0400 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <1349059260.18782.42.camel@innu> References: <506842CB.8080501@Media-Brokers.com> <1349059260.18782.42.camel@innu> Message-ID: <50695B27.8090802@Media-Brokers.com> On 2012-09-30 10:41 PM, Timo Sirainen wrote: > Dovecot only sees one remote IP address (%r) and one local IP address (% > l) for connections. %r is already logged, but you can add %l if that > helps. Other than that, I can't really help. Yeah, but that is the IP of the mail server, and since I have only one, it doesn't help any. No worries, I did see how to see this on my perimeter firewall (thanks Stan), so I can see what I'm looking for now. Thanks Timo, -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From davide.marchi at mail.cgilfe.it Mon Oct 1 12:00:14 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:00:14 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_011708_663506_8DEC5391) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) Message-ID: <50695B9E.3090502@mail.cgilfe.it> Thank you very much for the reply, i' have installed the supplied patch with the following command: - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) i compile the plugin and all go easy i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell me ..blah..blah.. [CANNOT] Failed to call crm114 binary I cant see in any log what's the problem .... Il 01/10/2012 01:16, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich > wrote: > >> Did you see anything in syslog? If not, let's begin with the attached >> patch. > > Ouch... too sleepy. Here's the correct patch. > From davide.marchi at mail.cgilfe.it Mon Oct 1 12:04:30 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:04:30 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) Message-ID: <50695C9E.5040101@mail.cgilfe.it> I downloaded and applaied the patch in the message 01:16 of 716 bytes: diff -r 7f94cc6b4d8e src/crm114.c --- a/src/crm114.c Fri May 11 04:05:59 2012 +0300 +++ b/src/crm114.c Mon Oct 01 02:15:40 2012 +0300 @@ -56,11 +56,17 @@ * really only needs the signature. */ if (pipe(pipes)) + { + i_debug("Failed to create pipes"); return -1; + } pid = fork(); if (pid < 0) + { + i_debug("Couldn't fork"); return -1; + } if (pid) { @@ -117,6 +123,7 @@ argv[i++] = spam ? cfg->spam : cfg->non_spam; + i_debug("Executing '%s %s'", cfg->binary, t_strarray_join(argv, " ")); execv(cfg->binary, (char *const *) argv); /* fall through if reaver can't be found */ i_debug("executing %s failed: %d (uid=%d, gid=%d)", cfg->binary, errno, Il 01/10/2012 11:00, Davide ha scritto: > Thank you very much for the reply, i' have installed the supplied patch > with the following command: > > - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) > i compile the plugin and all go easy > > i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell > me ..blah..blah.. [CANNOT] Failed to call crm114 binary > I cant see in any log what's the problem .... > > Il 01/10/2012 01:16, Eugene Paskevich ha scritto: >> On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich >> wrote: >> >>> Did you see anything in syslog? If not, let's begin with the attached >>> patch. >> >> Ouch... too sleepy. Here's the correct patch. >> From davide.marchi at mail.cgilfe.it Mon Oct 1 12:08:33 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:08:33 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695C9E.5040101@mail.cgilfe.it> (sfid-20121001_110525_429072_9195F28B) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) <50695C9E.5040101@mail.cgilfe.it> (sfid-20121001_110525_429072_9195F28B) Message-ID: <50695D91.3090204@mail.cgilfe.it> I have recived an empty message in response Il 01/10/2012 11:04, Davide ha scritto: > I downloaded and applaied the patch in the message 01:16 of 716 bytes: > > diff -r 7f94cc6b4d8e src/crm114.c > --- a/src/crm114.c Fri May 11 04:05:59 2012 +0300 > +++ b/src/crm114.c Mon Oct 01 02:15:40 2012 +0300 > @@ -56,11 +56,17 @@ > * really only needs the signature. > */ > if (pipe(pipes)) > + { > + i_debug("Failed to create pipes"); > return -1; > + } > > pid = fork(); > if (pid < 0) > + { > + i_debug("Couldn't fork"); > return -1; > + } > > if (pid) > { > @@ -117,6 +123,7 @@ > > argv[i++] = spam ? cfg->spam : cfg->non_spam; > > + i_debug("Executing '%s %s'", cfg->binary, t_strarray_join(argv, " ")); > execv(cfg->binary, (char *const *) argv); > /* fall through if reaver can't be found */ > i_debug("executing %s failed: %d (uid=%d, gid=%d)", cfg->binary, > errno, > > > Il 01/10/2012 11:00, Davide ha scritto: >> Thank you very much for the reply, i' have installed the supplied patch >> with the following command: >> >> - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) >> i compile the plugin and all go easy >> >> i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell >> me ..blah..blah.. [CANNOT] Failed to call crm114 binary >> I cant see in any log what's the problem .... >> >> Il 01/10/2012 01:16, Eugene Paskevich ha scritto: >>> On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich >>> wrote: >>> >>>> Did you see anything in syslog? If not, let's begin with the attached >>>> patch. >>> >>> Ouch... too sleepy. Here's the correct patch. >>> From fabio.ferrari at unimore.it Mon Oct 1 12:15:14 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Mon, 1 Oct 2012 11:15:14 +0200 Subject: [Dovecot] Problem with process_limit Message-ID: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Hello, i have a problem with the process_limit configuration. Occasionally, it happens that the dovecot.log shows this line: master: Warning: service(imap): process_limit reached, client connections are being dropped So I checked, the process number, with the command: ps auxwww | grep imap | grep -v login | wc -l and it shows 1024. Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line process_limit = 1500 I checked if the dovecot had accepted the change with the command doveconf -n | grep process_limit and it shows process_limit = 1500 But it is dropping the connections when they reach 1024 anyway, the configuration parameter is totally ignored. Can anyone help? Am I editing the right parameter? thanks in advance Fabio Ferrari P.S. The version of the dovecot is dovecot-2.0.1-1_118.el5 on Red Hat Enterprise Linux Server release 5.8 (Tikanga). From dovecot at tvetc.de Mon Oct 1 12:17:48 2012 From: dovecot at tvetc.de (Karim 'Kasi Mir' Senoucci) Date: Mon, 01 Oct 2012 11:17:48 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders Message-ID: <50695FBC.6080403@tvetc.de> Hello everyone, I recently updated to dovecot 2.0.19 (in fact, I updated the whole system to Ubuntu 12.04 LTS, I am using the dovecot from the Ubuntu packages) and just today found out that the dovecot-lda for some reason doesn't auto-create missing folders anymore as it did with my old installation (1.0.10). I have a "system users" layout and send my mails through a user-specific procmail filter. Every delivery in those filters is done via dovecot-lda using a line list this: > |$DELIVER -d archive -m lists.hylafax-`date "+%Y%m"` where $DELIVER is set to > DELIVER="/usr/bin/sudo /usr/lib/dovecot/dovecot-lda" (I use the sudo because some of the procmail lines deliver the mail to folders in a different user's mailbox.) This works fine for all users, unless the mail folder to be delivered to doesn't exist yet. Here's an example of such a delivery attempt: > Oct 01 10:57:09 lda: Debug: auth input: archive > system_groups_user=archive uid=1002 gid=1002 home=/home/archive > Oct 01 10:57:09 lda(archive): Debug: Effective uid=1002, gid=1002, > home=/home/archive > Oct 01 10:57:09 lda(archive): Debug: Namespace : type=private, > prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes > location=maildir:/home/archive/Maildir > Oct 01 10:57:09 lda(archive): Debug: maildir++: > root=/home/archive/Maildir, index=, control=, > inbox=/home/archive/Maildir, alt= > Oct 01 10:57:09 lda(archive): Debug: none: root=, index=, control=, > inbox=, alt= > Oct 01 10:57:09 lda(archive): Debug: Destination address: xxx at xxxx.de > (source: user at hostname) > Oct 01 10:57:09 lda(archive): Debug: Namespace : > /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using > default permissions > Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions > from /home/archive/Maildir: mode=0755 gid=-1 > Oct 01 10:57:09 lda(archive): Debug: Namespace : > /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using > default permissions > Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions > from /home/archive/Maildir: mode=0755 gid=-1 > Oct 01 10:57:09 lda(archive): Info: > msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: save failed > to open mailbox lists.hylafax-201210: Mailbox doesn't exist: > lists.hylafax-201210 > Oct 01 10:57:09 lda(archive): Info: > msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: saved mail to > INBOX Can anyone tell me what goes wrong here and how to fix it? From what I could find out dovecot-lda should auto-create those mailbox folders, but somehow it doesn't. Any help is appreciated. Thanks in advance Kasi Mir From eugene at raptor.kiev.ua Mon Oct 1 12:20:06 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 12:20:06 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695B9E.3090502@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 12:00:14 +0300, Davide wrote: > Thank you very much for the reply, i' have installed the supplied patch > with the following command: > > - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) > i compile the plugin and all go easy > > i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell > me ..blah..blah.. [CANNOT] Failed to call crm114 binary > I cant see in any log what's the problem .... Weird... Did you configure anything specific about logging? doveconf -n might show. You should also check your syslog configuration as to where debugging logging should go to. Otherwise I have no idea. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From robert at schetterer.org Mon Oct 1 12:46:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 01 Oct 2012 11:46:58 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders In-Reply-To: <50695FBC.6080403@tvetc.de> References: <50695FBC.6080403@tvetc.de> Message-ID: <50696692.3050700@schetterer.org> Am 01.10.2012 11:17, schrieb Karim 'Kasi Mir' Senoucci: > Hello everyone, > I recently updated to dovecot 2.0.19 (in fact, I updated the whole > system to Ubuntu 12.04 LTS, I am using the dovecot from the Ubuntu > packages) and just today found out that the dovecot-lda for some reason > doesn't auto-create missing folders anymore as it did with my old > installation (1.0.10). > > I have a "system users" layout and send my mails through a user-specific > procmail filter. Every delivery in those filters is done via dovecot-lda > using a line list this: > >> |$DELIVER -d archive -m lists.hylafax-`date "+%Y%m"` > > where $DELIVER is set to > >> DELIVER="/usr/bin/sudo /usr/lib/dovecot/dovecot-lda" > > (I use the sudo because some of the procmail lines deliver the mail to > folders in a different user's mailbox.) > > This works fine for all users, unless the mail folder to be delivered to > doesn't exist yet. Here's an example of such a delivery attempt: > >> Oct 01 10:57:09 lda: Debug: auth input: archive >> system_groups_user=archive uid=1002 gid=1002 home=/home/archive >> Oct 01 10:57:09 lda(archive): Debug: Effective uid=1002, gid=1002, >> home=/home/archive >> Oct 01 10:57:09 lda(archive): Debug: Namespace : type=private, >> prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes >> location=maildir:/home/archive/Maildir >> Oct 01 10:57:09 lda(archive): Debug: maildir++: >> root=/home/archive/Maildir, index=, control=, >> inbox=/home/archive/Maildir, alt= >> Oct 01 10:57:09 lda(archive): Debug: none: root=, index=, control=, >> inbox=, alt= >> Oct 01 10:57:09 lda(archive): Debug: Destination address: xxx at xxxx.de >> (source: user at hostname) >> Oct 01 10:57:09 lda(archive): Debug: Namespace : >> /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using >> default permissions >> Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions >> from /home/archive/Maildir: mode=0755 gid=-1 >> Oct 01 10:57:09 lda(archive): Debug: Namespace : >> /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using >> default permissions >> Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions >> from /home/archive/Maildir: mode=0755 gid=-1 >> Oct 01 10:57:09 lda(archive): Info: >> msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: save failed >> to open mailbox lists.hylafax-201210: Mailbox doesn't exist: >> lists.hylafax-201210 >> Oct 01 10:57:09 lda(archive): Info: >> msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: saved mail to >> INBOX > > Can anyone tell me what goes wrong here and how to fix it? From what I > could find out dovecot-lda should auto-create those mailbox folders, but > somehow it doesn't. Any help is appreciated. > > Thanks in advance > Kasi Mir my bet goes here http://wiki2.dovecot.org/LDA parameters lda_mailbox_autocreate lda_mailbox_autosubscribe -- Best Regards MfG Robert Schetterer From davide.marchi at mail.cgilfe.it Mon Oct 1 12:54:01 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:54:01 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_112051_451409_99CBB428) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_112051_451409_99CBB428) Message-ID: <50696839.7090901@mail.cgilfe.it> this is my dovecot configuration for antispam plugin logging: antispam_verbose_debug = 1 antispam_debug_target = syslog and this is info_log_path = /var/log/dovecot/dovecot.log log_path = /var/log/dovecot/dovecot-err.log debug_log_path = Il 01/10/2012 11:20, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:00:14 +0300, Davide > wrote: > >> Thank you very much for the reply, i' have installed the supplied >> patch with the following command: >> >> - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) >> i compile the plugin and all go easy >> >> i move my wrongly tagged mail from UNSURE to INBOX and thunderbird >> tell me ..blah..blah.. [CANNOT] Failed to call crm114 binary >> I cant see in any log what's the problem .... > > Weird... Did you configure anything specific about logging? > doveconf -n might show. You should also check your syslog configuration > as to where debugging logging should go to. Otherwise I have no idea. > From dovecot at tvetc.de Mon Oct 1 13:01:39 2012 From: dovecot at tvetc.de (Karim 'Kasi Mir' Senoucci) Date: Mon, 01 Oct 2012 12:01:39 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders In-Reply-To: <50696692.3050700@schetterer.org> References: <50695FBC.6080403@tvetc.de> <50696692.3050700@schetterer.org> Message-ID: <50696A03.3080605@tvetc.de> Hello everyone, Am 01.10.2012 11:46, schrieb Robert Schetterer: >> Can anyone tell me what goes wrong here and how to fix it? From what I >> could find out dovecot-lda should auto-create those mailbox folders, but >> somehow it doesn't. Any help is appreciated. > my bet goes here > > parameters > > lda_mailbox_autocreate > lda_mailbox_autosubscribe D*mn, I fell into the trap again. I was searching for such parameters before posting my question here, but couldn't find them - because I was looking in the 1.x docmentation, not the 2.x one. Thanks for pointing that out to me. Greetings Kasi Mir From eugene at raptor.kiev.ua Mon Oct 1 13:07:48 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 13:07:48 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50696839.7090901@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 12:54:01 +0300, Davide wrote: > this is my dovecot configuration for antispam plugin logging: > antispam_verbose_debug = 1 > antispam_debug_target = syslog Neither my plugin nor (I believe) main dovecot reads those two. You can remove them. > and this is > info_log_path = /var/log/dovecot/dovecot.log > log_path = /var/log/dovecot/dovecot-err.log > debug_log_path = Did you notice that patch is mainly for i_debug() invocations? :-) Try sitting debug_log_path to something meaningful. And of course restart dovecot. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From davide.marchi at mail.cgilfe.it Mon Oct 1 13:57:14 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 12:57:14 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_120839_719900_279FDE63) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> (sfid-20121001_120839_719900_279FDE63) Message-ID: <5069770A.5030007@mail.cgilfe.it> I deleted antispam_verbose_debug = 1 antispam_debug_target = syslog from dovecot.conf and i added debug_log_path = /var/log/dovecot/dovecot_debug.log restarted dovecot and now i see in the debug log 2012-10-01 12:33:31 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib90_antispam_plugin.so but nothing inherit [CANNOT] Failed to call crm114 binary Il 01/10/2012 12:07, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:54:01 +0300, Davide > wrote: > >> this is my dovecot configuration for antispam plugin logging: >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog > > Neither my plugin nor (I believe) main dovecot reads those two. You can > remove them. > >> and this is >> info_log_path = /var/log/dovecot/dovecot.log >> log_path = /var/log/dovecot/dovecot-err.log >> debug_log_path = > > Did you notice that patch is mainly for i_debug() invocations? :-) > Try sitting debug_log_path to something meaningful. And of course > restart dovecot. > From davide.marchi at mail.cgilfe.it Mon Oct 1 13:58:23 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 12:58:23 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_120839_719900_279FDE63) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> (sfid-20121001_120839_719900_279FDE63) Message-ID: <5069774F.30209@mail.cgilfe.it> Parameters i.e. antispam_crm_binary = /opt/crm114/mailreaver.crm must be enclosed on ""? so antispam_crm_binary = "/opt/crm114/mailreaver.crm" Il 01/10/2012 12:07, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:54:01 +0300, Davide > wrote: > >> this is my dovecot configuration for antispam plugin logging: >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog > > Neither my plugin nor (I believe) main dovecot reads those two. You can > remove them. > >> and this is >> info_log_path = /var/log/dovecot/dovecot.log >> log_path = /var/log/dovecot/dovecot-err.log >> debug_log_path = > > Did you notice that patch is mainly for i_debug() invocations? :-) > Try sitting debug_log_path to something meaningful. And of course > restart dovecot. > From eugene at raptor.kiev.ua Mon Oct 1 14:00:22 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 14:00:22 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069774F.30209@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069774F.30209@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 13:58:23 +0300, Davide wrote: > Parameters i.e. antispam_crm_binary = /opt/crm114/mailreaver.crm > must be enclosed on ""? so > antispam_crm_binary = "/opt/crm114/mailreaver.crm" No. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From eugene at raptor.kiev.ua Mon Oct 1 14:01:14 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 14:01:14 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069770A.5030007@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 13:57:14 +0300, Davide wrote: > I deleted > antispam_verbose_debug = 1 > antispam_debug_target = syslog > from dovecot.conf and i added > > debug_log_path = /var/log/dovecot/dovecot_debug.log > > restarted dovecot and now i see in the debug log > > 2012-10-01 12:33:31 imap: Debug: Module loaded: > /usr/local/lib/dovecot/lib90_antispam_plugin.so > > but nothing inherit [CANNOT] Failed to call crm114 binary Could you please post the contents of the debug file somewhere? -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From davide.marchi at mail.cgilfe.it Mon Oct 1 14:48:31 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 13:48:31 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_130205_017099_DD5462AE) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> (sfid-20121001_130205_017099_DD5462AE) Message-ID: <5069830F.4060401@mail.cgilfe.it> Can i attach compressed log in a post? I can produce output replacing crm binary with a script bash to ouput command,user etc etc Il 01/10/2012 13:01, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 13:57:14 +0300, Davide > wrote: > >> I deleted >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog >> from dovecot.conf and i added >> >> debug_log_path = /var/log/dovecot/dovecot_debug.log >> >> restarted dovecot and now i see in the debug log >> >> 2012-10-01 12:33:31 imap: Debug: Module loaded: >> /usr/local/lib/dovecot/lib90_antispam_plugin.so >> >> but nothing inherit [CANNOT] Failed to call crm114 binary > > Could you please post the contents of the debug file somewhere? > From davide.marchi at mail.cgilfe.it Mon Oct 1 14:53:08 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 13:53:08 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_134931_162985_54BB8C15) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> (sfid-20121001_130205_017099_DD5462AE) <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_134931_162985_54BB8C15) Message-ID: <50698424.8080208@mail.cgilfe.it> Restarting Dovecot instead to use doveadm reload i hav strange error: managesieve(root): Fatal: getcwd() failed: No such file or directory doveconf: Error: managesieve-login: dump-capability process returned 89 Il 01/10/2012 13:48, Davide ha scritto: > Can i attach compressed log in a post? > I can produce output replacing crm binary with a script bash to ouput > command,user etc etc > > > Il 01/10/2012 13:01, Eugene Paskevich ha scritto: >> On Mon, 01 Oct 2012 13:57:14 +0300, Davide >> wrote: >> >>> I deleted >>> antispam_verbose_debug = 1 >>> antispam_debug_target = syslog >>> from dovecot.conf and i added >>> >>> debug_log_path = /var/log/dovecot/dovecot_debug.log >>> >>> restarted dovecot and now i see in the debug log >>> >>> 2012-10-01 12:33:31 imap: Debug: Module loaded: >>> /usr/local/lib/dovecot/lib90_antispam_plugin.so >>> >>> but nothing inherit [CANNOT] Failed to call crm114 binary >> >> Could you please post the contents of the debug file somewhere? >> From eugene at raptor.kiev.ua Mon Oct 1 15:04:50 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 15:04:50 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069830F.4060401@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 14:48:31 +0300, Davide wrote: > Can i attach compressed log in a post? If it's of the sane size :-) But I'm actually interested in the output which was triggered by the mail move itself. > I can produce output replacing crm binary with a script bash to ouput > command,user etc etc Wait a second... So the script is ran correctly but reaver isn't? That probably means that either reaver dislikes its arguments or its stdin... -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From jbates at brightok.net Mon Oct 1 15:23:38 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 07:23:38 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: References: <506842CB.8080501@Media-Brokers.com> <5068582A.6030507@brightok.net> Message-ID: <50698B4A.7090604@brightok.net> On 10/1/2012 2:58 AM, David Ledger wrote: > At 09:33 -0500 30/9/12, Jack Bates wrote: >> On 9/30/2012 8:02 AM, Charles Marcus wrote: >>> Hi Timo/everyone, >>> >>> Currently we are logging the remote IP, but is there a way to show >>> the IP address that the NAT connection is coming from? >>> >>> The reason I ask is, we are changing ISPs, and I would like to see >>> in the logs when an external connection is coming from our OLD ISP >>> connection, and when it is coming through our new one. >>> >>> We have a Watchguard firewall, and I have both External connections >>> setup and working, and have just pointed our DNS records to the new >>> public IP, and would like to be able to see which WAN connection/IP >>> they are coming from. >> >> You could bind 2 internal IP Addresses to the server and have each >> NAT translation go to a different internal IP. >> >> Jack > > From my remembrances of the packet layout there is nowhere in the > packet for the pre-NAT address to live. The only place the mapping is > stored is in the internal tables of the NAT router which has to know > where to send the reply packets. > > David > > Public IP1 -> 192.168.0.33 Public IP2 -> 192.168.0.34 Now we just track the internal address in our logs, since each public IP is mapped to a different internal IP. Jack From jbates at brightok.net Mon Oct 1 15:35:03 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 07:35:03 -0500 Subject: [Dovecot] Problem with process_limit In-Reply-To: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: <50698DF7.60209@brightok.net> On 10/1/2012 4:15 AM, FABIO FERRARI wrote: > Hello, > > i have a problem with the process_limit configuration. > > Occasionally, it happens that the dovecot.log shows this line: > master: Warning: service(imap): process_limit reached, client connections > are being dropped > > So I checked, the process number, with the command: > ps auxwww | grep imap | grep -v login | wc -l > and it shows 1024. > > Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line > process_limit = 1500 > > I checked if the dovecot had accepted the change with the command > doveconf -n | grep process_limit > and it shows > process_limit = 1500 > > But it is dropping the connections when they reach 1024 anyway, the > configuration parameter is totally ignored. > > Can anyone help? Am I editing the right parameter? > > thanks in advance > > Fabio Ferrari > > P.S. The version of the dovecot is dovecot-2.0.1-1_118.el5 on Red Hat > Enterprise Linux Server release 5.8 (Tikanga). Don't forget to change the operating system limit as well. I added these lines to my /etc/sysconfig/dovecot on rhel6. I compiled dovecot myself, but I package similar to the system version. ulimit -n 4096 ulimit -u 5120 This increases the filehandles allowed by dovecot to 4096 and increases the number of processes per user to 5120. This is a proxy server, so I needed to support much larger numbers. than a silly 1024. Jack From davide.marchi at mail.cgilfe.it Mon Oct 1 18:46:53 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 17:46:53 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_143220_258050_A12B6D1E) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_143220_258050_A12B6D1E) Message-ID: <5069BAED.8020206@mail.cgilfe.it> I made an experiment because iwasn't able to output some error to syslog or stderr about my configuration... I've installed antispam plugin in a parallel server with the program of johannes Bergs that seem now upgraded to function with 2.X and i have same problem ("Failed to call crm114 binary.." in antispam dovecot from your program, Eugene, and "Failed to call reaver.." from Johannes program) but now i'm able to output to log with this config: antispam_debug_target = syslog antispam_debug_target = stderr antispam_verbose_debug = 1 antispam_backend = crm114 antispam_spam_pattern_ignorecase = spam* antispam_trash_pattern_ignorecase = trash* antispam_unsure_pattern_ignorecase = unsure* antispam_signature = X-CRM114-CacheID antispam_signature_missing = error antispam_crm_binary = /usr/local/bin/piper_log.sh antispam_crm_spam_arg = --spam antispam_crm_notspam_arg = --good # antispam_crm_binary = /opt/crm114/mailreaver.crm antispam_crm_args = -u;%h;--fileprefix=/opt/crm114/ # antispam_signature = X-CRM114-CacheID in dovecot-err.log now i have 2012-10-01 17:39:35 imap: Error: antispam: plugin initialising (2.0-4-g83b0b4b-dirty) 2012-10-01 17:39:35 imap: Error: antispam: verbose debug enabled 2012-10-01 17:39:35 imap: Error: antispam: "SPAM" is exact match spam folder 2012-10-01 17:39:35 imap: Error: antispam: "UNSURE" is exact match unsure folder 2012-10-01 17:39:35 imap: Error: antispam: "Trash" is exact match trash folder 2012-10-01 17:39:35 imap: Error: antispam: reaver binary set to /usr/local/bin/piper_log.sh 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg -u 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg /home/vpopmail/domains/mail.cgilfe.it/davide.marchi 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg --fileprefix=/opt/crm114/ 2012-10-01 17:39:35 imap: Error: antispam: signature header line is "X-CRM114-CacheID" 2012-10-01 17:39:39 imap: Error: antispam: plugin initialising (2.0-4-g83b0b4b-dirty) 2012-10-01 17:39:39 imap: Error: antispam: verbose debug enabled . . . 2012-10-01 17:39:42 imap: Error: antispam: mail copy: from trash: 0, to trash: 0 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_spam(SPAM): 1 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_spam(INBOX): 0 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_unsure(SPAM): 0 2012-10-01 17:39:42 imap: Error: antispam: mail copy: src spam: 1, dst spam: 0, src unsure: 0 (i moved a mail from SPAM to INBOX) this is the output for "call command crm args" /opt/crm114/mailreaver.crm --good -u /home/vpopmail/domains/mail.cgilfe.it/davide.marchi --fileprefix=/opt/crm114/ if i exec this command with user vpopmail in console the command is successful Il 01/10/2012 14:04, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 14:48:31 +0300, Davide > wrote: > >> Can i attach compressed log in a post? > > If it's of the sane size :-) > But I'm actually interested in the output which was triggered by the > mail move itself. > >> I can produce output replacing crm binary with a script bash to ouput >> command,user etc etc > > Wait a second... So the script is ran correctly but reaver isn't? > That probably means that either reaver dislikes its arguments or its > stdin... > From fabiodepin at gmail.com Mon Oct 1 22:20:50 2012 From: fabiodepin at gmail.com (Fabio Depin) Date: Mon, 1 Oct 2012 16:20:50 -0300 Subject: [Dovecot] BUG to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2 Message-ID: Hello, Today I needed to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2. When running 'make' getting the following error: -------------------------------------------------- -------- db-checkpassword.c: In function 'sigchld_handler': db-checkpassword.c: 426: error: assignment of read-only member '__in' db-checkpassword.c: 429: error: assignment of read-only member '__in' db-checkpassword.c: 431: error: assignment of read-only member '__in' db-checkpassword.c: 432: error: assignment of read-only member '__in' make [3]: ** [db-checkpassword.o] Error 1 make [3]: ** Waiting for other processes to finish. mv-f .deps / auth-worker-server.Tpo .deps / auth-worker-server.Po make [3]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src/auth ' make [2]: ** [all-recursive] Error 1 make [2]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src ' make [1]: ** [all-recursive] Error 1 make [1]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7 ' make: ** [all] Error 2 -------------------------------------------------- -------- -To work did the following: -------------------------------------------------- -------- 417a418 + Int stat = status-> status; 426c427 - If (WIFSIGNALED (status-> status)) { --- + If (WIFSIGNALED (stat)) { 429c430 - Dec2str (status-> pid), WTERMSIG (status-> status)); --- + Dec2str (status-> pid), WTERMSIG (stat)); 431.432 c432, 433 -} Else if (WIFEXITED (status-> status)) { - Request-> exit_status WEXITSTATUS = (status-> status); --- +} Else if (WIFEXITED (stat)) { + Request-> exit_status WEXITSTATUS = (stat); -------------------------------------------------- -------- With this change worked perfectly ntanto in debian 4 with gcc 4.1.2, as in debian 6 with gcc 4.4.5. -I wonder if I made the change may affect any function of dovecot, or if it is correct. Thank you for your attention. Fabio Depin From joe at netmusician.org Mon Oct 1 22:34:25 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 01 Oct 2012 15:34:25 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <506453CE.7000608@gmail.com> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> <506453CE.7000608@gmail.com> Message-ID: <5069F041.6060904@netmusician.org> Are performance issues a possible symptom of this problem? It was mentioned that this happens after disconnects, but does this break IDLE? > Birta Levente > September 27, 2012 9:25 AM > > Hi Timo > > I just want to ask you: this issue is still in your task list? > If you need more debug information please tell me how can I give you. > > Thanks, > Levi > > Timo Sirainen > September 24, 2012 10:58 AM > > Show your doveconf -n output at least. As for debugging information, > that would depend on how you installed Dovecot? From some RPM or sources? > > Birta Levente > September 24, 2012 10:55 AM > > > I have the same problem, but on centos 6.3 64bit. How can I give you > the debug information? > > Levi > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: postbox-contact.jpg Type: image/jpeg Size: 1305 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Mon Oct 1 23:13:50 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 01 Oct 2012 16:13:50 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <5069F041.6060904@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> <506453CE.7000608@gmail.com> <5069F041.6060904@netmusician.org> Message-ID: <5069F97E.5020900@netmusician.org> Are performance issues a possible symptom of this problem? It was mentioned that this happens after disconnects, but does this break IDLE? > > Birta Levente > September 27, 2012 9:25 AM > > Hi Timo > > I just want to ask you: this issue is still in your task list? > If you need more debug information please tell me how can I give you. > > Thanks, > Levi > > > Timo Sirainen > September 24, 2012 10:58 AM > > Show your doveconf -n output at least. As for debugging information, > that would depend on how you installed Dovecot? From some RPM or sources? > > > Birta Levente > September 24, 2012 10:55 AM > > > I have the same problem, but on centos 6.3 64bit. How can I give you > the debug information? > > Levi > > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------ From dovecot-in at keystealth.org Mon Oct 1 23:36:25 2012 From: dovecot-in at keystealth.org (Scott Neville) Date: Mon, 1 Oct 2012 13:36:25 -0700 (PDT) Subject: [Dovecot] Logging IP address for failed login Message-ID: Hi, I am trying to use the logs to show the IP that brute force activity comes from, but Im not succeeding. I have read the archives and seen the advice others have had. I can see logs for repeated bad logins, but I need the IP address from the attempts. dovecot 2.0.12 / CentOS 5.4 / imaps only (993) I have tried a bunch of different combinations of 10-logging.conf settings. This is what I have currently (that does not work the way I want): auth_verbose = yes #auth_verbose_passwords = no #auth_debug = yes #auth_debug_passwords = no #mail_debug = no I *dont* want to see the passwords, either failed or successful. I just want to see failed logins for whatever reason and the IP they came from. In /var/log/maillog I get lines like this: Oct 1 04:19:12 olive dovecot: auth: pam(marketing): unknown user Oct 1 04:19:17 olive dovecot: auth: pam(marketing): unknown user When i had debugging turned on, I would get lines like this: Sep 9 01:14:59 olive dovecot: auth: Debug: passwd(dbelan,62.128.300.94): lookup but only for successful logins. The brute force attempts dont log like that: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:58 olive dovecot: auth: pam(backup): unknown user No IP anywhere in that. fail2ban seems to rely on the pop-login or imap-login lines to pull the IP from. I get an imap-login for my real logins: Oct 1 12:38:56 olive dovecot: imap-login: Login: user=, method=PLAIN, rip=62.128.300.94, lip=204.152.189.165, mpid=20360, TLS but no similar line for the failed logins. So is this a dovecot logging configuration combination I need to find? Is it getting lost in pam? Is it specific to CentOS? Any help appreciated - happy to read up on it myself, but would need a pointer, since the docs so far either assume I get an imap-login line for failed logins which I dont, or they assume I just want to see the repeated attempts/passwords. Scott. From fabiodepin at gmail.com Mon Oct 1 23:51:39 2012 From: fabiodepin at gmail.com (Fabio Depin) Date: Mon, 1 Oct 2012 17:51:39 -0300 Subject: [Dovecot] BUG to compile Plugin - deleted-to-trash on dovecot 2.1+ Message-ID: Hello, Today I needed to compile the plugin deleted-to-trash for dovecot 1.2.10 (> 2.1) and had problems compiling. To solve the problem efetuei changes below: -------------------------------------------------- -------------------- deleted-to-trash-plugin.c 79.80 C79 - Mailbox_alloc box = (list, name, MAILBOX_FLAG_KEEP_RECENT | - MAILBOX_FLAG_NO_INDEX_FILES); --- + Box = mailbox_alloc (list, name, MAILBOX_FLAG_NO_INDEX_FILES); 136c135 - If (keywords! = NULL) mailbox_keywords_unref (trash_box, & keywords); --- + If (keywords! = NULL) mailbox_keywords_unref (& keywords); -------------------------------------------------- -------------------- I would like to verify that this is correct, or can generate a problem. Thank you for your attention. Fabio Depin From jbates at brightok.net Tue Oct 2 00:15:54 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 16:15:54 -0500 Subject: [Dovecot] Logging IP address for failed login In-Reply-To: References: Message-ID: <506A080A.60906@brightok.net> On 10/1/2012 3:36 PM, Scott Neville wrote: > > In /var/log/maillog I get lines like this: > Oct 1 04:19:12 olive dovecot: auth: pam(marketing): unknown user > Oct 1 04:19:17 olive dovecot: auth: pam(marketing): unknown user > I'm guessing you are using a centos package. This may be package version specific. Here is RHEL6's dovecot 2.0.9 default except for setting auth_verbose = yes. Sep 28 21:12:10 compiler dovecot: auth: pam(test,::1): unknown user Sep 28 21:12:24 compiler dovecot: auth: pam(validuser,::1): pam_authenticate() failed: Authentication failure (password mismatch?) 2.1.9/2.1.10 which I packaged shows similar. Since I connected localhost, the IP is IPv6, of course. Jack From jbates at brightok.net Tue Oct 2 04:42:36 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 20:42:36 -0500 Subject: [Dovecot] BUG to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2 In-Reply-To: References: Message-ID: <506A468C.10505@brightok.net> It looks like this might be a bug in glibc 2.3.3 http://sourceware.org/bugzilla/show_bug.cgi?id=1392 Jack On 10/1/2012 2:20 PM, Fabio Depin wrote: > Hello, > > Today I needed to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2. > When running 'make' getting the following error: > -------------------------------------------------- -------- > db-checkpassword.c: In function 'sigchld_handler': > db-checkpassword.c: 426: error: assignment of read-only member '__in' > db-checkpassword.c: 429: error: assignment of read-only member '__in' > db-checkpassword.c: 431: error: assignment of read-only member '__in' > db-checkpassword.c: 432: error: assignment of read-only member '__in' > make [3]: ** [db-checkpassword.o] Error 1 > make [3]: ** Waiting for other processes to finish. > mv-f .deps / auth-worker-server.Tpo .deps / auth-worker-server.Po > make [3]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src/auth ' > make [2]: ** [all-recursive] Error 1 > make [2]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src ' > make [1]: ** [all-recursive] Error 1 > make [1]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7 ' > make: ** [all] Error 2 > -------------------------------------------------- -------- > > -To work did the following: > -------------------------------------------------- -------- > 417a418 > + Int stat = status-> status; > 426c427 > - If (WIFSIGNALED (status-> status)) { > --- > + If (WIFSIGNALED (stat)) { > 429c430 > - Dec2str (status-> pid), WTERMSIG (status-> status)); > --- > + Dec2str (status-> pid), WTERMSIG (stat)); > 431.432 c432, 433 > -} Else if (WIFEXITED (status-> status)) { > - Request-> exit_status WEXITSTATUS = (status-> status); > --- > +} Else if (WIFEXITED (stat)) { > + Request-> exit_status WEXITSTATUS = (stat); > -------------------------------------------------- -------- > > With this change worked perfectly ntanto in debian 4 with gcc 4.1.2, as in > debian 6 with gcc 4.4.5. > -I wonder if I made the change may affect any function of dovecot, or if it is > correct. > > Thank you for your attention. > Fabio Depin > From amateo at um.es Tue Oct 2 11:41:51 2012 From: amateo at um.es (Angel L. Mateo) Date: Tue, 02 Oct 2012 10:41:51 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> Message-ID: <506AA8CF.8090605@um.es> Hello, I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: root at myotis30:/etc/dovecot/conf.d# doveadm user angel.luis at um.es userdb: angel.luis at um.es mail : mdbox:/home/alumnos/46/113246/mdbox:INDEX=/var/indexes/mdbox/angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=10G root at myotis30:/etc/dovecot/conf.d# doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=10G I guess I'm using different keys depending the user database used. I have configured three user databases, one for master-password, one for a ldap server and the other with pam (I need it because my webmail users authenticate in my SSO system through PAM). This is my config: passdb { driver = passwd-file master = yes args = /etc/dovecot/master-users # Unless you're using PAM, you probably still want the destination user to # be looked up from passdb that it really exists. pass=yes does that. pass = yes } passdb { driver = pam # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=] # [cache_key=] [] #args = dovecot args = session=yes cache_key=%n dovecot } passdb { driver = ldap # Path for LDAP configuration file, see example-config/dovecot-ldap.conf.ext args = /etc/dovecot/dovecot-ldap.conf.ext } # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # userdb { driver = prefetch } userdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf.ext # Default fields can be used to specify defaults that LDAP may override #default_fields = home=/home/virtual/%u } In my ldap configuration, I have a filter that looks for the uid of the user or the hole email: user_filter = (&()(|(uid=%u)(mail=%u))) I need this, because I have users that authenticate with just his/her login, not the complete email address. How can I unify those entries, so they use always just the login as key? El 18/09/12 18:31, Timo Sirainen escribi?: > On 18.9.2012, at 9.59, Angel L. Mateo wrote: > >>>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >>> >>> Works in my tests. >>> >> Is this cache the same than the user information cache? > > Yes. > >> The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: >> >> doveadm auth cache flush > > What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: > > doveadm auth cache flush foo # make sure it isn't there > doveadm user foo > doveadm auth cache flush foo > > Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. > > You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. > -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.4.0-030400-generic x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1000 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_exceeded_message = El mensaje no se ha entregado porque el destinatario del mismo tiene el buz?n lleno. quota_rule = *:storage=20G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 2003 } service auth { client_limit = 3000 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 2 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 2 } ssl = no ssl_cert = } From davide.marchi at mail.cgilfe.it Tue Oct 2 11:57:33 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Tue, 02 Oct 2012 10:57:33 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_143220_258050_A12B6D1E) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_143220_258050_A12B6D1E) Message-ID: <506AAC7D.8060408@mail.cgilfe.it> I'm unable to output nothing to syslog nor in other dovecot's log files about problem of plugin. If i use the test server the output is this: Oct 2 10:38:34 debnew imap: antispam: mailbox_is_unsure(SPAM): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_trash(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_trash(SPAM): 0 Oct 2 10:38:34 debnew imap: antispam: mail copy: from trash: 0, to trash: 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_spam(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_spam(SPAM): 1 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_unsure(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Oct 2 10:38:34 debnew imap: antispam: /usr/local/bin/piper_log.sh --spam -u /home/vpopmail/domains/mail.cgilfe.it/davide.marchi --fileprefix=/opt/crm114/ and this is correct but anyway thunderbird say [CANNOT] Failed to call reaver Il 01/10/2012 14:04, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 14:48:31 +0300, Davide > wrote: > >> Can i attach compressed log in a post? > > If it's of the sane size :-) > But I'm actually interested in the output which was triggered by the > mail move itself. > >> I can produce output replacing crm binary with a script bash to ouput >> command,user etc etc > > Wait a second... So the script is ran correctly but reaver isn't? > That probably means that either reaver dislikes its arguments or its > stdin... > From tss at iki.fi Tue Oct 2 21:37:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:37:09 +0300 Subject: [Dovecot] LTMP Proxy failure fix/hack In-Reply-To: <50660897.6040008@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> <506604BC.6050503@brightok.net> <50660897.6040008@brightok.net> Message-ID: On 28.9.2012, at 23.29, Jack Bates wrote: > On 9/28/2012 3:12 PM, Jack Bates wrote: >> >> Code needs to be written to handle the special case of us not having any proxy callbacks as they are all bad. >> > > > Timo, please check and approve. This was diff'd on 2.1.10 on my test server (2.1.9 and 2.1.10 at least had this callback issue). > > *** lmtp-proxy.c-orig 2012-09-28 20:17:36.138916678 +0000 > --- lmtp-proxy.c 2012-09-28 20:18:12.241940780 +0000 > *************** > *** 300,303 **** > --- 300,304 ---- > lmtp_client_send(conn->client, conn->data_input); > lmtp_client_send_more(conn->client); > } > + lmtp_proxy_try_finish(proxy); > } Looks ok. Added: http://hg.dovecot.org/dovecot-2.1/rev/38727d3e90ec From tss at iki.fi Tue Oct 2 21:41:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:41:12 +0300 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924184157.GA75341@kyoko.org> References: <20120924134810.GA62723@kyoko.org> <20120924150440.GA85969@kyoko.org> <20120924184157.GA75341@kyoko.org> Message-ID: On 24.9.2012, at 21.41, Philippe Chevalier wrote: > As for the ldap message, it errors if there's no domain in the login. > > In the doc, it says that %d is empty if there's no domain part. So I > guess it's an enhancement request : a configuration option to have it > filled out with a default domain if there's no one supplied by the > client. Maybe this is enough? auth_bind_userdn = dc=%Du,ou=Domains,ou=Mail,dc=dspnet,dc=fr See %D in http://wiki2.dovecot.org/Variables From listas at adminlinux.com.br Tue Oct 2 21:45:39 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Tue, 02 Oct 2012 15:45:39 -0300 Subject: [Dovecot] About ManageSieve and TLS Message-ID: <506B3653.5020804@adminlinux.com.br> Hi, I have a "ubuntu10.04 + dovecot-2.0.13" configuration in my server. It works fine with ~50k accounts. Recently I enabled TLS: $ cat /etc/dovecot/dovecot.conf ... # Use SSL ? ssl = yes ... The goal was to provide only IMAPS and POP3S. But Managesieve says "STARTTLS": $ telnet _MY_IP_ 2000 Trying _MY_IP_... Connected to _MY_IP_. Escape character is '^]'. "IMPLEMENTATION" "K8 ManageSieve" "SIEVE" "comparator-i;ascii-numeric copy envelope fileinto imapflags include notify regex reject relational subaddress vacation" "SASL" "PLAIN LOGIN" "STARTTLS" "VERSION" "1.0" OK "K8 IMAP/POP3 server" doveconf -a shows: service managesieve-login { ... inet_listener sieve { address = _MY_IP_ port = 4190 ssl = no } inet_listener sieve_deprecated { address = _MY_IP_ port = 2000 ssl = no } ... } I think there is something wrong there but I don't know. I think Managesieve should not say "STARTTLS". Can someone help me? Thanks. -- Thiago Henrique adminlinux.com.br From tss at iki.fi Tue Oct 2 21:56:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:56:33 +0300 Subject: [Dovecot] Spurious " Renaming not supported across conflicting directory permissions" In-Reply-To: <505DA946.90409@yahoo.com> References: <505DA946.90409@yahoo.com> Message-ID: On 22.9.2012, at 15.04, tlhackque wrote: > Dovecot 2.1.10 > Client = Thunderbird. Local disks. mbox format. > > Attempted to rename a folder, failed with: > > CANNOT Renaming not supported across conflicting directory permissions Fixed: http://hg.dovecot.org/dovecot-2.1/rev/83695d6d41aa From tss at iki.fi Tue Oct 2 22:28:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:28:22 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C393.5050209@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: On 19.9.2012, at 16.07, Alessio Cecchi wrote: > #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( > backend=0x1944240) at acl-backend-vfile-acllist.c:297 This backtrace is rather weird. Could you also do (instead of bt full): fr 1 p *ns p *ns.user p *auser It crashes because auser->dict = NULL, but it should never be NULL. From stephan at rename-it.nl Tue Oct 2 22:31:05 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 02 Oct 2012 21:31:05 +0200 Subject: [Dovecot] About ManageSieve and TLS In-Reply-To: <506B3653.5020804@adminlinux.com.br> References: <506B3653.5020804@adminlinux.com.br> Message-ID: <506B40F9.1000905@rename-it.nl> On 10/2/2012 8:45 PM, 3.listas at adminlinux.com.br wrote: > Hi, > > I have a "ubuntu10.04 + dovecot-2.0.13" configuration in my server. It > works fine with ~50k accounts. > > Recently I enabled TLS: > $ cat /etc/dovecot/dovecot.conf > ... > # Use SSL ? > ssl = yes > ... > > The goal was to provide only IMAPS and POP3S. But Managesieve says > "STARTTLS": > $ telnet _MY_IP_ 2000 > Trying _MY_IP_... > Connected to _MY_IP_. > Escape character is '^]'. > "IMPLEMENTATION" "K8 ManageSieve" > "SIEVE" "comparator-i;ascii-numeric copy envelope fileinto imapflags > include notify regex reject relational subaddress vacation" > "SASL" "PLAIN LOGIN" > "STARTTLS" > "VERSION" "1.0" > OK "K8 IMAP/POP3 server" > > I think there is something wrong there but I don't know. I think > Managesieve should not say "STARTTLS". > Can someone help me? The STARTTLS capability means that ManageSieve is prepared to accept a STARTTLS command that invokes the TLS handshake. Basically, the protocol starts in plaintext and switches to a TLS/SSL secured channel once the STARTTLS command is issued. However, the client can also choose not to use it. Therefore, it really shouldn't influence whether ManageSieve works properly (unless the client messes up TLS somehow). If you really want to, you can disable TLS for ManageSieve specifically by putting a ssl=no inside the protocol sieve {} section. Regards, Stephan. From tss at iki.fi Tue Oct 2 22:38:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:38:58 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <5062DF3C.3050601@um.es> References: <5062DF3C.3050601@um.es> Message-ID: On 26.9.2012, at 13.55, Angel L. Mateo wrote: > doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do > doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; > done -S auth-userdb? You've named it completely wrong if that works :) > The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: > > e62e0d3834ed094e5c7900007efb8a67 66 > e62e0d3834ed094e5c7900007efb8a67 71 > e62e0d3834ed094e5c7900007efb8a67 74 > e62e0d3834ed094e5c7900007efb8a67 > 75 e62e0d3834ed094e5c7900007efb8a67 > 77 e62e0d3834ed094e5c7900007efb8a67 > 78 e62e0d3834ed094e5c7900007efb8a67 Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/94c7e875f9b9 From tss at iki.fi Tue Oct 2 22:50:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:50:08 +0300 Subject: [Dovecot] Problem with process_limit In-Reply-To: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: On 1.10.2012, at 12.15, FABIO FERRARI wrote: > Occasionally, it happens that the dovecot.log shows this line: > master: Warning: service(imap): process_limit reached, client connections > are being dropped .. > Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line > process_limit = 1500 But did you set it inside service imap {}? All of the services have process_limit parameter. From tss at iki.fi Tue Oct 2 23:12:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:12:51 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <5060AE92.5040904@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> Message-ID: On 24.9.2012, at 22.03, Joe Auty wrote: > #2 0x00007ff30074633d in master_service_settings_cache_deinit (_cache=) > at master-service-settings-cache.c:86 Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 From tss at iki.fi Tue Oct 2 23:15:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:15:22 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <83B37619-1CE8-4C5D-8147-A3C0E1C99CDC@iki.fi> On 2.10.2012, at 22.28, Timo Sirainen wrote: > On 19.9.2012, at 16.07, Alessio Cecchi wrote: > >> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > > This backtrace is rather weird. Could you also do (instead of bt full): Also, can you reproduce the crash always by running "doveadm quota recalc -u user at domain"? From tss at iki.fi Tue Oct 2 23:18:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:18:48 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <506AA8CF.8090605@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> Message-ID: <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> On 2.10.2012, at 11.41, Angel L. Mateo wrote: > I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). You could flush the whole cache also. > I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: Flush both the user and user at domain entries? From florob at babelmonkeys.de Wed Oct 3 00:05:56 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 02 Oct 2012 23:05:56 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme Message-ID: <506B5734.30906@babelmonkeys.de> Hello, attached is an hg export on top of the current dovecot-2.2 branch, which adds support for a SCRAM-SHA-1 password scheme. Ideally I'd want doveadm pw's rounds flag to apply to this, but that's currently specific to the crypt password scheme, so I left it out for now. Regards, Florian Zeitz -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1348017219 -7200 # Node ID 21a0d1b4daa7bb924f1666f0bb7c7e697a19c950 # Parent 8802322d72573ee17c52ce5e972e77e6f8ad69d1 auth: Add and use SCRAM-SHA-1 password scheme diff --git a/src/auth/Makefile.am b/src/auth/Makefile.am --- a/src/auth/Makefile.am +++ b/src/auth/Makefile.am @@ -44,6 +44,7 @@ password-scheme.c \ password-scheme-crypt.c \ password-scheme-md5crypt.c \ + password-scheme-scram.c \ password-scheme-otp.c \ password-scheme-rpa.c diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -1,11 +1,13 @@ /* * SCRAM-SHA-1 SASL authentication, see RFC-5802 * - * Copyright (c) 2011 Florian Zeitz + * Copyright (c) 2011-2012 Florian Zeitz * * This software is released under the MIT license. */ +#include + #include "auth-common.h" #include "base64.h" #include "buffer.h" @@ -29,45 +31,22 @@ /* sent: */ const char *server_first_message; - unsigned char salt[16]; - unsigned char salted_password[SHA1_RESULTLEN]; + const char *snonce; /* received: */ const char *gs2_cbind_flag; const char *cnonce; - const char *snonce; const char *client_first_message_bare; const char *client_final_message_without_proof; buffer_t *proof; + + /* stored */ + buffer_t *stored_key; + buffer_t *server_key; }; -static void Hi(const unsigned char *str, size_t str_size, - const unsigned char *salt, size_t salt_size, unsigned int i, - unsigned char result[SHA1_RESULTLEN]) -{ - struct hmac_context ctx; - unsigned char U[SHA1_RESULTLEN]; - unsigned int j, k; - - /* Calculate U1 */ - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, salt, salt_size); - hmac_update(&ctx, "\0\0\0\1", 4); - hmac_final(&ctx, U); - - memcpy(result, U, SHA1_RESULTLEN); - - /* Calculate U2 to Ui and Hi */ - for (j = 2; j <= i; j++) { - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, U, sizeof(U)); - hmac_final(&ctx, U); - for (k = 0; k < SHA1_RESULTLEN; k++) - result[k] ^= U[k]; - } -} - -static const char *get_scram_server_first(struct scram_auth_request *request) +static const char *get_scram_server_first(struct scram_auth_request *request, + int iter, const char *salt) { unsigned char snonce[SCRAM_SERVER_NONCE_LEN+1]; string_t *str; @@ -84,12 +63,9 @@ snonce[sizeof(snonce)-1] = '\0'; request->snonce = p_strndup(request->pool, snonce, sizeof(snonce)); - random_fill(request->salt, sizeof(request->salt)); - - str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(request->salt))); - str_printfa(str, "r=%s%s,s=", request->cnonce, request->snonce); - base64_encode(request->salt, sizeof(request->salt), str); - str_printfa(str, ",i=%d", SCRAM_ITERATE_COUNT); + str = t_str_new(sizeof(snonce)); + str_printfa(str, "r=%s%s,s=%s,i=%d", request->cnonce, request->snonce, + salt, iter); return str_c(str); } @@ -105,15 +81,8 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Server Key", 10); - hmac_final(&ctx, server_key); - - safe_memset(request->salted_password, 0, - sizeof(request->salted_password)); - - hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_init(&ctx, request->server_key->data, request->server_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, server_signature); @@ -211,8 +180,7 @@ return TRUE; } -static bool verify_credentials(struct scram_auth_request *request, - const unsigned char *credentials, size_t size) +static bool verify_credentials(struct scram_auth_request *request) { struct hmac_context ctx; const char *auth_message; @@ -221,54 +189,76 @@ unsigned char stored_key[SHA1_RESULTLEN]; size_t i; - /* FIXME: credentials should be SASLprepped UTF8 data here */ - Hi(credentials, size, request->salt, sizeof(request->salt), - SCRAM_ITERATE_COUNT, request->salted_password); - - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Client Key", 10); - hmac_final(&ctx, client_key); - - sha1_get_digest(client_key, sizeof(client_key), stored_key); - auth_message = t_strconcat(request->client_first_message_bare, ",", request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_init(&ctx, request->stored_key->data, request->stored_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) - client_signature[i] ^= client_key[i]; + client_key[i] = + ((char*)request->proof->data)[i] ^ client_signature[i]; + + sha1_get_digest(client_key, sizeof(client_key), stored_key); safe_memset(client_key, 0, sizeof(client_key)); - safe_memset(stored_key, 0, sizeof(stored_key)); + safe_memset(client_signature, 0, sizeof(client_signature)); - return memcmp(client_signature, request->proof->data, - request->proof->used) == 0; + return memcmp(stored_key, request->stored_key->data, + request->stored_key->used) == 0; } static void credentials_callback(enum passdb_result result, const unsigned char *credentials, size_t size, struct auth_request *auth_request) { + const char *const *fields; + size_t len; + int iter; + const char *salt; struct scram_auth_request *request = (struct scram_auth_request *)auth_request; - const char *server_final_message; switch (result) { case PASSDB_RESULT_OK: - if (!verify_credentials(request, credentials, size)) { + fields = t_strsplit(t_strndup(credentials, size), ","); + + iter = atoi(fields[0]); + salt = fields[1]; + + len = strlen(fields[2]); + request->stored_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[2], len, NULL, + request->stored_key) < 0) { auth_request_log_info(auth_request, "scram-sha-1", - "password mismatch"); + "Invalid base64 encoding" + "of StoredKey in passdb"); auth_request_fail(auth_request); - } else { - server_final_message = get_scram_server_final(request); - auth_request_success(auth_request, server_final_message, - strlen(server_final_message)); + break; } + + len = strlen(fields[3]); + request->server_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[3], len, NULL, + request->server_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of ServerKey in passdb"); + auth_request_fail(auth_request); + break; + } + + request->server_first_message = p_strdup(request->pool, + get_scram_server_first(request, iter, salt)); + + auth_request_handler_reply_continue(auth_request, + request->server_first_message, + strlen(request->server_first_message)); break; case PASSDB_RESULT_INTERNAL_FAILURE: auth_request_internal_failure(auth_request); @@ -333,8 +323,6 @@ request->client_final_message_without_proof = p_strdup(request->pool, t_strarray_join(fields, ",")); - auth_request_lookup_credentials(&request->auth_request, "PLAIN", - credentials_callback); return TRUE; } @@ -345,22 +333,35 @@ struct scram_auth_request *request = (struct scram_auth_request *)auth_request; const char *error = NULL; + const char *server_final_message; + int len; if (!request->client_first_message_bare) { /* Received client-first-message */ if (parse_scram_client_first(request, data, data_size, &error)) { - request->server_first_message = p_strdup(request->pool, - get_scram_server_first(request)); - auth_request_handler_reply_continue(auth_request, - request->server_first_message, - strlen(request->server_first_message)); + auth_request_lookup_credentials(&request->auth_request, + "SCRAM-SHA1", + credentials_callback); return; } } else { /* Received client-final-message */ - if (parse_scram_client_final(request, data, data_size, &error)) - return; + if (parse_scram_client_final(request, data, data_size, + &error)) { + if (!verify_credentials(request)) { + auth_request_log_info(auth_request, + "scram-sha-1", + "password mismatch"); + } else { + server_final_message = + get_scram_server_final(request); + len = strlen(server_final_message); + auth_request_success(auth_request, + server_final_message, len); + return; + } + } } if (error != NULL) diff --git a/src/auth/password-scheme-scram.c b/src/auth/password-scheme-scram.c new file mode 100644 --- /dev/null +++ b/src/auth/password-scheme-scram.c @@ -0,0 +1,139 @@ +/* + * SCRAM-SHA-1 SASL authentication, see RFC-5802 + * + * Copyright (c) 2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include + +#include "lib.h" +#include "safe-memset.h" +#include "base64.h" +#include "buffer.h" +#include "hmac.h" +#include "randgen.h" +#include "sha1.h" +#include "str.h" +#include "password-scheme.h" + +/* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ +#define SCRAM_ITERATE_COUNT 4096 + +static void Hi(const unsigned char *str, size_t str_size, + const unsigned char *salt, size_t salt_size, unsigned int i, + unsigned char result[SHA1_RESULTLEN]) +{ + struct hmac_context ctx; + unsigned char U[SHA1_RESULTLEN]; + unsigned int j, k; + + /* Calculate U1 */ + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); + + memcpy(result, U, SHA1_RESULTLEN); + + /* Calculate U2 to Ui and Hi */ + for (j = 2; j <= i; j++) { + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); + for (k = 0; k < SHA1_RESULTLEN; k++) + result[k] ^= U[k]; + } +} + +/* password string format: iter,salt,stored_key,server_key */ + +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED) +{ + struct hmac_context ctx; + string_t *str; + const char *const *fields; + int iter; + const unsigned char *salt; + size_t salt_len; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + fields = t_strsplit(t_strndup(raw_password, size), ","); + iter = atoi(fields[0]); + salt = buffer_get_data(t_base64_decode_str(fields[1]), &salt_len); + str = t_str_new(strlen(fields[2])); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, salt_len, + iter, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + base64_encode(stored_key, sizeof(stored_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + return strcmp(fields[2], str_c(str)) == 0 ? 1 : 0; +} + +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r) +{ + string_t *str; + struct hmac_context ctx; + unsigned char salt[16]; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char server_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + random_fill(salt, sizeof(salt)); + + str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(salt))); + str_printfa(str, "%i,", SCRAM_ITERATE_COUNT); + base64_encode(salt, sizeof(salt), str); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, + sizeof(salt), SCRAM_ITERATE_COUNT, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + str_append_c(str, ','); + base64_encode(stored_key, sizeof(stored_key), str); + + /* Calculate ServerKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); + str_append_c(str, ','); + base64_encode(server_key, sizeof(server_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(server_key, 0, sizeof(server_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + *raw_password_r = (const unsigned char *)str_c(str); + *size_r = str_len(str); +} diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -822,6 +822,8 @@ { "PLAIN-TRUNC", PW_ENCODING_NONE, 0, plain_trunc_verify, plain_generate }, { "CRAM-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, + { "SCRAM-SHA1", PW_ENCODING_NONE, 0, scram_sha1_verify, + scram_sha1_generate}, { "HMAC-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, { "DIGEST-MD5", PW_ENCODING_HEX, MD5_RESULTLEN, diff --git a/src/auth/password-scheme.h b/src/auth/password-scheme.h --- a/src/auth/password-scheme.h +++ b/src/auth/password-scheme.h @@ -85,6 +85,12 @@ const unsigned char *raw_password, size_t size, const char **error_r); +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED); +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r); + /* check wich of the algorithms Blowfisch, SHA-256 and SHA-512 are supported by the used libc's/glibc's crypt() */ void password_scheme_register_crypt(void); From joe at netmusician.org Wed Oct 3 00:09:19 2012 From: joe at netmusician.org (Joe Auty) Date: Tue, 02 Oct 2012 17:09:19 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> Message-ID: <506B57FF.40809@netmusician.org> > > Timo Sirainen > October 2, 2012 4:12 PM > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 Awesome! Will this fix make it into 2.1.11, or should I toy with incorporating your change and compiling a new copy of 2.1.10 by hand? Also, will the seg fault have caused performance issues in breaking IDLE connections? Just wondering what sort of impact this fix might have... Thanks for working on this, it is most appreciated, I'm a big fan of Dovecot! > > > Joe Auty > September 24, 2012 3:03 PM >> >> Timo Sirainen >> September 24, 2012 10:32 AM >> >> Well, the good news is that it crashes only after it has already >> disconnected the client anyway. But I thought I fixed this bug in >> v2.1.10 and I'm not able to reproduce it myself.. Having debugging >> information available might show something useful. Try installing >> dovecot-dbg package and getting the bt full again? > > Thanks Timo, I have done so. Here is the results of my debugging info > now: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...Reading symbols > from /usr/lib/debug/usr/lib/dovecot/imap-login...done. > (no debugging symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...Reading > symbols from > /usr/lib/debug/usr/lib/dovecot/libdovecot-login.so.0.0.0...done. > (no debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...Reading > symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot.so.0.0.0...done. > (no debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 hash_table_destroy (_table=0x28) at hash.c:106 > 106 hash.c: No such file or directory. > in hash.c > (gdb) bt full > #0 hash_table_destroy (_table=0x28) at hash.c:106 > table = > #1 0x00007ff300721054 in settings_parser_deinit (_ctx= optimized out>) at settings-parser.c:237 > ctx = 0x0 > #2 0x00007ff30074633d in master_service_settings_cache_deinit > (_cache=) > at master-service-settings-cache.c:86 > cache = 0x9f9a60 > entry = 0xa016e0 > next = 0x0 > __FUNCTION__ = "master_service_settings_cache_deinit" > #3 0x00007ff3009a5018 in main_deinit (binary=, > argc=2, argv=0x9f8370) at main.c:355 > No locals. > #4 login_binary_run (binary=, argc=2, > argv=0x9f8370) at main.c:407 > set_pool = 0x9f8a30 > allow_core_dumps = > login_socket = > c = > #5 0x00007ff3003c0c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #6 0x0000000000402459 in _start () > No symbol table info available. > (gdb) > > >> >> >> Joe Auty >> September 23, 2012 7:05 AM >>> >>> Timo Sirainen >>> September 23, 2012 5:58 AM >>> >>> >>> You should have a similar log line about the crash in mail.log (or >>> wherever "doveadm log find" says that errors get logged). Find those >>> lines, then configure login processes to dump core files. This >>> probably should work: >>> >>> service imap-login { >>> executable = imap-login -D >>> } >>> >>> Next time it crashes hopefully you'll have >>> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >>> send it: >>> >>> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >>> bt full >> >> I hope I'm doing this correctly! >> >> # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> GNU gdb (GDB) 7.0.1-debian >> Copyright (C) 2009 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later >> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show >> copying" >> and "show warranty" for details. >> This GDB was configured as "x86_64-linux-gnu". >> For bug reporting instructions, please see: >> ... >> Reading symbols from /usr/lib/dovecot/imap-login...(no debugging >> symbols found)...done. >> >> warning: Can't read pathname for load map: Input/output error. >> Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no >> debugging symbols found)...done. >> Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 >> Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging >> symbols found)...done. >> Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 >> Reading symbols from /lib/libc.so.6...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libc.so.6 >> Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols >> found)...done. >> Loaded symbols for /usr/lib/libssl.so.0.9.8 >> Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging >> symbols found)...done. >> Loaded symbols for /usr/lib/libcrypto.so.0.9.8 >> Reading symbols from /lib/librt.so.1...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/librt.so.1 >> Reading symbols from /lib/libdl.so.2...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libdl.so.2 >> Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging >> symbols found)...done. >> Loaded symbols for /lib64/ld-linux-x86-64.so.2 >> Reading symbols from /usr/lib/libz.so.1...(no debugging symbols >> found)...done. >> Loaded symbols for /usr/lib/libz.so.1 >> Reading symbols from /lib/libpthread.so.0...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libpthread.so.0 >> Core was generated by `dovecot/imap-login ?'. >> Program terminated with signal 11, Segmentation fault. >> #0 0x00007f789cd08e14 in hash_table_destroy () from >> /usr/lib/dovecot/libdovecot.so.0 >> (gdb) bt full >> #0 0x00007f789cd08e14 in hash_table_destroy () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00007f789ccda054 in settings_parser_deinit () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () >> from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #3 0x00007f789cf5e018 in login_binary_run () from >> /usr/lib/dovecot/libdovecot-login.so.0 >> No symbol table info available. >> #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 >> No symbol table info available. >> #5 0x0000000000402459 in ?? () >> No symbol table info available. >> #6 0x00007fff8a9c65f8 in ?? () >> No symbol table info available. >> #7 0x000000000000001c in ?? () >> No symbol table info available. >> #8 0x0000000000000002 in ?? () >> No symbol table info available. >> #9 0x00007fff8a9c7e6a in ?? () >> No symbol table info available. >> #10 0x00007fff8a9c7e7d in ?? () >> No symbol table info available. >> #11 0x0000000000000000 in ?? () >> No symbol table info available. >> >> >>> >>> >>> Joe Auty >>> September 22, 2012 3:57 PM >>> Hello, >>> >>> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >>> I suspect this might be causing performance issues. Any suggestions >>> what I can try to fix this? >>> >>> >>> I'm using the 2.1.10 packages obtained with the following in my >>> sources.list: >>> >>> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >>> >>> >>> I need to use Dovecot 2.1.x because I need to support handling >>> multiple SSL certs. >>> ------------------------------------------------------------------------ >>> >> >> >> >> >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full > > ------------------------------------------------------------------------ From tss at iki.fi Wed Oct 3 00:13:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:13:10 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <506B57FF.40809@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> <506B57FF.40809@netmusician.org> Message-ID: <9AC6F07E-8751-47C0-AF6C-82722D1BAE5D@iki.fi> On 3.10.2012, at 0.09, Joe Auty wrote: >> Timo Sirainen >> October 2, 2012 4:12 PM >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 > > Awesome! > > Will this fix make it into 2.1.11, or should I toy with incorporating your change and compiling a new copy of 2.1.10 by hand? All changes added to dovecot-2.1 hg go to the next 2.1 release. But I don't know when v2.1.11 will be released, probably a few weeks at least. > Also, will the seg fault have caused performance issues in breaking IDLE connections? Just wondering what sort of impact this fix might have... It shouldn't have caused any user-visible problems. From tss at iki.fi Wed Oct 3 00:27:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:27:13 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B5734.30906@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> Message-ID: On 3.10.2012, at 0.05, Florian Zeitz wrote: > attached is an hg export on top of the current dovecot-2.2 branch, which > adds support for a SCRAM-SHA-1 password scheme. > > Ideally I'd want doveadm pw's rounds flag to apply to this, but that's > currently specific to the crypt password scheme, so I left it out for now. Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. From cor at xs4all.nl Wed Oct 3 00:39:39 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 2 Oct 2012 23:39:39 +0200 Subject: [Dovecot] possible nfs issue Message-ID: Hi all, we've started receiving complaints from users that seemingly use more quota than they actually have. We noticed that these users have (in some cases many) .nfs files in their mailspool. Some of our admins checked their own dirs, and noticed them there as well. This could of course be unrelated to dovecot (kernel issue, netapp issue) but maybe somehow has an idea about if dovecot could cause this. This has been going on for at least a year, not really enough to notice before now. root at userimap1# find . -type f|grep -i .nfs ./cur/.nfs00000000003967ad003c0603 ./cur/.nfs000000000757b44b003be609 ./cur/.nfs00000000035e89bd003be60b ./cur/.nfs000000000796251c003be60c ./cur/.nfs000000000796251f003be60e ./cur/.nfs000000000262f9a1003be33a ./cur/.nfs00000000096513f3003be524 ./cur/.nfs0000000007962525003be60f ./cur/.nfs0000000003e7d8ab003be62b ./cur/.nfs00000000026f4fad003be50d ./cur/.nfs0000000000bdaeab003c0611 ./cur/.nfs0000000005da42c7003be525 ./cur/.nfs0000000003d74729003be526 ./cur/.nfs000000000229769e003be535 ./cur/.nfs000000000440969e003be516 With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. Anyone else with NFS mailspools seeing this? Cor From tss at iki.fi Wed Oct 3 00:45:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:45:35 +0300 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: On 3.10.2012, at 0.39, Cor Bosman wrote: > With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) From florob at babelmonkeys.de Wed Oct 3 01:12:29 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 00:12:29 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: References: <506B5734.30906@babelmonkeys.de> Message-ID: <506B66CD.6010302@babelmonkeys.de> Am 02.10.2012 23:27, schrieb Timo Sirainen: > On 3.10.2012, at 0.05, Florian Zeitz wrote: > >> attached is an hg export on top of the current dovecot-2.2 branch, which >> adds support for a SCRAM-SHA-1 password scheme. >> >> Ideally I'd want doveadm pw's rounds flag to apply to this, but that's >> currently specific to the crypt password scheme, so I left it out for now. > > Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. > I shall. For the iteration count the endless loop should be fixed by restricting the largest value to UINT_MAX-1, right? I'm not too fond of stopping people from wasting their CPU time on Hi calculation beyond this. I can try to guestimate a "sane" upper limit, but given time I have an icky feeling that it will end up being too low. Thoughts? From tss at iki.fi Wed Oct 3 01:31:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 01:31:51 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B66CD.6010302@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <506B66CD.6010302@babelmonkeys.de> Message-ID: On 3.10.2012, at 1.12, Florian Zeitz wrote: > Am 02.10.2012 23:27, schrieb Timo Sirainen: >> On 3.10.2012, at 0.05, Florian Zeitz wrote: >> >>> attached is an hg export on top of the current dovecot-2.2 branch, which >>> adds support for a SCRAM-SHA-1 password scheme. >>> >>> Ideally I'd want doveadm pw's rounds flag to apply to this, but that's >>> currently specific to the crypt password scheme, so I left it out for now. >> >> Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. >> > I shall. For the iteration count the endless loop should be fixed by > restricting the largest value to UINT_MAX-1, right? Yeah. > I'm not too fond of > stopping people from wasting their CPU time on Hi calculation beyond > this. I can try to guestimate a "sane" upper limit, but given time I > have an icky feeling that it will end up being too low. Thoughts? Looks like RFC 5802 doesn't give any kind of a limit. But since it gets sent to various client implementations, INT_MAX is probably a good limit? Also 0 isn't a valid iteration count. From tss at iki.fi Wed Oct 3 01:35:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 01:35:20 +0300 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> On 3.10.2012, at 0.45, Timo Sirainen wrote: > On 3.10.2012, at 0.39, Cor Bosman wrote: > >> With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. > > How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) You could see if this old test program leaves .nfs files lying around: http://dovecot.org/tmp/readdir.c Just comment out the line: close(fd); From jtam.home at gmail.com Wed Oct 3 01:42:23 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 2 Oct 2012 15:42:23 -0700 (PDT) Subject: [Dovecot] Logging IP address for failed login In-Reply-To: References: Message-ID: Scott Neville writes: > I am trying to use the logs to show the IP that brute force activity > comes from, but Im not succeeding. I have read the archives and seen > the advice others have had. I can see logs for repeated bad logins, > but I need the IP address from the attempts. > > ... > but only for successful logins. The brute force attempts dont log like that: > > Sep 16 00:02:58 olive dovecot: auth: pam(backup): unknown user This was similar to another complaint several months ago. I conjectured that these attempts are SMTP AUTH, not IMAP, brute forcing. Are you using the dovecot's SASL feature to authenticate outgoing Email (i.e. via Postfix?). Maybe you verify this hypothesis by checking the Postfix logs. Joseph Tam From cor at xs4all.nl Wed Oct 3 01:53:15 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 3 Oct 2012 00:53:15 +0200 Subject: [Dovecot] possible nfs issue In-Reply-To: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> References: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> Message-ID: On Oct 3, 2012, at 12:35 AM, Timo Sirainen wrote: > On 3.10.2012, at 0.45, Timo Sirainen wrote: > >> On 3.10.2012, at 0.39, Cor Bosman wrote: >> >>> With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. >> >> How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) > > You could see if this old test program leaves .nfs files lying around: > > http://dovecot.org/tmp/readdir.c > > Just comment out the line: > > close(fd); > I meant the .nfs1234 indeed, but it seemed very unlikely. Thanks for clarifying. The readdir program leaves no .nfs files. We'll have to explore other possibilities. Cor From tss at iki.fi Wed Oct 3 02:42:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 02:42:17 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B5734.30906@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> Message-ID: <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> On 3.10.2012, at 0.05, Florian Zeitz wrote: > attached is an hg export on top of the current dovecot-2.2 branch, which > adds support for a SCRAM-SHA-1 password scheme. Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. From florob at babelmonkeys.de Wed Oct 3 02:54:21 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 01:54:21 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> Message-ID: <506B7EAD.1080108@babelmonkeys.de> Am 03.10.2012 01:42, schrieb Timo Sirainen: > On 3.10.2012, at 0.05, Florian Zeitz wrote: > >> attached is an hg export on top of the current dovecot-2.2 branch, which >> adds support for a SCRAM-SHA-1 password scheme. > > Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. > Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the RFC, and SHA-1 is the hash name registered with IANA [1]. I did call the password scheme SCRAM-SHA1 to be consistent with other current password schemes. I'm not 100% sure which one to use, or whether a mix might even be the way to go ("correct" messages, but minimum user confusion for password schemes). [1] https://www.iana.org/assignments/hash-function-text-names/hash-function-text-names.xml From tss at iki.fi Wed Oct 3 02:58:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 02:58:25 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B7EAD.1080108@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> Message-ID: On 3.10.2012, at 2.54, Florian Zeitz wrote: > Am 03.10.2012 01:42, schrieb Timo Sirainen: >> On 3.10.2012, at 0.05, Florian Zeitz wrote: >> >>> attached is an hg export on top of the current dovecot-2.2 branch, which >>> adds support for a SCRAM-SHA-1 password scheme. >> >> Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. >> > Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the > RFC, and SHA-1 is the hash name registered with IANA [1]. > I did call the password scheme SCRAM-SHA1 to be consistent with other > current password schemes. I'm not 100% sure which one to use, or whether > a mix might even be the way to go ("correct" messages, but minimum user > confusion for password schemes). Hmm. Probably not worth it to have both SCRAM-SHA1 and SCRAM-SHA-1. And now I see that the user-visible strings are about SCRAM-SHA-1 mechanism, not the hash. So yeah, I guess the best way to avoid confusion is to call it SCRAM-SHA-1 everywhere. From florob at babelmonkeys.de Wed Oct 3 03:10:41 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 02:10:41 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> Message-ID: <506B8281.9060703@babelmonkeys.de> Am 03.10.2012 01:58, schrieb Timo Sirainen: > On 3.10.2012, at 2.54, Florian Zeitz wrote: > >> Am 03.10.2012 01:42, schrieb Timo Sirainen: >>> On 3.10.2012, at 0.05, Florian Zeitz wrote: >>> >>>> attached is an hg export on top of the current dovecot-2.2 branch, which >>>> adds support for a SCRAM-SHA-1 password scheme. >>> >>> Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. >>> >> Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the >> RFC, and SHA-1 is the hash name registered with IANA [1]. >> I did call the password scheme SCRAM-SHA1 to be consistent with other >> current password schemes. I'm not 100% sure which one to use, or whether >> a mix might even be the way to go ("correct" messages, but minimum user >> confusion for password schemes). > > Hmm. Probably not worth it to have both SCRAM-SHA1 and SCRAM-SHA-1. And now I see that the user-visible strings are about SCRAM-SHA-1 mechanism, not the hash. So yeah, I guess the best way to avoid confusion is to call it SCRAM-SHA-1 everywhere. > Seems sensible. Attached is a new export incorporating your feedback. The iteration count is now limited to [4096, INT_MAX]. The lower bound is a recommendation of the RFC. -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1348017219 -7200 # Node ID a0b0eece12335905500631477ec1d6ab31014469 # Parent 99843f74422ac68bfde86e9cee6920164eae4d5d auth: Add and use SCRAM-SHA-1 password scheme diff --git a/src/auth/Makefile.am b/src/auth/Makefile.am --- a/src/auth/Makefile.am +++ b/src/auth/Makefile.am @@ -44,6 +44,7 @@ password-scheme.c \ password-scheme-crypt.c \ password-scheme-md5crypt.c \ + password-scheme-scram.c \ password-scheme-otp.c \ password-scheme-rpa.c diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -1,11 +1,14 @@ /* * SCRAM-SHA-1 SASL authentication, see RFC-5802 * - * Copyright (c) 2011 Florian Zeitz + * Copyright (c) 2011-2012 Florian Zeitz * * This software is released under the MIT license. */ +#include +#include + #include "auth-common.h" #include "base64.h" #include "buffer.h" @@ -15,6 +18,7 @@ #include "safe-memset.h" #include "str.h" #include "strfuncs.h" +#include "strnum.h" #include "mech.h" /* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ @@ -29,45 +33,22 @@ /* sent: */ const char *server_first_message; - unsigned char salt[16]; - unsigned char salted_password[SHA1_RESULTLEN]; + const char *snonce; /* received: */ const char *gs2_cbind_flag; const char *cnonce; - const char *snonce; const char *client_first_message_bare; const char *client_final_message_without_proof; buffer_t *proof; + + /* stored */ + buffer_t *stored_key; + buffer_t *server_key; }; -static void Hi(const unsigned char *str, size_t str_size, - const unsigned char *salt, size_t salt_size, unsigned int i, - unsigned char result[SHA1_RESULTLEN]) -{ - struct hmac_context ctx; - unsigned char U[SHA1_RESULTLEN]; - unsigned int j, k; - - /* Calculate U1 */ - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, salt, salt_size); - hmac_update(&ctx, "\0\0\0\1", 4); - hmac_final(&ctx, U); - - memcpy(result, U, SHA1_RESULTLEN); - - /* Calculate U2 to Ui and Hi */ - for (j = 2; j <= i; j++) { - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, U, sizeof(U)); - hmac_final(&ctx, U); - for (k = 0; k < SHA1_RESULTLEN; k++) - result[k] ^= U[k]; - } -} - -static const char *get_scram_server_first(struct scram_auth_request *request) +static const char *get_scram_server_first(struct scram_auth_request *request, + int iter, const char *salt) { unsigned char snonce[SCRAM_SERVER_NONCE_LEN+1]; string_t *str; @@ -84,12 +65,9 @@ snonce[sizeof(snonce)-1] = '\0'; request->snonce = p_strndup(request->pool, snonce, sizeof(snonce)); - random_fill(request->salt, sizeof(request->salt)); - - str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(request->salt))); - str_printfa(str, "r=%s%s,s=", request->cnonce, request->snonce); - base64_encode(request->salt, sizeof(request->salt), str); - str_printfa(str, ",i=%d", SCRAM_ITERATE_COUNT); + str = t_str_new(sizeof(snonce)); + str_printfa(str, "r=%s%s,s=%s,i=%d", request->cnonce, request->snonce, + salt, iter); return str_c(str); } @@ -105,15 +83,8 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Server Key", 10); - hmac_final(&ctx, server_key); - - safe_memset(request->salted_password, 0, - sizeof(request->salted_password)); - - hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_init(&ctx, request->server_key->data, request->server_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, server_signature); @@ -211,8 +182,7 @@ return TRUE; } -static bool verify_credentials(struct scram_auth_request *request, - const unsigned char *credentials, size_t size) +static bool verify_credentials(struct scram_auth_request *request) { struct hmac_context ctx; const char *auth_message; @@ -221,54 +191,90 @@ unsigned char stored_key[SHA1_RESULTLEN]; size_t i; - /* FIXME: credentials should be SASLprepped UTF8 data here */ - Hi(credentials, size, request->salt, sizeof(request->salt), - SCRAM_ITERATE_COUNT, request->salted_password); - - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Client Key", 10); - hmac_final(&ctx, client_key); - - sha1_get_digest(client_key, sizeof(client_key), stored_key); - auth_message = t_strconcat(request->client_first_message_bare, ",", request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_init(&ctx, request->stored_key->data, request->stored_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) - client_signature[i] ^= client_key[i]; + client_key[i] = + ((char*)request->proof->data)[i] ^ client_signature[i]; + + sha1_get_digest(client_key, sizeof(client_key), stored_key); safe_memset(client_key, 0, sizeof(client_key)); - safe_memset(stored_key, 0, sizeof(stored_key)); + safe_memset(client_signature, 0, sizeof(client_signature)); - return memcmp(client_signature, request->proof->data, - request->proof->used) == 0; + return memcmp(stored_key, request->stored_key->data, + request->stored_key->used) == 0; } static void credentials_callback(enum passdb_result result, const unsigned char *credentials, size_t size, struct auth_request *auth_request) { + const char *const *fields; + size_t len; + unsigned int iter; + const char *salt; struct scram_auth_request *request = (struct scram_auth_request *)auth_request; - const char *server_final_message; switch (result) { case PASSDB_RESULT_OK: - if (!verify_credentials(request, credentials, size)) { + fields = t_strsplit(t_strndup(credentials, size), ","); + + if (str_array_length(fields) != 4) { auth_request_log_info(auth_request, "scram-sha-1", - "password mismatch"); + "Invalid passdb entry"); auth_request_fail(auth_request); - } else { - server_final_message = get_scram_server_final(request); - auth_request_success(auth_request, server_final_message, - strlen(server_final_message)); + break; } + + if (str_to_uint(fields[0], &iter) || (iter < 4096) || + (iter > INT_MAX)) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid iteration count"); + auth_request_fail(auth_request); + break; + } + + salt = fields[1]; + + len = strlen(fields[2]); + request->stored_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[2], len, NULL, + request->stored_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of StoredKey in passdb"); + auth_request_fail(auth_request); + break; + } + + len = strlen(fields[3]); + request->server_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[3], len, NULL, + request->server_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of ServerKey in passdb"); + auth_request_fail(auth_request); + break; + } + + request->server_first_message = p_strdup(request->pool, + get_scram_server_first(request, iter, salt)); + + auth_request_handler_reply_continue(auth_request, + request->server_first_message, + strlen(request->server_first_message)); break; case PASSDB_RESULT_INTERNAL_FAILURE: auth_request_internal_failure(auth_request); @@ -333,8 +339,6 @@ request->client_final_message_without_proof = p_strdup(request->pool, t_strarray_join(fields, ",")); - auth_request_lookup_credentials(&request->auth_request, "PLAIN", - credentials_callback); return TRUE; } @@ -345,22 +349,35 @@ struct scram_auth_request *request = (struct scram_auth_request *)auth_request; const char *error = NULL; + const char *server_final_message; + int len; if (!request->client_first_message_bare) { /* Received client-first-message */ if (parse_scram_client_first(request, data, data_size, &error)) { - request->server_first_message = p_strdup(request->pool, - get_scram_server_first(request)); - auth_request_handler_reply_continue(auth_request, - request->server_first_message, - strlen(request->server_first_message)); + auth_request_lookup_credentials(&request->auth_request, + "SCRAM-SHA-1", + credentials_callback); return; } } else { /* Received client-final-message */ - if (parse_scram_client_final(request, data, data_size, &error)) - return; + if (parse_scram_client_final(request, data, data_size, + &error)) { + if (!verify_credentials(request)) { + auth_request_log_info(auth_request, + "scram-sha-1", + "password mismatch"); + } else { + server_final_message = + get_scram_server_final(request); + len = strlen(server_final_message); + auth_request_success(auth_request, + server_final_message, len); + return; + } + } } if (error != NULL) diff --git a/src/auth/password-scheme-scram.c b/src/auth/password-scheme-scram.c new file mode 100644 --- /dev/null +++ b/src/auth/password-scheme-scram.c @@ -0,0 +1,139 @@ +/* + * SCRAM-SHA-1 SASL authentication, see RFC-5802 + * + * Copyright (c) 2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include + +#include "lib.h" +#include "safe-memset.h" +#include "base64.h" +#include "buffer.h" +#include "hmac.h" +#include "randgen.h" +#include "sha1.h" +#include "str.h" +#include "password-scheme.h" + +/* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ +#define SCRAM_ITERATE_COUNT 4096 + +static void Hi(const unsigned char *str, size_t str_size, + const unsigned char *salt, size_t salt_size, unsigned int i, + unsigned char result[SHA1_RESULTLEN]) +{ + struct hmac_context ctx; + unsigned char U[SHA1_RESULTLEN]; + unsigned int j, k; + + /* Calculate U1 */ + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); + + memcpy(result, U, SHA1_RESULTLEN); + + /* Calculate U2 to Ui and Hi */ + for (j = 2; j <= i; j++) { + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); + for (k = 0; k < SHA1_RESULTLEN; k++) + result[k] ^= U[k]; + } +} + +/* password string format: iter,salt,stored_key,server_key */ + +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED) +{ + struct hmac_context ctx; + string_t *str; + const char *const *fields; + int iter; + const unsigned char *salt; + size_t salt_len; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + fields = t_strsplit(t_strndup(raw_password, size), ","); + iter = atoi(fields[0]); + salt = buffer_get_data(t_base64_decode_str(fields[1]), &salt_len); + str = t_str_new(strlen(fields[2])); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, salt_len, + iter, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + base64_encode(stored_key, sizeof(stored_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + return strcmp(fields[2], str_c(str)) == 0 ? 1 : 0; +} + +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r) +{ + string_t *str; + struct hmac_context ctx; + unsigned char salt[16]; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char server_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + random_fill(salt, sizeof(salt)); + + str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(salt))); + str_printfa(str, "%i,", SCRAM_ITERATE_COUNT); + base64_encode(salt, sizeof(salt), str); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, + sizeof(salt), SCRAM_ITERATE_COUNT, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + str_append_c(str, ','); + base64_encode(stored_key, sizeof(stored_key), str); + + /* Calculate ServerKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); + str_append_c(str, ','); + base64_encode(server_key, sizeof(server_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(server_key, 0, sizeof(server_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + *raw_password_r = (const unsigned char *)str_c(str); + *size_r = str_len(str); +} diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -822,6 +822,8 @@ { "PLAIN-TRUNC", PW_ENCODING_NONE, 0, plain_trunc_verify, plain_generate }, { "CRAM-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, + { "SCRAM-SHA-1", PW_ENCODING_NONE, 0, scram_sha1_verify, + scram_sha1_generate}, { "HMAC-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, { "DIGEST-MD5", PW_ENCODING_HEX, MD5_RESULTLEN, diff --git a/src/auth/password-scheme.h b/src/auth/password-scheme.h --- a/src/auth/password-scheme.h +++ b/src/auth/password-scheme.h @@ -85,6 +85,12 @@ const unsigned char *raw_password, size_t size, const char **error_r); +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED); +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r); + /* check wich of the algorithms Blowfisch, SHA-256 and SHA-512 are supported by the used libc's/glibc's crypt() */ void password_scheme_register_crypt(void); From tss at iki.fi Wed Oct 3 03:50:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 03 Oct 2012 03:50:00 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B8281.9060703@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> <506B8281.9060703@babelmonkeys.de> Message-ID: <1349225400.18782.54.camel@innu> On Wed, 2012-10-03 at 02:10 +0200, Florian Zeitz wrote: > Attached is a new export incorporating your feedback. Committed. Also what do you think about the attached patch? (Compiles, untested.) -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 9633 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 3 05:23:57 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 02 Oct 2012 21:23:57 -0500 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <506BA1BD.1070903@brightok.net> On 10/2/2012 4:39 PM, Cor Bosman wrote: > > Anyone else with NFS mailspools seeing this? > > Cor > > I haven't seen them yet, however, to help troubleshoot, see this link and follow it's links for more details on .nfs files http://wordpress.org/support/topic/how-can-i-prevent-unwanted-nfs-files-from-being-created Jack From amateo at um.es Wed Oct 3 09:20:00 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 03 Oct 2012 08:20:00 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: References: <5062DF3C.3050601@um.es> Message-ID: <506BD910.90200@um.es> El 02/10/12 21:38, Timo Sirainen escribi?: > On 26.9.2012, at 13.55, Angel L. Mateo wrote: > >> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >> done > > -S auth-userdb? You've named it completely wrong if that works :) > auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? >> The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: >> >> e62e0d3834ed094e5c7900007efb8a67 66 >> e62e0d3834ed094e5c7900007efb8a67 71 >> e62e0d3834ed094e5c7900007efb8a67 74 >> e62e0d3834ed094e5c7900007efb8a67 >> 75 e62e0d3834ed094e5c7900007efb8a67 >> 77 e62e0d3834ed094e5c7900007efb8a67 >> 78 e62e0d3834ed094e5c7900007efb8a67 > > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/94c7e875f9b9 > Thanks, I'll check as soon as I can. From amateo at um.es Wed Oct 3 09:25:38 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 03 Oct 2012 08:25:38 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> Message-ID: <506BDA62.2010604@um.es> El 02/10/12 22:18, Timo Sirainen escribi?: > On 2.10.2012, at 11.41, Angel L. Mateo wrote: > >> I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). > > You could flush the whole cache also. > Oh... I was so obfuscated trying to expire just the user that I forgot I could flush the whole cache :-( >> I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: > > Flush both the user and user at domain entries? > Yes, I could do this, but why there are entries with user and user at domain?, because I have three user databases: * master password: it is not normally used * pam: I have the cache_key=%n on it * ldap: I don't know to configure cache_key (I tried args = cache_key=%n /etc/dovecot/dovecot-ldap.conf.ext but it didn't work) From tss at iki.fi Wed Oct 3 16:26:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 16:26:34 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <506BDA62.2010604@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> <506BDA62.2010604@um.es> Message-ID: <5B43FF27-C875-48D7-91DA-FA86848B02B9@iki.fi> On 3.10.2012, at 9.25, Angel L. Mateo wrote: >>> I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: >> >> Flush both the user and user at domain entries? >> > Yes, I could do this, but why there are entries with user and user at domain?, because I have three user databases: > > * master password: it is not normally used > * pam: I have the cache_key=%n on it > * ldap: I don't know to configure cache_key (I tried args = cache_key=%n /etc/dovecot/dovecot-ldap.conf.ext but it didn't work) For LDAP the cache_key is figured out automatically based on the used %variables. You can't override the cache key. The only way to make it work would be to change the LDAP query to use only %n and no %u/%d (which I guess would be possible by checking for %n@* ?) From tss at iki.fi Wed Oct 3 16:27:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 16:27:32 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <506BD910.90200@um.es> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> Message-ID: <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> On 3.10.2012, at 9.20, Angel L. Mateo wrote: > El 02/10/12 21:38, Timo Sirainen escribi?: >> On 26.9.2012, at 13.55, Angel L. Mateo wrote: >> >>> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >>> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >>> done >> >> -S auth-userdb? You've named it completely wrong if that works :) >> > auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? From florob at babelmonkeys.de Wed Oct 3 16:33:32 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 15:33:32 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <1349225400.18782.54.camel@innu> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> <506B8281.9060703@babelmonkeys.de> <1349225400.18782.54.camel@innu> Message-ID: <506C3EAC.6000504@babelmonkeys.de> Am 03.10.2012 02:50, schrieb Timo Sirainen: > On Wed, 2012-10-03 at 02:10 +0200, Florian Zeitz wrote: >> Attached is a new export incorporating your feedback. > > Committed. Also what do you think about the attached patch? (Compiles, > untested.) > Moving the passdb parsing into a separate function seems like a nice idea to me. Style changes and removing an unused variable is obviously fine (I'm a bit surprised I got no compiler warning about the latter, but oh well). I did a quick test. Login and error checking seem to still work fine with this patch in place. Wouldn't have seen anything in the code to suggest otherwise either. From list at airstreamcomm.net Wed Oct 3 17:34:51 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 03 Oct 2012 09:34:51 -0500 Subject: [Dovecot] LDA vs LMTP index files Message-ID: <506C4D0B.8030709@airstreamcomm.net> In the docs it states that LDA " ...takes mail from anMTAand delivers it to a user's mailbox, while keeping Dovecot index files up to date." I am wondering if LMTP also interacts with the Dovecot index files and keeps them up to date? From jbates at brightok.net Wed Oct 3 18:06:08 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 03 Oct 2012 10:06:08 -0500 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C4D0B.8030709@airstreamcomm.net> References: <506C4D0B.8030709@airstreamcomm.net> Message-ID: <506C5460.205@brightok.net> On 10/3/2012 9:34 AM, list at airstreamcomm.net wrote: > In the docs it states that LDA " ...takes mail from anMTAand delivers > it to a user's mailbox, while keeping Dovecot index files up to > date." I am wondering if LMTP also interacts with the Dovecot index > files and keeps them up to date? > Brand new account created with LMTP. So I'd say yes. -rw-------. 1 vmail vmail 16384 Oct 2 20:21 dovecot.index.cache -rw-------. 1 vmail vmail 560 Oct 2 20:21 dovecot.index.log -rw-------. 1 vmail vmail 152 Oct 2 20:21 dovecot-uidlist -rw-------. 1 vmail vmail 8 Oct 2 18:58 dovecot-uidvalidity -r--r--r--. 1 vmail vmail 0 Oct 2 18:58 dovecot-uidvalidity.506b393c -rw-------. 1 vmail vmail 28 Oct 2 20:21 maildirsize Jack From CMarcus at Media-Brokers.com Wed Oct 3 18:03:18 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 03 Oct 2012 11:03:18 -0400 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C4D0B.8030709@airstreamcomm.net> References: <506C4D0B.8030709@airstreamcomm.net> Message-ID: <506C53B6.6070706@Media-Brokers.com> On 2012-10-03 10:34 AM, list at airstreamcomm.net wrote: > In the docs it states that LDA " ...takes mail from anMTAand delivers > it to a user's mailbox, while keeping Dovecot index files up to > date." I am wondering if LMTP also interacts with the Dovecot index > files and keeps them up to date? Yes... although it doesn't seem to state that explicitly in the wiki... http://wiki2.dovecot.org/LMTP -- Best regards, Charles From jbates at brightok.net Wed Oct 3 18:13:40 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 03 Oct 2012 10:13:40 -0500 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C53B6.6070706@Media-Brokers.com> References: <506C4D0B.8030709@airstreamcomm.net> <506C53B6.6070706@Media-Brokers.com> Message-ID: <506C5624.8090609@brightok.net> On 10/3/2012 10:03 AM, Charles Marcus wrote: > On 2012-10-03 10:34 AM, list at airstreamcomm.net > wrote: >> In the docs it states that LDA " ...takes mail from anMTAand delivers >> it to a user's mailbox, while keeping Dovecot index files up to >> date." I am wondering if LMTP also interacts with the Dovecot index >> files and keeps them up to date? > > Yes... although it doesn't seem to state that explicitly in the wiki... > > http://wiki2.dovecot.org/LMTP > "The main difference is that the LDA is a short-running process, started as a binary from command line, while LMTP is a long-running process started by Dovecot's master process." I think they are trying to reduce duplication of information. Jack From patrickdk at patrickdk.com Wed Oct 3 19:02:20 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 03 Oct 2012 12:02:20 -0400 Subject: [Dovecot] possible nfs issue In-Reply-To: References: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> Message-ID: <20121003120220.Horde.5oXwJJLnE6FQbGGMsr6Tr4A@mail.patrickdk.com> Maybe it's a cross program issue? We used to randomly have this happen a long time ago, when using postfix and dovecot. Since switching to using the dovecot lda/lmtp instead of postfix for mailbox delievery, I haven't seen this happen at all anymore. I'm not saying that postfix is at fault for this, but could be a timing/race issue between postfix/dovecot accesses to the mailbox. Quoting Cor Bosman : > On Oct 3, 2012, at 12:35 AM, Timo Sirainen wrote: > >> On 3.10.2012, at 0.45, Timo Sirainen wrote: >> >>> On 3.10.2012, at 0.39, Cor Bosman wrote: >>> >>>> With NFS these files are created when a file gets unlinked, but >>>> another process still has it open. It disappears as soon as the >>>> other process closes it. For some reason they dont disappear. As >>>> far as I can tell we've had no server crashes that could explain >>>> this. One possible theory is that a rename happens after an >>>> unlink. In that case the file remains. This could possibly be a >>>> dovecot issue. >>> >>> How can a rename happen after unlink? The rename should fail. >>> (Unless doing rename(.nfs1234, something), but Dovecot definitely >>> isn't doing that.) >> >> You could see if this old test program leaves .nfs files lying around: >> >> http://dovecot.org/tmp/readdir.c >> >> Just comment out the line: >> >> close(fd); >> > > I meant the .nfs1234 indeed, but it seemed very unlikely. Thanks for > clarifying. The readdir program leaves no .nfs files. We'll have to > explore other possibilities. > > Cor From list at airstreamcomm.net Wed Oct 3 22:26:20 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 03 Oct 2012 14:26:20 -0500 Subject: [Dovecot] LMTP userdb lookup Message-ID: <506C915C.2070709@airstreamcomm.net> Is it possible to have separate userdb lookups for LMTP and POP/IMAP? From marc at perkel.com Wed Oct 3 22:48:21 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 12:48:21 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release Message-ID: <506C9685.8070906@perkel.com> Hi, I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first. I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives. I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend. However What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders? Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains? One thing I can do is deliver the spam to 3 different places so it's visible on all levels. I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system. From ecasarero at gmail.com Wed Oct 3 22:58:35 2012 From: ecasarero at gmail.com (Eduardo Casarero) Date: Wed, 3 Oct 2012 16:58:35 -0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: Hi Marc, i solved this using an automated report for users quarantine. In front of dovecot i have 2 mailscanner boxes that stores spam emails in quarantine and logs them to a database, periodically there is a script that sends an html report to users that recieved spam in the last interval (1h, 4h, 24hs depending on the user preferences) showing a list of time-from-subject of all new items in quarantine. There is also a link to release the email from quarantine and the users recieves it on his inbox. So our users can release emails without bothering anyone. (There is also an admin view where the admin can see all the trafic for the domain). my 2cents. regards, eduardo. 2012/10/3 Marc Perkel > Hi, > > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. (Junk > Email Filter) I want to add a system where I store a copy of spam on a > server and make it available to the customer to review and maybe resent on > false positives. > > I know I could do something simple where I deliver all spam to a domain > account and make it available to an administrator. Then if it's a false > positive they would drag the message to a "resend" folder. I'll have > something the checks the folder one a minute to pick up and resend. > > However > > What would be very cool is delivering the spam to individual accounts. So > a user who logs in individually can see their own spam. But the admin for > the domain would be able to see all users. Maybe the users would appear as > folders? > > Then a master account (me) would be able to log in and see all the domains > as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And I'll > need an authentication system. > > From campbell at cnpapers.com Wed Oct 3 22:59:09 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Wed, 03 Oct 2012 15:59:09 -0400 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506C990D.50800@cnpapers.com> If you ever figure out how to do this, I've got an excellent name for it: MailWatch http://sourceforge.net/projects/mailwatch/ steve On 10/3/2012 3:48 PM, Marc Perkel wrote: > Hi, > > I'm looking for some advice to do a really advanced trick with > Dovecot. I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of > spam on a server and make it available to the customer to review and > maybe resent on false positives. > > I know I could do something simple where I deliver all spam to a > domain account and make it available to an administrator. Then if it's > a false positive they would drag the message to a "resend" folder. > I'll have something the checks the folder one a minute to pick up and > resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the > admin for the domain would be able to see all users. Maybe the users > would appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > From tss at iki.fi Wed Oct 3 23:04:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 03 Oct 2012 23:04:39 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: Maildir, layout=fs /var/vmail/domain/user/ Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Marc Perkel wrote: Hi, I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first. I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives. I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend. However What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders? Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains? One thing I can do is deliver the spam to 3 different places so it's visible on all levels. I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system. From hakon at alstadheim.priv.no Wed Oct 3 23:14:37 2012 From: hakon at alstadheim.priv.no (=?ISO-8859-1?Q?H=E5kon_Alstadheim?=) Date: Wed, 03 Oct 2012 22:14:37 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506C9CAD.5080004@alstadheim.priv.no> On 03. okt. 2012 21:48, Marc Perkel wrote: > Hi, > > I'm looking for some advice to do a really advanced trick with > Dovecot. I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of > spam on a server and make it available to the customer to review and > maybe resent on false positives. > > I know I could do something simple where I deliver all spam to a > domain account and make it available to an administrator. Then if it's > a false positive they would drag the message to a "resend" folder. > I'll have something the checks the folder one a minute to pick up and > resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the > admin for the domain would be able to see all users. Maybe the users > would appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > > Check out the dovecot sieve plugin. I use the following default pre-filter for all users: --- require ["regex", "fileinto", "imap4flags"]; # Catch mail tagged as Spam, except Spam retrained and delivered to the mailbox if allof (header :regex "X-DSPAM-Result" "^(Spam|Virus|Bl[ao]cklisted)$", not header :contains "X-DSPAM-Reclassified" "Innocent", not header :contains "Received-SPF" "pass .securityfocus.com") { # Mark as read #setflag "\\Seen"; addflag "$junk"; # Move into the Junk folder fileinto "INBOX.Junk"; # Stop processing here stop; } ----- Together with the dovecot antispam plugin this makes the beginnings of a very intuitive system. I just click to remove the junk flag on any false positive, and it gets re-delivered to me. The dovecot lda also supports a switch to deliver to a specific folder I believe. This would be an alternative if you get the spam delivered through a separate channel anyway. ------- The other part of your requirements could be met by using dovecot public folders, which I have never used myself. Maybe set up so admins can subscribe to the junk-folder of any user they want ? Refiling false positives might get messy for an admin though. Regards, H?kon. From tss at iki.fi Wed Oct 3 23:58:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 23:58:21 +0300 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <506C915C.2070709@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> Message-ID: On 3.10.2012, at 22.26, list at airstreamcomm.net wrote: > Is it possible to have separate userdb lookups for LMTP and POP/IMAP? protocol lmtp { userdb { .. } } protocol !lmtp { userdb { .. } } From marc at perkel.com Thu Oct 4 02:42:52 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 16:42:52 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> Message-ID: <506CCD7C.6070507@perkel.com> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? On 10/3/2012 1:04 PM, Timo Sirainen wrote: > Maildir, layout=fs > /var/vmail/domain/user/ > > Spams get delivered there. User has access there. Domain admin has > mail_location=/var/vmail/domain, you have mail_location=/var/vmail > -- > Sent from my Android phone with K-9 Mail. Please excuse my brevity. > > Marc Perkel wrote: > > Hi, > > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of spam > on a server and make it available to the customer to review and maybe > resent on false positives. > > I know I could do something simple where I deliver all spam to a domain > account and make it available to an administrator. Then if it's a false > positive they would drag the message to a "resend" folder. I'll have > something the checks the folder one a minute to pick up and resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the admin > > for the domain would be able to see all users. Maybe the users would > appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > From tss at iki.fi Thu Oct 4 02:46:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 02:46:34 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506CCD7C.6070507@perkel.com> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: On 4.10.2012, at 2.42, Marc Perkel wrote: > On 10/3/2012 1:04 PM, Timo Sirainen wrote: >> Maildir, layout=fs >> /var/vmail/domain/user/ >> >> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail > I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? users: mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs domain admins: mail_location = maildir:/var/vmail/%d:LAYOUT=fs full admins: mail_location = maildir:/var/vmail:LAYOUT=fs The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. From marc at perkel.com Thu Oct 4 03:10:23 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 17:10:23 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: <506CD3EF.6080704@perkel.com> On 10/3/2012 4:46 PM, Timo Sirainen wrote: > On 4.10.2012, at 2.42, Marc Perkel wrote: > >> On 10/3/2012 1:04 PM, Timo Sirainen wrote: >>> Maildir, layout=fs >>> /var/vmail/domain/user/ >>> >>> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail >> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? > users: > mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs > > domain admins: > mail_location = maildir:/var/vmail/%d:LAYOUT=fs > > full admins: > mail_location = maildir:/var/vmail:LAYOUT=fs > > The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. > > I'm testing it now and the user level works. But the other levels I don't see anything. I am a little brain dead today though. I'll test more From kgc at corp.sonic.net Thu Oct 4 04:55:42 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 03 Oct 2012 18:55:42 -0700 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <506CEC9E.9060105@corp.sonic.net> On 10/2/2012 2:39 PM, Cor Bosman wrote: > Anyone else with NFS mailspools seeing this? Yes, it is like 1999 all over again. I haven't had a chance to track them down or setup a cron job to rm them all. All of the ones I'm seeing are ex dovecot.index files but it looks like yours are ex messages? I figured this was a probably a regression in the RHEL6.3 (Sl6.3) (2.6.32-279.9.1.el6.x86_64) kernel. What are you running Cor? -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From marc at perkel.com Thu Oct 4 05:28:19 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 19:28:19 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: <506CF443.5080904@perkel.com> On 10/3/2012 4:46 PM, Timo Sirainen wrote: > On 4.10.2012, at 2.42, Marc Perkel wrote: > >> On 10/3/2012 1:04 PM, Timo Sirainen wrote: >>> Maildir, layout=fs >>> /var/vmail/domain/user/ >>> >>> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail >> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? > users: > mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs > > domain admins: > mail_location = maildir:/var/vmail/%d:LAYOUT=fs > > full admins: > mail_location = maildir:/var/vmail:LAYOUT=fs > > The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. > > Hi Timo, Thanks for your help. I think I'm close. This works: mail_location = maildir:/email/%d/%n:LAYOUT=fs This doesn't: mail_location = maildir:/email/%d:LAYOUT=fs The email client doesn't see the directories as folders and nothing is visible. I must be missing something. From marc at perkel.com Thu Oct 4 07:40:37 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 21:40:37 -0700 Subject: [Dovecot] Getting rid of the subscription file Message-ID: <506D1345.9070406@perkel.com> I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? Thanks in advance. From daniel.parthey at informatik.tu-chemnitz.de Thu Oct 4 07:58:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 4 Oct 2012 06:58:18 +0200 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1345.9070406@perkel.com> References: <506D1345.9070406@perkel.com> Message-ID: <20121004045818.GA15696@daniel.localdomain> Marc Perkel wrote: > I'd like to eliminate the subscription file and have it just list > the folders that are there. How do I do that? Dovecot allows to automatically subscribe folders when mails are delivered: lda_mailbox_autosubscribe = yes RFC 3501 tells that the server side MUST NOT unilaterally remove an existing mailbox name from the subscription list even if a mailbox by that name no longer exists, see SUBSCRIBE in section 6.3.6: http://tools.ietf.org/html/rfc3501#section-6.3.6 Regards Daniel -- https://plus.google.com/103021802792276734820 From marc at perkel.com Thu Oct 4 08:06:28 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 22:06:28 -0700 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <20121004045818.GA15696@daniel.localdomain> References: <506D1345.9070406@perkel.com> <20121004045818.GA15696@daniel.localdomain> Message-ID: <506D1954.1000204@perkel.com> On 10/3/2012 9:58 PM, Daniel Parthey wrote: > Marc Perkel wrote: >> I'd like to eliminate the subscription file and have it just list >> the folders that are there. How do I do that? > Dovecot allows to automatically subscribe folders when mails are delivered: > lda_mailbox_autosubscribe = yes > > RFC 3501 tells that the server side MUST NOT unilaterally remove an > existing mailbox name from the subscription list even if a mailbox > by that name no longer exists, see SUBSCRIBE in section 6.3.6: > http://tools.ietf.org/html/rfc3501#section-6.3.6 > > Regards > Daniel In my case I don't care what the standard is. I want to get rid of the subscription concept completely. From amateo at um.es Thu Oct 4 09:14:29 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 04 Oct 2012 08:14:29 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> Message-ID: <506D2945.9040208@um.es> El 03/10/12 15:27, Timo Sirainen escribi?: > On 3.10.2012, at 9.20, Angel L. Mateo wrote: > >> El 02/10/12 21:38, Timo Sirainen escribi?: >>> On 26.9.2012, at 13.55, Angel L. Mateo wrote: >>> >>>> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >>>> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >>>> done >>> >>> -S auth-userdb? You've named it completely wrong if that works :) >>> >> auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? > > -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? > Yes, I have tried and it works without using -S. So, what is the reason for this option? Because I'm sure I'm using because I've read it in examples (not with auth-userdb obviously, this is my mistake) From robert at schetterer.org Thu Oct 4 09:36:39 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 04 Oct 2012 08:36:39 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506D2E77.5030303@schetterer.org> Am 03.10.2012 21:48, schrieb Marc Perkel: > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of spam > on a server and make it available to the customer to review and maybe > resent on false positives. this is the job of your filter comapny first, anyway , dont use them anymore and use i.e amavis with quarantaine i dont think other cases make sense in real by getting very complicated -- Best Regards MfG Robert Schetterer From cor at xs4all.nl Thu Oct 4 11:11:56 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 4 Oct 2012 10:11:56 +0200 Subject: [Dovecot] possible nfs issue In-Reply-To: <506CEC9E.9060105@corp.sonic.net> References: <506CEC9E.9060105@corp.sonic.net> Message-ID: <5A995EF4-6EAA-41FF-926B-912FFD59EC07@xs4all.nl> On Oct 4, 2012, at 3:55 AM, Kelsey Cummings wrote: > On 10/2/2012 2:39 PM, Cor Bosman wrote: >> Anyone else with NFS mailspools seeing this? > > Yes, it is like 1999 all over again. I haven't had a chance to track them down or setup a cron job to rm them all. All of the ones I'm seeing are ex dovecot.index files but it looks like yours are ex messages? > > I figured this was a probably a regression in the RHEL6.3 (Sl6.3) (2.6.32-279.9.1.el6.x86_64) kernel. What are you running Cor? We're running debian with a 3.2.2 kernel. Just yesterday one of my colleagues had a few new ones in his mailspool. Definitely no server crash or anything. Something is creating these outside the 'normal' parameters for .nfs files. My colleague said these were emails he deleted that day. We've set up a cleaning run, and are probably going to ignore it for now. These things are near impossible to track down without a lot of debugging. Cor From marc at perkel.com Thu Oct 4 15:54:35 2012 From: marc at perkel.com (Marc Perkel) Date: Thu, 04 Oct 2012 05:54:35 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506D2E77.5030303@schetterer.org> References: <506C9685.8070906@perkel.com> <506D2E77.5030303@schetterer.org> Message-ID: <506D870B.5020001@perkel.com> On 10/3/2012 11:36 PM, Robert Schetterer wrote: > Am 03.10.2012 21:48, schrieb Marc Perkel: >> I'm looking for some advice to do a really advanced trick with Dovecot. >> I'm not sure if this can be done. I need to describe first. >> >> I have a spam filtering company that does front end spam filtering. >> (Junk Email Filter) I want to add a system where I store a copy of spam >> on a server and make it available to the customer to review and maybe >> resent on false positives. > this is the job of your filter comapny first, > anyway , dont use them anymore and use i.e amavis with quarantaine > i dont think other cases make sense in real by getting very complicated I am the spam filtering company. :) From benedetto.vassallo at unipa.it Thu Oct 4 16:00:03 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Thu, 04 Oct 2012 15:00:03 +0200 Subject: [Dovecot] Maildir hardlinks Message-ID: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Hello list, Excuse me for my poor english. I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. All works fine, but with the new version it seems that dovecot don't do hardlinks when deliver a message to multiple users. I have checked my config and the only rule I can see aboout that is maildir_copy_with_hardlinks = yes in /etc/dovecot/conf.d/10-mail.conf My mail location config is mail_location = maildir:~/MailDir:LAYOUT=fs I tryed using lmtp directly issuing 'telnet localhost 24' and sending a test message to 3 recipients. Then issuing a 'ls -il' in the "new" directory of that users, I saw the inode was not the same. I rechecked my config and take a look in the wiki and in the list for someone who had the same problems, but found nothing. Maybe I did something wrong, but I can't understand what. Can you help me? Thank you -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From tss at iki.fi Thu Oct 4 17:20:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:20:03 +0300 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1345.9070406@perkel.com> References: <506D1345.9070406@perkel.com> Message-ID: On 4.10.2012, at 7.40, Marc Perkel wrote: > I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? Write a plugin. From tss at iki.fi Thu Oct 4 17:21:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:21:02 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <506D2945.9040208@um.es> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> <506D2945.9040208@um.es> Message-ID: <2E23C2EC-6324-4182-9828-1F063F63C86C@iki.fi> On 4.10.2012, at 9.14, Angel L. Mateo wrote: >> -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? >> > Yes, I have tried and it works without using -S. So, what is the reason for this option? Because I'm sure I'm using because I've read it in examples (not with auth-userdb obviously, this is my mistake) I'm guessing that it's used only when the user lookup isn't returning proxy=y From tss at iki.fi Thu Oct 4 17:29:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:29:02 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506CF443.5080904@perkel.com> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> Message-ID: <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> On 4.10.2012, at 5.28, Marc Perkel wrote: > Thanks for your help. I think I'm close. > > This works: > mail_location = maildir:/email/%d/%n:LAYOUT=fs > > This doesn't: > mail_location = maildir:/email/%d:LAYOUT=fs > > The email client doesn't see the directories as folders and nothing is visible. I must be missing something. Dunno. At least this method of testing works: create test mail: doveadm -O -o mail=maildir:/tmp/vmail/domain/user mailbox create INBOX touch /tmp/vmail/domain/user/cur/newmail test that user at domain works: ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs a select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS test that domain works: ./imap -O -o mail=maildir:/tmp/vmail/domain:LAYOUT=fs b list "" * * LIST (\HasNoChildren) "/" "user" * LIST (\HasNoChildren) "/" "INBOX" b OK List completed. c select user * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 0 RECENT test that superuser works: ./imap -O -o mail=maildir:/tmp/vmail:LAYOUT=fs d list "" * * LIST (\Noselect \HasChildren) "/" "domain" * LIST (\HasNoChildren) "/" "domain/user" * LIST (\HasNoChildren) "/" "INBOX" d OK List completed. e select domain/user * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS From list at airstreamcomm.net Thu Oct 4 17:41:39 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 04 Oct 2012 09:41:39 -0500 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: References: <506C915C.2070709@airstreamcomm.net> Message-ID: <506DA023.5030609@airstreamcomm.net> On 10/3/12 3:58 PM, Timo Sirainen wrote: > On 3.10.2012, at 22.26, list at airstreamcomm.net wrote: > >> Is it possible to have separate userdb lookups for LMTP and POP/IMAP? > protocol lmtp { > userdb { > .. > } > } > protocol !lmtp { > userdb { > .. > } > } > > Forgot to mention I am running 2.0.17. I separated all the userdb passdb lookups into their own protocol configuration like so: protocol imap { userdb { .. } passdb { .. } } protocol pop3 { userdb { .. } passdb { .. } } protocol lmtp { userdb { .. } } And I am getting the following error: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one From a previous post it appears that Dovecot cannot run without a global lookups specified: http://www.dovecot.org/list/dovecot/2012-March/064407.html Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? From tss at iki.fi Thu Oct 4 17:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:58:53 +0300 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <506DA023.5030609@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> Message-ID: On 4.10.2012, at 17.41, list at airstreamcomm.net wrote: >> protocol lmtp { >> userdb { >> .. >> } >> } >> protocol !lmtp { >> userdb { >> .. >> } >> } >> >> > Forgot to mention I am running 2.0.17. The above works in v2.1. > And I am getting the following error: > > auth: Fatal: No passdbs specified in configuration file. PLAIN > mechanism needs one > > > From a previous post it appears that Dovecot cannot run without a global lookups specified: > > http://www.dovecot.org/list/dovecot/2012-March/064407.html > > Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? The !lmtp version avoids that fatal problem. So the solution is: upgrade. From CMarcus at Media-Brokers.com Thu Oct 4 18:03:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 04 Oct 2012 11:03:02 -0400 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Message-ID: <506DA526.4020606@Media-Brokers.com> On 2012-10-04 9:00 AM, Benedetto Vassallo wrote: > Hello list, > Excuse me for my poor english. > I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. If you are going to update, why ohy why update to an outdated version? Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... From benedetto.vassallo at unipa.it Thu Oct 4 19:13:41 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Thu, 04 Oct 2012 18:13:41 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <506DA526.4020606@Media-Brokers.com> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <506DA526.4020606@Media-Brokers.com> Message-ID: <20121004181341.14266g7w0m2ie75h@webmail.unipa.it> Def. Quota Charles Marcus : > On 2012-10-04 9:00 AM, Benedetto Vassallo > wrote: >> Hello list, >> Excuse me for my poor english. >> I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. > > If you are going to update, why ohy why update to an outdated version? > > Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... > I did it, but still don't work :-( -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From slusarz at curecanti.org Thu Oct 4 21:57:45 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Thu, 04 Oct 2012 12:57:45 -0600 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1954.1000204@perkel.com> References: <506D1345.9070406@perkel.com> <20121004045818.GA15696@daniel.localdomain> <506D1954.1000204@perkel.com> Message-ID: <20121004125745.Horde.Wz49XoF5lbhQbdwp53YwxRA@bigworm.curecanti.org> Quoting Marc Perkel : > On 10/3/2012 9:58 PM, Daniel Parthey wrote: >> Marc Perkel wrote: >>> I'd like to eliminate the subscription file and have it just list >>> the folders that are there. How do I do that? [snip] >> RFC 3501 tells that the server side MUST NOT unilaterally remove an >> existing mailbox name from the subscription list even if a mailbox >> by that name no longer exists, see SUBSCRIBE in section 6.3.6: >> http://tools.ietf.org/html/rfc3501#section-6.3.6 >> >> Regards >> Daniel > > In my case I don't care what the standard is. I want to get rid of > the subscription concept completely. Use an MUA that allows configuration to explicitly ignore the subscription concept. michael From micha at krausam.de Fri Oct 5 10:45:07 2012 From: micha at krausam.de (Micha Krause) Date: Fri, 05 Oct 2012 09:45:07 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> Message-ID: <506E9003.7030201@krausam.de> Hi, > ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs Wow, thats a really cool way to debug/test mailboxes, is this documented somewhere? What does -O do, any other interesting options? Micha Krause From stsiol at yahoo.co.uk Fri Oct 5 15:00:20 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Fri, 5 Oct 2012 13:00:20 +0100 (BST) Subject: [Dovecot] horde sync status ? Message-ID: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Hello all, I had a quick look at the horde site and noticed that horde is being advertised as, let's say, "smartphone friendly". Does anyone know if the newest horde version can "talk" to? smart phones in regards to e-mails ? In other words, can a user owning a smartphone get his/her e-mails on it apart from the webpage ? Thank you, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From h.reindl at thelounge.net Fri Oct 5 15:05:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 05 Oct 2012 14:05:03 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506ECCEF.1020904@thelounge.net> Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > Hello all, > > I had a quick look at the horde site and noticed that > horde is being advertised as, let's say, "smartphone friendly". > > Does anyone know if the newest horde version can "talk" to > smart phones in regards to e-mails ? > > In other words, can a user owning a smartphone get his/her > e-mails on it apart from the webpage? a little off-topic at all, but however horde/imp is a webmail and accessing the mailserver via IMAP smart-phone freindly means it can be used on smartphones without scrolling to dead horde is not for having a layer between mail-client on the smartphone and the server - this makes pretty no sense each smartphone these days has a mail-client like K9 on android and can access imap/exchange directly - why should horde be involved here als additional layer? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From patrickdk at patrickdk.com Fri Oct 5 17:17:44 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Fri, 05 Oct 2012 10:17:44 -0400 Subject: [Dovecot] horde sync status ? In-Reply-To: <506ECCEF.1020904@thelounge.net> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506ECCEF.1020904@thelounge.net> Message-ID: <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> Quoting Reindl Harald : > Am 05.10.2012 14:00, schrieb Spyros Tsiolis: >> Hello all, >> >> I had a quick look at the horde site and noticed that >> horde is being advertised as, let's say, "smartphone friendly". >> >> Does anyone know if the newest horde version can "talk" to >> smart phones in regards to e-mails ? >> >> In other words, can a user owning a smartphone get his/her >> e-mails on it apart from the webpage? > > a little off-topic at all, but however > > horde/imp is a webmail and accessing the mailserver via IMAP > smart-phone freindly means it can be used on smartphones > without scrolling to dead > > horde is not for having a layer between mail-client on the > smartphone and the server - this makes pretty no sense > > each smartphone these days has a mail-client like K9 on > android and can access imap/exchange directly - why should > horde be involved here als additional layer? Many reasons for this, I personally use it for contact and calender sync, and the new version of horde that is still in beta, can also be used for email sync too. This will simplify setup for many people, using autodiscovery feature of activesync. From tss at iki.fi Fri Oct 5 17:31:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 5 Oct 2012 17:31:18 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506E9003.7030201@krausam.de> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> Message-ID: <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> On 5.10.2012, at 10.45, Micha Krause wrote: >> ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs > > Wow, thats a really cool way to debug/test mailboxes, is this documented somewhere? No. The -O, -o, -k and some other options should be put into some new global.inc where it gets included to all doveadm/dovecot/doveconf man pages.. > What does -O do, any other interesting options? All the global settings are: -O ignores dovecot.conf and just uses the default settings. -o = can be used multiple times to override any setting -k preserves environment variables (which can also be used to override settings, e.g. MAIL=foo) -c changes dovecot.conf path -i changes to dovecot.conf used by the given instance name -L logs directly to destination specified by log_path/info_log_path/debug_log_path, bypassing log process (allowing logging to different location than normally, log process always logs only to one location) From h.reindl at thelounge.net Fri Oct 5 17:38:50 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 05 Oct 2012 16:38:50 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506ECCEF.1020904@thelounge.net> <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> Message-ID: <506EF0FA.1010307@thelounge.net> Am 05.10.2012 16:17, schrieb Patrick Domack: >> each smartphone these days has a mail-client like K9 on >> android and can access imap/exchange directly - why should >> horde be involved here als additional layer? > > Many reasons for this, I personally use it for contact and calender sync, and the new version of horde that is > still in beta, can also be used for email sync too. This will simplify setup for many people, using autodiscovery > feature of activesync. why does one need this with IMAP as mail-backend? sounds more like "having solution and searching fro problem" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From robert at schetterer.org Fri Oct 5 17:56:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 05 Oct 2012 16:56:36 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506EF524.7060604@schetterer.org> Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > Hello all, > > I had a quick look at the horde site and noticed that > horde is being advertised as, let's say, "smartphone friendly". > > Does anyone know if the newest horde version can "talk" to > smart phones in regards to e-mails ? yes since version 5 > > In other words, can a user owning a smartphone get his/her > e-mails on it apart from the webpage ? horde 5 acts as active-sync server for mail , calendar, adressbook ,tasks ,notes syncml with funambol app on the smartphone side for calendar, adressbook ,tasks ,notes roadmap 5.1 is planned as card/caldav server http://wiki.horde.org/ActiveSync > > Thank you, > > spyros > > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > this is off topic with the dovecot list -- Best Regards MfG Robert Schetterer From mikydevel at yahoo.fr Fri Oct 5 20:30:45 2012 From: mikydevel at yahoo.fr (Mik J) Date: Fri, 5 Oct 2012 18:30:45 +0100 (BST) Subject: [Dovecot] Dovecot configuration and question about IP trusted Message-ID: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> Hello list, I've just finished to install Dovecot and things seems to work so far. After some little efforts though. My version is 2.0.20 Question 1: I'm trying to tighten the security a little bit and added in dovecot.conf login_trusted_networks = 192.168.1.0/30 Then restarted Dovecot My client has the IP 192.168.1.20 and it's still able to retrieve emails. I expected it to be forbidden. Am I missing something ? # dovecot -n | grep trust login_trusted_networks = 192.168.1.0/30 Question 2: I feel that Dovecot is slow. I'm doing my test with my iphone as an imap client. Test 1: I retrieve a mail on a remote server provided by a hosting company, it takes 2 seconds Test 2: I retrieve a mail on my server which is on my LAN, the mail includes a few letters in the subject and a few letters in the body. The action takes about 8 seconds. It's quite subtule to measure so first I would like to know if Dovecot tries to do a dns reverse lookup or something like that. And it would explain the overhead. Thank you From bob at computerisms.ca Fri Oct 5 21:01:13 2012 From: bob at computerisms.ca (Bob Miller) Date: Fri, 05 Oct 2012 11:01:13 -0700 Subject: [Dovecot] Dovecot configuration and question about IP trusted In-Reply-To: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> Message-ID: <1349460073.4213.59.camel@worklian> Hi, > I > 'm trying to tighten the security a little bit and added in dovecot.conf > login_trusted_networks = 192.168.1.0/30 > Then restarted Dovecot > > > My client has the IP 192.168.1.20 and it's still able to retrieve emails. I expected it to be forbidden. Am I missing something ? My interpretation of the documentation indicates that the trusted network setting causes certain authentication and security checks to be bypassed if a computer is in the trusted network, and to not bypass those authentication and security checks if the computer is not in the trusted range. I see nothing indicating this setting will "forbid" anything... > I feel that Dovecot is slow. I'm doing my test with my iphone as an imap client. > Test 1: I retrieve a mail on a remote server provided by a hosting company, it takes 2 seconds > Test 2: I retrieve a mail on my server which is on my LAN, the mail includes a few letters in the subject and a few letters in the body. The action takes about 8 seconds. > It's quite subtule to measure so first I would like to know if Dovecot tries to do a dns reverse lookup or something like that. And it would explain the overhead. I don't know about the reverse lookup, but this sounds like a caching issue to me. http://wiki2.dovecot.org/IndexFiles > > Thank you -- Computerisms Bob Miller 867-334-7117 / 867 633 3760 http://computerisms.ca From lists at luigirosa.com Fri Oct 5 21:14:53 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Fri, 05 Oct 2012 20:14:53 +0200 Subject: [Dovecot] IPv6 & SSL Message-ID: <506F239D.6090007@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I have a dual stack server with Dovecot 2.1.10 listening on v4 and v6 Dovecot has a Comodo SSL certificate issued via NameCheap that works as expected with IPv4 in 10-ssl.conf I have enabled these configuraction directives: ssl = yes ssl_cert = < /path/to/file.crt ssl_key = < /path/to/file.key ssl_parameters_regenerate = 202 hours If I connect to Dovecot using the IPv6 address of the server with Thunderbird 15.0.1 uising CRAM-MD5 averything is ok. If I enable SSL _and_ IPv6 on Thunderbird I get this error: Oct 5 20:05:04 mail dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=2001:470:1f09:203:fdbf:508e:4a29:56c5, lip=2001:470:1f09:203::badd:ecaf, TLS: SSL_read() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, session= Ciao, luigi - -- / +--[Luigi Rosa]-- \ I will tell you a great secret, Captain. Perhaps the greatest of all time. The molecules of your body are the same molecules that make up this station and the nebula outside, that burn inside the stars themselves. We are star stuff, we are the universe made manifest, trying to figure itself out. As we have both learned, sometimes the universe requires a change of perspective." --Delenn, "Distant Star", Babylon 5 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBvI50ACgkQ3kWu7Tfl6ZRBSACfRkp4FYpWaEZUQhIh0t6Vfs/I JbcAoKGZ769yogYS7faCXKvPTuhQiHA8 =jxCB -----END PGP SIGNATURE----- From lists at wildgooses.com Fri Oct 5 22:37:37 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 05 Oct 2012 20:37:37 +0100 Subject: [Dovecot] horde sync status ? In-Reply-To: <506EF524.7060604@schetterer.org> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506EF524.7060604@schetterer.org> Message-ID: <506F3701.5050805@wildgooses.com> On 05/10/2012 15:56, Robert Schetterer wrote: > Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > >> In other words, can a user owning a smartphone get his/her >> e-mails on it apart from the webpage ? > horde 5 acts as active-sync server > for mail , calendar, adressbook ,tasks ,notes > > syncml with funambol app on the smartphone side > for calendar, adressbook ,tasks ,notes > > roadmap > 5.1 is planned as card/caldav server > > http://wiki.horde.org/ActiveSync > Also see Sogo (and owncloud). Plus the Sogosync connector This is a developing area (at last) Ed W From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 5 22:48:47 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 5 Oct 2012 21:48:47 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> Message-ID: <20121005194847.GA15222@daniel.localdomain> Timo Sirainen wrote: > -i changes to dovecot.conf used by the given instance name This does not seem to work, at least not with version 2.1.10: mail01:~# doveadm instance list path name last used running /var/run/dovecot dovecot-mailbox 2012-10-05 19:19:33 yes /var/run/dovecot-director dovecot-director 2012-10-05 19:20:13 yes mail01:~# doveadm -c /etc/dovecot-director/dovecot-director.conf director status dparthey at example.org Current: 10.129.3.192 (expires 2012-10-07 20:10:25) Hashed: 10.129.3.192 Initial config: 10.129.3.192 mail01:~# doveadm -i dovecot-director director status dparthey at example.org doveadm(root): Fatal: read(/var/run/dovecot/director-admin) failed: Connection reset by peer Regards Daniel -- https://plus.google.com/103021802792276734820 From nick+dovecot at bunbun.be Fri Oct 5 23:47:53 2012 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Fri, 05 Oct 2012 22:47:53 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F239D.6090007@luigirosa.com> References: <506F239D.6090007@luigirosa.com> Message-ID: <506F4779.4040109@bunbun.be> Luigi Rosa wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > I have a dual stack server with Dovecot 2.1.10 listening on v4 and v6 > > Dovecot has a Comodo SSL certificate issued via NameCheap that works as > expected with IPv4 > > in 10-ssl.conf I have enabled these configuraction directives: > > ssl = yes > ssl_cert =< /path/to/file.crt > ssl_key =< /path/to/file.key > ssl_parameters_regenerate = 202 hours > > > If I connect to Dovecot using the IPv6 address of the server with Thunderbird > 15.0.1 uising CRAM-MD5 averything is ok. > If I enable SSL _and_ IPv6 on Thunderbird I get this error: How do you enable this in Thunderbird? If by "enabling IPv6" you mean you put in the IPv6 address in stead of the hostname, that's probably where you're wrong. The certificate contains your hostname, not the IP-address so the hostname verification check fails if you insert the IPv6 address (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your connection fails). I've verified this by changing the hostname to IPv6 in Thunderbird and got the same error as you do. You would get the same error if you configure the IPv4 address in TB. > Oct 5 20:05:04 mail dovecot: imap-login: Disconnected (no auth attempts in 1 > secs): user=<>, rip=2001:470:1f09:203:fdbf:508e:4a29:56c5, > lip=2001:470:1f09:203::badd:ecaf, TLS: SSL_read() failed: error:14094418:SSL > routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, > session= This is a valid connection when I use the hostname: 2012-10-04T18:07:51.614187+02:00 mail dovecot: imap-login: Login: user=, method=CRAM-MD5, rip=yyyy:yyyy:::yyyy, lip=xxxx:xxxx:::xxxx, mpid=58179, TLS, TLSv1 with cipher RC4-MD5 (128/128 bits) Configure your DNS so your hostname points to both the IPv6 and IPv4 address. Your client will take take whichever protocol is preferred (IPv4 or IPv6). Rgds, N. > > Ciao, > luigi > > - -- > / > +--[Luigi Rosa]-- > \ > > I will tell you a great secret, Captain. Perhaps the greatest of all > time. The molecules of your body are the same molecules that make up > this station and the nebula outside, that burn inside the stars > themselves. We are star stuff, we are the universe made manifest, > trying to figure itself out. As we have both learned, sometimes > the universe requires a change of perspective." > --Delenn, "Distant Star", Babylon 5 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Mozilla -http://www.enigmail.net/ > > iEYEARECAAYFAlBvI50ACgkQ3kWu7Tfl6ZRBSACfRkp4FYpWaEZUQhIh0t6Vfs/I > JbcAoKGZ769yogYS7faCXKvPTuhQiHA8 > =jxCB > -----END PGP SIGNATURE----- From lists at luigirosa.com Sat Oct 6 08:20:20 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 07:20:20 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F4779.4040109@bunbun.be> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> Message-ID: <506FBF94.30607@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nick Rosier said the following on 05/10/12 22:47: >> How do you enable this in Thunderbird? If by "enabling IPv6" you mean you >> put in the IPv6 address in stead of the hostname, that's probably where >> you're wrong. The certificate contains your hostname, not the IP-address >> so the hostname verification check fails if you insert the IPv6 address >> (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your >> connection fails). Good point. But does not explain why it works if I put the IPv4 address of the server (the local LAN IPv4, not the public IPv4). >> I've verified this by changing the hostname to IPv6 in Thunderbird and >> got the same error as you do. You would get the same error if you >> configure the IPv4 address in TB. The server I am referring to has 2 NICs one with a public IP and the other with a local IP address (10.0.0.254) If I put 10.0.0.254 instead of the IPv6 address I can successfully connect using TLS: Oct 6 07:13:44 mail dovecot: imap-login: Login: user=, method=CRAM-MD5, rip=10.0.0.155, lip=10.0.0.254, mpid=17812, TLS, session= >> Configure your DNS so your hostname points to both the IPv6 and IPv4 >> address. Your client will take take whichever protocol is preferred (IPv4 >> or IPv6). Thunderbird uses IPv4 as mail protocol, I wanted to test IPv6... Thank you for your help Ciao, luigi - -- / +--[Luigi Rosa]-- \ Success is 99% failure. --Soichiro Honda -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBvv4kACgkQ3kWu7Tfl6ZQp2wCgvXPgRGANlAIaVkMvXZHIThYE OiwAoIOqIMD+3mT1znMl6lCCbHanwBta =B/r2 -----END PGP SIGNATURE----- From kamath at moltingpenguin.com Sat Oct 6 08:44:56 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Fri, 5 Oct 2012 22:44:56 -0700 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506FBF94.30607@luigirosa.com> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> <506FBF94.30607@luigirosa.com> Message-ID: <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> On Oct 5, 2012, at 10:20 PM, Luigi Rosa wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Nick Rosier said the following on 05/10/12 22:47: > >>> How do you enable this in Thunderbird? If by "enabling IPv6" you mean you >>> put in the IPv6 address in stead of the hostname, that's probably where >>> you're wrong. The certificate contains your hostname, not the IP-address >>> so the hostname verification check fails if you insert the IPv6 address >>> (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your >>> connection fails). > > Good point. But does not explain why it works if I put the IPv4 address of the > server (the local LAN IPv4, not the public IPv4). > >>> I've verified this by changing the hostname to IPv6 in Thunderbird and >>> got the same error as you do. You would get the same error if you >>> configure the IPv4 address in TB. > > The server I am referring to has 2 NICs one with a public IP and the other > with a local IP address (10.0.0.254) > > If I put 10.0.0.254 instead of the IPv6 address I can successfully connect > using TLS: > > Oct 6 07:13:44 mail dovecot: imap-login: Login: user=, > method=CRAM-MD5, rip=10.0.0.155, lip=10.0.0.254, mpid=17812, TLS, > session= And do you have a PTR record for 10.0.0.254? Sean From lists at luigirosa.com Sat Oct 6 09:33:31 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 08:33:31 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> <506FBF94.30607@luigirosa.com> <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> Message-ID: <506FD0BB.2020000@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sean Kamath said the following on 06/10/12 07:44: >> Oct 6 07:13:44 mail dovecot: imap-login: Login: >> user=, method=CRAM-MD5, rip=10.0.0.155, >> lip=10.0.0.254, mpid=17812, TLS, session= > > And do you have a PTR record for 10.0.0.254? No, no PTR o other DNS entry for that address. No entry of that address in /etc/hosts on the Linux with Thunderbird or on the Linux with Dovecot. Ciao, luigi - -- / +--[Luigi Rosa]-- \ The past was erased, the erasure was forgotten, the lie became truth. --George Orwell, "1984" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBv0LsACgkQ3kWu7Tfl6ZRTUgCgh1epu40NUiZ6CPlBrcFZezt/ nMYAnjUS5IxodwJfW7o9pJHfKoVCc9xK =8O4T -----END PGP SIGNATURE----- From pw at wk-serv.de Sat Oct 6 10:29:05 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 06 Oct 2012 09:29:05 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F239D.6090007@luigirosa.com> References: <506F239D.6090007@luigirosa.com> Message-ID: Can you provide the output of doveconf -n? Regards Patrick From lists at luigirosa.com Sat Oct 6 11:10:40 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 10:10:40 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: References: <506F239D.6090007@luigirosa.com> Message-ID: <506FE780.9000900@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Patrick Westenberg said the following on 06/10/12 09:29: > Can you provide the output of doveconf -n? Sure, here it is: # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.1.1.el5.centos.plus x86_64 CentOS release 5.8 (Final) auth_cache_negative_ttl = 0 auth_cache_size = 100 k auth_cache_ttl = 8 hours auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes base_dir = /var/run/dovecot/ login_greeting = Ready. login_trusted_networks = 10.0.0.0/24 mail_plugins = " stats" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 10s stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service stats { fifo_listener stats-mail { mode = 0666 } } ssl_cert = References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> Message-ID: <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> > De?: Bob Miller > > Hi, >> I'm trying to tighten the security a little bit and added in dovecot.conf > >> login_trusted_networks = 192.168.1.0/30 >> Then restarted Dovecot >> >> >> My client has the IP 192.168.1.20 and it's still able to retrieve > emails. I expected it to be forbidden. Am I missing something ? > > My interpretation of the documentation indicates that the trusted > network setting causes certain authentication and security checks to be > bypassed if a computer is in the trusted network, and to not bypass > those authentication and security checks if the computer is not in the > trusted range.? I see nothing indicating this setting will "forbid" > anything... > >> I feel that Dovecot is slow. I'm doing my test with my iphone as an > imap client. >> Test 1: I retrieve a mail on a remote server provided by a hosting company, > it takes 2 seconds >> Test 2: I retrieve a mail on my server which is on my LAN, the mail > includes a few letters in the subject and a few letters in the body. The action > takes about 8 seconds. >> It's quite subtule to measure so first I would like to know if Dovecot > tries to do a dns reverse lookup or something like that. And it would explain > the overhead. > > I don't know about the reverse lookup, but this sounds like a caching > issue to me.? http://wiki2.dovecot.org/IndexFiles Hello Bob, Thank you for this clarification about the parameter login_trusted_networks. Regarding the indexfiles, I've read the page but I don't see at any moment, how to enable or disable the indexes. Also how do you understand this sentence "# Don't use mmap() at all. This is required if you store indexes to shared # filesystems (NFS or clustered filesystem) or for some operating systems # which use a separate cache for mmap, such as OpenBSD. mmap_disable = yes" I've read it 10 times, and I don't know if this should be set to yes or no (probably because my english is not perfect). My operating system is OpenBSD and I don't share NFS or cluster filesystems. Thank you From pw at wk-serv.de Sat Oct 6 13:02:30 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 06 Oct 2012 12:02:30 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506FE780.9000900@luigirosa.com> References: <506F239D.6090007@luigirosa.com> <506FE780.9000900@luigirosa.com> Message-ID: <507001B6.2000704@wk-serv.de> Hi Luigi, with regard to SSL my configuration is much more simple and it works fine with IPv4 and IPv6. But you have of course to use a hostname matching the certificates common name. # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain login director_mail_servers = 172.17.1.1 172.17.1.2 director_servers = 172.17.1.3 172.17.1.4 lmtp_proxy = yes log_path = /var/log/dovecot.log managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacati on subaddress comparator-i;ascii-numeric relational regex imap4flags copy includ e variables body enotify environment mailbox date ihave protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 172.17.1.3 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 172.17.1.3 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = References: <506F239D.6090007@luigirosa.com> <506FE780.9000900@luigirosa.com> <507001B6.2000704@wk-serv.de> Message-ID: <507044D3.3030309@puzzled.xs4all.nl> On 10/06/2012 12:02 PM, Patrick Westenberg wrote: > Hi Luigi, > > with regard to SSL my configuration is much more simple and it works > fine with IPv4 and IPv6. But you have of course to use a hostname > matching the certificates common name. You could add additional hostnames in the certificate by specifying them in SubjectAltName. I use that so my certificate works with both the public FQDN going over the Internet as well as the internal hostname when using a VPN or on the local LAN. Regards, Patrick From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 6 18:53:53 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 6 Oct 2012 17:53:53 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <98D34C84-B1F4-47B3-9145-06E262FC11D7@iki.fi> References: <53B237A0-3A44-47DC-B41A-82CB5D174254@iki.fi> <20120710224947.GA10641@daniel.localdomain> <20120801193209.GA9069@daniel.localdomain> <20120801202502.GA9951@daniel.localdomain> <98D34C84-B1F4-47B3-9145-06E262FC11D7@iki.fi> Message-ID: <20121006155353.GA11391@daniel.localdomain> Hi Timo and list members, Timo Sirainen wrote: > On 1.8.2012, at 23.25, Daniel Parthey wrote: > > >> The error is still the same "config permission denied" shown above? I found that also from my server and added a debug patch, but it hasn't crashed yet. Could you try the attached patch and getting a gdb backtrace from the resulting core file? (Or at least the raw backtrace - getting a core file might be tricky.) > > > > Running command on a four host setup with mailbox+director instance each: > > /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A > > > > dovecot: doveadm(username at example.org): Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(vmail) egid=123(vmail) missing +r perm: /var/run/dovecot/auth-userdb, we're not in group 122(dovecot), dir owned by 0:0 mode=0755) > > Ah, so the original patch helped! This is a different error. > > > srw-rw---- 1 dovecot dovecot 0 2012-07-11 18:35 auth-userdb > > Simplest solution now would be to make this world-rw, see the auth-userdb socket configuration in http://wiki2.dovecot.org/LDA#Virtual_users > > But I guess this should also be fixed by doveadm-server. Although I don't think this should be happening by default anyway. Maybe this is also solved by the http://hg.dovecot.org/dovecot-2.1/rev/476381017ec7 patch? I finally found time to update from 2.1.8 to 2.1.10 and change service auth-user db socket to default mode of 0666. Unfortunately, the issue is still not solved and I did not manage to get a gdb backtrace, since it does not crash or assert. Current configuration of both mailbox and director is attached. The error "Permission denied" from the mailbox logs is gone, but the director doveadm command: /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A still throws the error message: doveadm(username at example.org): Error: doveadm server failure doveadm: Error: Failed to iterate through some users Any idea what I could do in addition to making /var/run/dovecot/auth-userdb world-rw? Regards Daniel -- https://plus.google.com/103021802792276734820 -------------- next part -------------- # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-43-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = imapc_features = rfc822.size imapc_host = local-mailbox imapc_port = 18143 instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota stats mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota stats imap_quota imap_stats } protocol lmtp { mail_plugins = quota stats sieve } -------------- next part -------------- # 2.1.10: /etc/dovecot-director/dovecot-director.conf # OS: Linux 2.6.32-43-server x86_64 Ubuntu 10.04.4 LTS auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot-director deliver_log_format = director: deliver: msgid=%m from=%f: %$ director_doveadm_port = 20000 director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_user_expire = 2 days disable_plaintext_auth = no doveadm_password = doveadm_proxy_port = 19000 instance_name = dovecot-director lmtp_proxy = yes login_greeting = Mail Balancer login_log_format = director: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "director: mail: %s(%u): " mail_max_userip_connections = 20 mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /etc/dovecot-director/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { executable = doveadm-server director inet_listener doveadm-server { port = 20000 } } service imap-login { executable = imap-login director inet_listener imap { port = 20143 } inet_listener imaps { port = 20993 ssl = yes } } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { address = * port = 20024 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 20200 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 20110 } inet_listener pop3s { port = 20995 ssl = yes } } ssl_cert = References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> Message-ID: <1349543117.2086.10.camel@worklian> Hi Mik, > > Also how do you understand this sentence > "# Don't use mmap() at all. This is required if you store indexes to shared > # filesystems (NFS or clustered filesystem) or for some operating systems > # which use a separate cache for mmap, such as OpenBSD. > mmap_disable = yes" > I've read it 10 times, and I don't know if this should be set to yes or no (probably because my english is not perfect). > My operating system is OpenBSD and I don't share NFS or cluster filesystems. Well, your english is monumentally better than my second language (if you could even say I have one), so good on you... I interpret this sentence as an if statement: if [[ (using NFS||Cluster) == true || (using OS w separate cache for mmap, such as OpenBSD) == true ]]; then setting is required (set to yes/true) fi > > Thank you -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca From marc at perkel.com Sat Oct 6 22:08:12 2012 From: marc at perkel.com (Marc Perkel) Date: Sat, 06 Oct 2012 12:08:12 -0700 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: References: <506D1345.9070406@perkel.com> Message-ID: <5070819C.40104@perkel.com> On 10/4/2012 7:20 AM, Timo Sirainen wrote: > On 4.10.2012, at 7.40, Marc Perkel wrote: > >> I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? > Write a plugin. > I have had some luck using an external script to generate the subscriptions files. From p.heinlein at heinlein-support.de Sun Oct 7 00:32:56 2012 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Sat, 06 Oct 2012 23:32:56 +0200 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files Message-ID: <5070A388.8070205@heinlein-support.de> Several times we already had the problems, that accounts with more the 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if vsize_limit of 750 MB is set. In this case, the lmtpd-process haven't been able to allocate more memory to read/write/update the index-files and crashed (and the index-files become corrupted at the end.) [Please -- don't discuss about the need of INBOXes with 1.7 million (unread) e-mails (don't discuss that with ME. Personally, I agree, that there's NO need for that...).] But: We also noticed accounts with ~ 300.000 e-Mails running out of memory in the same situations. This happends, if the subject is very large (subject or some other header attributes). And: We've been able to reproduce out-of-memory-Problems with just 13.000 e-mails with VERY long subjects (e.g.: network monitoring status informations), even with a vsize_limit of 750 MB (which is already very much). 13.000 e-mails isn't very much. And it's easy to inject several thousands of prepared e-mails. Having many mails for accounts with huge (and broken) index-files slows down the delivery rate VERY much and increases the need for memory and cpu resources and I/O very much. So: This could be used for a very easy to do denial-of-service attac against Dovecot-based mailservers. I don't have a clear solution for that, Dovecot needs the subject information in its index files. But it looks like, it isn't a good idea to put the whole subject into the index. Maybe it's better/necessary to use just the first 50-70 characters for that and to keep the rest away from the index? I think I would prefer that even if that means, that accessing those folders with "special" e-mails will become slower because Dovecot has to get those informations directly from the e-mail. This performance issue is just a problem for the user. But crashing lmtpd-processes and lowering the delivery rate is a *real* problem for the whole IMAP-cluster. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From slitt at troubleshooters.com Sun Oct 7 02:44:24 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Sat, 6 Oct 2012 19:44:24 -0400 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <5070A388.8070205@heinlein-support.de> References: <5070A388.8070205@heinlein-support.de> Message-ID: <20121006194424.47f7f80b@mydesk> On Sat, 06 Oct 2012 23:32:56 +0200, Peer Heinlein said: > > Several times we already had the problems, that accounts with more the > 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if > vsize_limit of 750 MB is set. > > In this case, the lmtpd-process haven't been able to allocate more > memory to read/write/update the index-files and crashed (and the > index-files become corrupted at the end.) > > [Please -- don't discuss about the need of INBOXes with 1.7 million > (unread) e-mails (don't discuss that with ME. Personally, I agree, > that there's NO need for that...).] > > But: We also noticed accounts with ~ 300.000 e-Mails running out of > memory in the same situations. This happends, if the subject is very > large (subject or some other header attributes). > > And: We've been able to reproduce out-of-memory-Problems with just > 13.000 e-mails with VERY long subjects (e.g.: network monitoring > status informations), even with a vsize_limit of 750 MB (which is > already very much). > > 13.000 e-mails isn't very much. And it's easy to inject several > thousands of prepared e-mails. > > Having many mails for accounts with huge (and broken) index-files > slows down the delivery rate VERY much and increases the need for > memory and cpu resources and I/O very much. > > So: This could be used for a very easy to do denial-of-service attac > against Dovecot-based mailservers. > > I don't have a clear solution for that, Dovecot needs the subject > information in its index files. But it looks like, it isn't a good > idea to put the whole subject into the index. Maybe it's > better/necessary to use just the first 50-70 characters for that and > to keep the rest away from the index? > > I think I would prefer that even if that means, that accessing those > folders with "special" e-mails will become slower because Dovecot has > to get those informations directly from the e-mail. > > This performance issue is just a problem for the user. > > But crashing lmtpd-processes and lowering the delivery rate is a > *real* problem for the whole IMAP-cluster. > > Peer While the real solution is being decided, can I avoid this possible DOS attack by using procmail to /dev/null anything with more than a 256 byte subject, before it ever gets to Dovecot IMAP? Thanks SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From mikydevel at yahoo.fr Sun Oct 7 12:36:59 2012 From: mikydevel at yahoo.fr (Mik J) Date: Sun, 7 Oct 2012 10:36:59 +0100 (BST) Subject: [Dovecot] Dovecot configuration and question about IP trusted In-Reply-To: <1349543117.2086.10.camel@worklian> References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349543117.2086.10.camel@worklian> Message-ID: <1349602619.92555.YahooMailNeo@web28801.mail.ir2.yahoo.com> > De?: Bob Miller > Hi Mik, > >> >> Also how do you understand this sentence >> "# Don't use mmap() at all. This is required if you store indexes > to shared >> # filesystems (NFS or clustered filesystem) or for some operating systems >> # which use a separate cache for mmap, such as OpenBSD. >> mmap_disable = yes" >> I've read it 10 times, and I don't know if this should be set to > yes or no (probably because my english is not perfect). >> My operating system is OpenBSD and I don't share NFS or cluster > filesystems. > > Well, your english is monumentally better than my second language (if > you could even say I have one), so good on you... > > I interpret this sentence as an if statement: > > if [[ (using NFS||Cluster) == true || (using OS w separate cache for > mmap, such as OpenBSD) == true ]]; then > ??? setting is required (set to yes/true) > fi Thank you for your answers. Have a nice week end From marc at perkel.com Sun Oct 7 22:47:44 2012 From: marc at perkel.com (Marc Perkel) Date: Sun, 07 Oct 2012 12:47:44 -0700 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? Message-ID: <5071DC60.8090108@perkel.com> Here's what I'm trying to do. I have a spam filtering operation as a front end for other servers. I've created a virtual server for spam storage where the user will be able to log in using squirrelmail/dovecot to review and release their spam. The email is stored in maildir format. Piecing it together I can use squirrelmail to pipe the email into something so that if a use finds a false positive they can hit the "release" button and the message is sent on to the recipient. Squirrelmail sends the message as you would receive it as stored in maildir format. I need to take this format and translate it to send it on to the user. Wondering what is the best way to do that. Something that translated it into bsmtp format would be great. I'm sure thare must be something out there. I just haven't found it yet. Thanks in advance. From robert at schetterer.org Sun Oct 7 23:48:10 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 07 Oct 2012 22:48:10 +0200 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? In-Reply-To: <5071DC60.8090108@perkel.com> References: <5071DC60.8090108@perkel.com> Message-ID: <5071EA8A.3060605@schetterer.org> Am 07.10.2012 21:47, schrieb Marc Perkel: > Here's what I'm trying to do. I have a spam filtering operation as a > front end for other servers. I've created a virtual server for spam > storage where the user will be able to log in using squirrelmail/dovecot > to review and release their spam. The email is stored in maildir format. > > Piecing it together I can use squirrelmail to pipe the email into > something so that if a use finds a false positive they can hit the > "release" button and the message is sent on to the recipient. > Squirrelmail sends the message as you would receive it as stored in > maildir format. > > I need to take this format and translate it to send it on to the user. > > Wondering what is the best way to do that. Something that translated it > into bsmtp format would be great. I'm sure thare must be something out > there. I just haven't found it yet. > > Thanks in advance. > sorry my hard words ,thats enorm complicated the whole idea sounds broken somehow why not reject spam in smtp income level i.e with clamav-milter and sanesecurity antispam signatures and spamass-milter reject all mail tagged over i.e level 15 for the rest ( which will be quite low ) use i.e some quarantaine feature amavis etc if users should manage it ie http://www.maiamailguard.com/maia/wiki/AboutMaia or equal may a good idea and i am quite sure there are some other well done projects out there which doing equal stuff -- Best Regards MfG Robert Schetterer From tss at iki.fi Mon Oct 8 03:11:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 8 Oct 2012 03:11:15 +0300 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <5070A388.8070205@heinlein-support.de> References: <5070A388.8070205@heinlein-support.de> Message-ID: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> On 7.10.2012, at 0.32, Peer Heinlein wrote: > Several times we already had the problems, that accounts with more the > 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if > vsize_limit of 750 MB is set. > > In this case, the lmtpd-process haven't been able to allocate more > memory to read/write/update the index-files and crashed (and the > index-files become corrupted at the end.) I don't think dovecot.index file is much of a problem. With 1M mails it usually only takes something like 8-32 MB of memory depending on what mailbox format is used. dovecot.index.log file doesn't depend on the mailbox size at all. The main problem is dovecot.index.cache file. I've thought about the cache file problems earlier also, but it's a bit difficult to figure out the best solution for it. And since nobody had actually complained about it, I hadn't really done anything about it. Also I hadn't previously thought of LMTP/LDA processes crashing because of it, that's a bigger problem than IMAP process crashing. Although I think you're getting a lot more of "mmap(dovecot.index.cache) failed: Out of memory" errors than crashes for large mailboxes? So, subproblems related to this: 1. Filling out dovecot.index.cache too easily. A rather simple possibility that would catch all the possible ways would be to limit the max. size of a single message's cache entry to X kilobytes (64?). If it becomes larger, it's simply not written to the cache file. 2. Filling out memory too easily. If a long header is wanted to be cached or used for other purposes (e.g. Message-ID), it's still fully read into memory. Add some reasonable limit to max. length of a single header. Can't be too small, because some headers are legitimately pretty long (DKIM and such). Maybe something like 10kB would be safe enough for everyone? 3. If existing dovecot.index.cache is larger than X MB, shrink it first below X. Shrinking could begin with trying to do it the nice way of removing only unneeded data, but if that fails it could forcibly just remove some old messages. The X would have to be related to the process's VSZ limit. 4. Dovecot currently doesn't close index files immediately when mailbox is closed, because it's thinking that IMAP clients might reopen the index soon anyway. Max 3 indexes can be kept open, so 3x already different very large indexes can be too much. I'm not sure if this is actually useful at all. Maybe I should disable it for LMTP, or maybe just remove it completely. The 3. part is what I like changing the least. An alternative solution would be to just not map the entire cache file into memory all at once. The code was actually originally designed to do just that, but munmap()ing + mmap()ing again wasn't very efficient. But for LMTP there's really no need to map the whole file. All it really wants is to read a couple of header records and then append to the file. Maybe it could use an alternative code path that would simply do that instead of mmap()ing anything. It wouldn't solve it for IMAP though. > I don't have a clear solution for that, Dovecot needs the subject > information in its index files. But it looks like, it isn't a good idea > to put the whole subject into the index. Maybe it's better/necessary to > use just the first 50-70 characters for that and to keep the rest away > from the index? 50-70 is way too little. The cached subject gets sent to the IMAP client. I think 200 bytes would be minimum and 1000 would be something I could probably even hardcode. But anyway, subject isn't the only way to trigger this and 1000 bytes is too low for some headers. From tss at iki.fi Mon Oct 8 04:45:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 08 Oct 2012 04:45:14 +0300 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> References: <5070A388.8070205@heinlein-support.de> <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> Message-ID: <1349660714.13571.75.camel@hurina> On Mon, 2012-10-08 at 03:11 +0300, Timo Sirainen wrote: > The 3. part is what I like changing the least. An alternative solution > would be to just not map the entire cache file into memory all at > once. The code was actually originally designed to do just that, but > munmap()ing + mmap()ing again wasn't very efficient. But for LMTP > there's really no need to map the whole file. All it really wants is > to read a couple of header records and then append to the file. Maybe > it could use an alternative code path that would simply do that > instead of mmap()ing anything. It wouldn't solve it for IMAP though. Attached patch changes LMTP/LDA to not mmap() the target mailbox's cache file. I did a few quick tests and it seems to work. I'll probably commit it to 2.1 hg after some more tests. -------------- next part -------------- A non-text attachment was scrubbed... Name: index-saveonly.diff Type: text/x-patch Size: 17995 bytes Desc: not available URL: From stan at hardwarefreak.com Mon Oct 8 09:11:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 08 Oct 2012 01:11:50 -0500 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? In-Reply-To: <5071DC60.8090108@perkel.com> References: <5071DC60.8090108@perkel.com> Message-ID: <50726EA6.4030702@hardwarefreak.com> This request for assistance is a train wreck, with cars strewn everywhere, chaos ensuing, the carnage preventing everyone from being able to see what's actually going on... On 10/7/2012 2:47 PM, Marc Perkel wrote: > Here's what I'm trying to do. I have a spam filtering operation as a > front end for other servers. I've created a virtual server for spam > storage where the user will be able to log in using squirrelmail/dovecot > to review and release their spam. The email is stored in maildir format. So you're trying to somewhat duplicate the functionality of a Barracuda or other AS gateway appliance, with vanilla SM and Dovecot with very little modification. Good luck with this. You're going to need to write a pretty complex shell or perl script to do most of the work, and call it from the SM "release button" routine. > Piecing it together I can use squirrelmail to pipe the email into > something so that if a use finds a false positive they can hit the > "release" button and the message is sent on to the recipient. > Squirrelmail sends the message as you would receive it as stored in > maildir format. SM only sends messages via SMTP submitted to the SMTP relay host specified in the config file, or via dropping to the local MTA via stdin/out. Maildir is a mail storage directory and file format protocol, not a transmission protocol. It's physically impossible to "send" a msg in maildir format, or mbox, or dbox, etc. The format of the SMTP headers and message body is plain text, possibly with MIME encoding. So what you really mean is you need a way to read a maildir message file, scrape the recipient address, strip all of the AS headers you've inserted, drop this 'new' message to the MTA, which sends the message to the recipient. When it arrives it appears never to have been molested by your AS software, with only the proper headers and original body. Then your script needs to check the log for successful delivery (250), then send the proper commands to dovecot to log into the account as the user and delete the message. I can't begin to estimate the amount of coding and testing required here, but it will be high. > I need to take this format and translate it to send it on to the user. Translate it? I've never used maildir, but I can't imagine the on disk message file contents need "translating". See above. > Wondering what is the best way to do that. Something that translated it > into bsmtp format would be great. I'm sure thare must be something out > there. I just haven't found it yet. BSMTP isn't a file format. It's a simple Mail User Agent with some unique capabilities. Given you already have an MTA on the host, why would you need BSMTP to submit or deliver the msg? You simply need to learn the proper commands to submit mail to your local MTA. With Postfix you'd use postdrop or the sendmail compatibility command which use stdin to read the message. My no BS assessment and recommendation are that you're likely in way over your head here (and don't realize it yet), and you need to focus your efforts on locating an integrated FOSS solution, or even a commercial solution, to fit your needs. I seem to understand better than you what is required to pull this off, and I wouldn't touch this project with a 50 ft pole. I simply don't have the coding skills for this. -- Stan From stan at hardwarefreak.com Mon Oct 8 10:44:57 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 08 Oct 2012 02:44:57 -0500 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> References: <5070A388.8070205@heinlein-support.de> <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> Message-ID: <50728479.6070003@hardwarefreak.com> On 10/7/2012 7:11 PM, Timo Sirainen wrote: > I don't think dovecot.index file is much of a problem. With 1M mails it usually only takes something like 8-32 MB of memory depending on what mailbox format is used. dovecot.index.log file doesn't depend on the mailbox size at all. The main problem is dovecot.index.cache file. > > I've thought about the cache file problems earlier also, but it's a bit difficult to figure out the best solution for it. And since nobody had actually complained about it, I hadn't really done anything about it. Also I hadn't previously thought of LMTP/LDA processes crashing because of it, that's a bigger problem than IMAP process crashing. Although I think you're getting a lot more of "mmap(dovecot.index.cache) failed: Out of memory" errors than crashes for large mailboxes? > > So, subproblems related to this: > > 1. Filling out dovecot.index.cache too easily. A rather simple possibility that would catch all the possible ways would be to limit the max. size of a single message's cache entry to X kilobytes (64?). If it becomes larger, it's simply not written to the cache file. > > 2. Filling out memory too easily. If a long header is wanted to be cached or used for other purposes (e.g. Message-ID), it's still fully read into memory. Add some reasonable limit to max. length of a single header. Can't be too small, because some headers are legitimately pretty long (DKIM and such). Maybe something like 10kB would be safe enough for everyone? > > 3. If existing dovecot.index.cache is larger than X MB, shrink it first below X. Shrinking could begin with trying to do it the nice way of removing only unneeded data, but if that fails it could forcibly just remove some old messages. The X would have to be related to the process's VSZ limit. > > 4. Dovecot currently doesn't close index files immediately when mailbox is closed, because it's thinking that IMAP clients might reopen the index soon anyway. Max 3 indexes can be kept open, so 3x already different very large indexes can be too much. I'm not sure if this is actually useful at all. Maybe I should disable it for LMTP, or maybe just remove it completely. > > The 3. part is what I like changing the least. An alternative solution would be to just not map the entire cache file into memory all at once. The code was actually originally designed to do just that, but munmap()ing + mmap()ing again wasn't very efficient. But for LMTP there's really no need to map the whole file. All it really wants is to read a couple of header records and then append to the file. Maybe it could use an alternative code path that would simply do that instead of mmap()ing anything. It wouldn't solve it for IMAP though. > 50-70 is way too little. The cached subject gets sent to the IMAP client. I think 200 bytes would be minimum and 1000 would be something I could probably even hardcode. But anyway, subject isn't the only way to trigger this and 1000 bytes is too low for some headers. Nearly all mail servers have two resources of interest here in great excess: CPU cycles, and cache/RAM b/w, due to multicore CPUs and 2-4 memory channels per socket. The two bottlenecks are IO bandwidth/latency, and, for many, RAM capacity. So let's take advantage of both the strengths and weaknesses of our hardware to possibly address the above issue. What happens if we insert a subroutine to compress/decompress each field in the cache array files individually, in real time? You should still be able to mmap the files. The individual array fields and total cache file sizes would be much smaller on disk and in memory. Any cache file contents mapped to memory, that aren't currently being used, are stored compressed in memory, directly addressing the problem in this thread. When a field is needed we decompress it on the fly after reading it from memory. This should be very fast as the fields are relatively small. When it's written out we compress on the fly. With each field stored compressed on disk, not only is file size decreased, but more importantly, each read/write moves more data per physical IO. So not only are increasing storage capacity, we're also decreasing IOPS. It would be preferable to do this de/compression in kernel rather than user space, but I don't think that's a real option. However, libz and libbz2 are pretty fast and small, and the code easily fits in CPU cache. Combined with the massive L1/L2/L3 and RAM b/w of modern systems, execution in user space should still be very fast, and not noticeably degrade performance. I'm not a programmer, so I have no idea if this is even plausible, or possible. But if it is, it seems worth exploring, as it would seem to benefit Dovecot performance in multiples areas, and possibly solve this, and other current/future memory capacity and/or performance related problems. -- Stan From benedetto.vassallo at unipa.it Mon Oct 8 15:44:05 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Mon, 08 Oct 2012 14:44:05 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <506DA526.4020606@Media-Brokers.com> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <506DA526.4020606@Media-Brokers.com> Message-ID: <20121008144405.21350fs5aq2vwl91@webmail.unipa.it> Def. Quota Charles Marcus : > On 2012-10-04 9:00 AM, Benedetto Vassallo > wrote: >> Hello list, >> Excuse me for my poor english. >> I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. > > If you are going to update, why ohy why update to an outdated version? > > Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... > Hello, I upgraded to 2.1.10 but still dont't have hardlinks working. May I have something else to do to make them working? Thanks. -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From tibby at tibby.hu Mon Oct 8 16:49:33 2012 From: tibby at tibby.hu (Tibby) Date: Mon, 8 Oct 2012 15:49:33 +0200 (CEST) Subject: [Dovecot] another mysql quota problem Message-ID: <1182860249.2592.1349704173895.JavaMail.root@tibby.hu> Hello! I am using dovecot 1.2.15 on Debian 6 I have Postfix configured with mysql, and also dovecot is auth-ing form mysql users table. The users table has username password quota stored. When I set qouta to whatever number, it doesnt even show in dovecot. telnet localhost 143 a login username at domain.tld password a getquotaroot inbox * QUOTAROOT "INBOX" a OK Getquotaroot completed. Basicly it doesn't show any kind of quota. in the /etc/dovecot/dovecot-sql.conf: driver = mysql connect = host=127.0.0.1 dbname=mail user=USERNAME password=PASSWORD default_pass_scheme = CRYPT password_query = SELECT email as user, password FROM users WHERE email='%u'; user_query = SELECT CONCAT(('/home/vmail/'), SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1),'/') AS home, 5000 AS uid, 5000 AS gid, CONCAT('maildir:storage=', floor(quota/1024)) AS quota FROM users WHERE email='%u'; Why it doesn't get the quota out from the mail DB's users table? From roundcube222 at alaadin.org Mon Oct 8 18:12:58 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Mon, 08 Oct 2012 18:12:58 +0300 Subject: [Dovecot] Dovecot Hangs while mutile users download mail for same account using pop3 Message-ID: <793760c2702e89acc526a66c0b543293@Coptics.org> Hello, I have a weird problem in dovecot, Dovecot Hangs while multiple users download mail for same account using pop3 Three persons use 1 same email , and three of them use outlook express to check That specific mail .. Also some times one of the three users check the mail for this Account using imap (squirrel mail) And here comes the problem. suddenly all the three users are unable to check the mail And when the outlook express connect to the pop3 server, they are unable to disconnect And the outlook express keep asking "the server is not responding for 60 mins would you like to wait ? i checked /var/log/maillog, i can see the following errors dovecot: pop3(sales): Disconnected: Storage error during logout. to=, orig_to=, relay=local, delay=357, delays=338/0.01/0/19, dsn=4.2.0, status=deferred (cannot update mailbox /var/mail/sales for user sales. unable to lock for exclusive access: Resource temporarily unavailable) When i used lsof | grep sales pop3 4278 sales 10u REG 8,3 22897673 12615705 /var/spool/mail/sales i went to /home/sales/mail/.imap/INBOX , but i didnot find any lock files -rw------- 1 sales sales 944 Oct 7 13:35 dovecot.index -rw------- 1 sales sales 49152 Oct 7 16:57 dovecot.index.cache -rw------- 1 sales sales 14044 Oct 7 16:57 dovecot.index.log 1- So what is exactly the problem? is the problem that three users are checking the same exact mail ? 2- is the problem that 2 users checking mail using pop3 and the third one checking it using imap insame time ? 3- is dovecot locking the inbox file while one of the users getting the mail ? 4- What is the cause of the problem ? Please help Regards Robert JR From marc at perkel.com Mon Oct 8 19:23:27 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 09:23:27 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? Message-ID: <5072FDFF.8050402@perkel.com> I'm trying to grasp the namespace and prefix stuff and looking for a good tutorial page that explains what it is and how to use it. Thanks in advance From robert at schetterer.org Mon Oct 8 20:59:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 19:59:34 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5072FDFF.8050402@perkel.com> References: <5072FDFF.8050402@perkel.com> Message-ID: <50731486.4080003@schetterer.org> Am 08.10.2012 18:23, schrieb Marc Perkel: > I'm trying to grasp the namespace and prefix stuff and looking for a > good tutorial page that explains what it is and how to use it. > > Thanks in advance > look http://wiki2.dovecot.org/Namespaces -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:09:00 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:09:00 -0700 Subject: [Dovecot] Master User Question Message-ID: <507316BC.1050008@perkel.com> OK - I'm trying to do weird stuff so rather not answer why I'm doing this. Trying to log in using a master user: user at example.com*master at master.com When debugging the master authenticates - but then it tries to authenticate user at example.com and it's not found. And - it is true that the user doesn't actually exist. What I want to do is allow it to log in without checking if the user exists, just on the credentials of the master. I'm not quite understanding what the login attempt for user at example.com is trying to do. Is it looking for more necessary information? Hope I asked this clearly enough. Thanks in advance. From marc at perkel.com Mon Oct 8 21:10:21 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:10:21 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <50731486.4080003@schetterer.org> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> Message-ID: <5073170D.2040508@perkel.com> On 10/8/2012 10:59 AM, Robert Schetterer wrote: > Am 08.10.2012 18:23, schrieb Marc Perkel: >> I'm trying to grasp the namespace and prefix stuff and looking for a >> good tutorial page that explains what it is and how to use it. >> >> Thanks in advance >> > look > > http://wiki2.dovecot.org/Namespaces > I've read that but it doesn't tell me what a prefix is or what a namespace is. I'm having a hard time grasping the overall concept. From robert at schetterer.org Mon Oct 8 21:26:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:26:14 +0200 Subject: [Dovecot] Master User Question In-Reply-To: <507316BC.1050008@perkel.com> References: <507316BC.1050008@perkel.com> Message-ID: <50731AC6.4060707@schetterer.org> Am 08.10.2012 20:09, schrieb Marc Perkel: > OK - I'm trying to do weird stuff so rather not answer why I'm doing this. > > Trying to log in using a master user: > > user at example.com*master at master.com > > When debugging the master authenticates - but then it tries to > authenticate user at example.com and it's not found. And - it is true that > the user doesn't actually exist. > > What I want to do is allow it to log in without checking if the user > exists, just on the credentials of the master. i dont understand why login should work with masteruser to user , when user does not exist, would you like to get the user created on the fly by masteruser login etc ? > > I'm not quite understanding what the login attempt for user at example.com > is trying to do. Is it looking for more necessary information? > > Hope I asked this clearly enough. Thanks in advance. > > you may manipulate your master query in some magic way http://wiki2.dovecot.org/Authentication/MasterUsers -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Oct 8 21:31:52 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:31:52 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5073170D.2040508@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> Message-ID: <50731C18.5060009@schetterer.org> Am 08.10.2012 20:10, schrieb Marc Perkel: > > On 10/8/2012 10:59 AM, Robert Schetterer wrote: >> Am 08.10.2012 18:23, schrieb Marc Perkel: >>> I'm trying to grasp the namespace and prefix stuff and looking for a >>> good tutorial page that explains what it is and how to use it. >>> >>> Thanks in advance >>> >> look >> >> http://wiki2.dovecot.org/Namespaces >> > > I've read that but it doesn't tell me what a prefix is or what a > namespace is. I'm having a hard time grasping the overall concept. > lookink to this http://www.ietf.org/rfc/rfc2342.txt ? -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:32:51 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:32:51 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731AC6.4060707@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> Message-ID: <50731C53.2050309@perkel.com> On 10/8/2012 11:26 AM, Robert Schetterer wrote: > Am 08.10.2012 20:09, schrieb Marc Perkel: >> OK - I'm trying to do weird stuff so rather not answer why I'm doing this. >> >> Trying to log in using a master user: >> >> user at example.com*master at master.com >> >> When debugging the master authenticates - but then it tries to >> authenticate user at example.com and it's not found. And - it is true that >> the user doesn't actually exist. >> >> What I want to do is allow it to log in without checking if the user >> exists, just on the credentials of the master. > i dont understand why login should work with masteruser to user , when > user does not exist, would you like to get the user created on the fly > by masteruser login etc ? The directory structure for user at example.com exists. Just not in the regular mysql database. If I create a fake passdb system it works: passdb { driver = static args = nopassword=y } However - it leaves the system wide open for anyone. Basically - only master users are going to log in and manage stored spam for regular users. The master user will be able to spoof being the regular user to review stored spam. From slusarz at curecanti.org Mon Oct 8 21:28:18 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 08 Oct 2012 12:28:18 -0600 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5073170D.2040508@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> Message-ID: <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> Quoting Marc Perkel : > On 10/8/2012 10:59 AM, Robert Schetterer wrote: >> Am 08.10.2012 18:23, schrieb Marc Perkel: >>> I'm trying to grasp the namespace and prefix stuff and looking for a >>> good tutorial page that explains what it is and how to use it. >>> >>> Thanks in advance >>> >> look >> >> http://wiki2.dovecot.org/Namespaces >> > > I've read that but it doesn't tell me what a prefix is or what a > namespace is. I'm having a hard time grasping the overall concept. Try reading the defining RFC document itself: http://tools.ietf.org/html/rfc2342 michael From marc at perkel.com Mon Oct 8 21:42:02 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:42:02 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> Message-ID: <50731E7A.9020001@perkel.com> On 10/8/2012 11:28 AM, Michael M Slusarz wrote: > Quoting Marc Perkel : > >> On 10/8/2012 10:59 AM, Robert Schetterer wrote: >>> Am 08.10.2012 18:23, schrieb Marc Perkel: >>>> I'm trying to grasp the namespace and prefix stuff and looking for a >>>> good tutorial page that explains what it is and how to use it. >>>> >>>> Thanks in advance >>>> >>> look >>> >>> http://wiki2.dovecot.org/Namespaces >>> >> >> I've read that but it doesn't tell me what a prefix is or what a >> namespace is. I'm having a hard time grasping the overall concept. > > Try reading the defining RFC document itself: > > http://tools.ietf.org/html/rfc2342 > > michael > I went there and maybe I'm just stupid. I don't understand what a prefix is. I'm trying to grasp the whole process. From robert at schetterer.org Mon Oct 8 21:43:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:43:20 +0200 Subject: [Dovecot] Master User Question In-Reply-To: <50731C53.2050309@perkel.com> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> Message-ID: <50731EC8.6010101@schetterer.org> Am 08.10.2012 20:32, schrieb Marc Perkel: > > On 10/8/2012 11:26 AM, Robert Schetterer wrote: >> Am 08.10.2012 20:09, schrieb Marc Perkel: >>> OK - I'm trying to do weird stuff so rather not answer why I'm doing >>> this. >>> >>> Trying to log in using a master user: >>> >>> user at example.com*master at master.com >>> >>> When debugging the master authenticates - but then it tries to >>> authenticate user at example.com and it's not found. And - it is true that >>> the user doesn't actually exist. >>> >>> What I want to do is allow it to log in without checking if the user >>> exists, just on the credentials of the master. >> i dont understand why login should work with masteruser to user , when >> user does not exist, would you like to get the user created on the fly >> by masteruser login etc ? > > The directory structure for user at example.com exists. Just not in the > regular mysql database. If I create a fake passdb system it works: > > passdb { > driver = static > args = nopassword=y > } > > However - it leaves the system wide open for anyone. Basically - only > master users are going to log in and manage stored spam for regular > users. The master user will be able to spoof being the regular user to > review stored spam. > > whats the problem in simply create the user mailbox? i.e lda etc may do it on the fly be recieving a mail http://wiki2.dovecot.org/LDA -m : Destination mailbox (default is INBOX). If the mailbox doesn't exist, it will not be created (unless the lda_mailbox_autocreate setting is set to yes). If message couldn't be saved to the mailbox for any reason, it's delivered to INBOX instead. -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Oct 8 21:49:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:49:33 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <50731E7A.9020001@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> <50731E7A.9020001@perkel.com> Message-ID: <5073203D.7060801@schetterer.org> Am 08.10.2012 20:42, schrieb Marc Perkel: > > On 10/8/2012 11:28 AM, Michael M Slusarz wrote: >> Quoting Marc Perkel : >> >>> On 10/8/2012 10:59 AM, Robert Schetterer wrote: >>>> Am 08.10.2012 18:23, schrieb Marc Perkel: >>>>> I'm trying to grasp the namespace and prefix stuff and looking for a >>>>> good tutorial page that explains what it is and how to use it. >>>>> >>>>> Thanks in advance >>>>> >>>> look >>>> >>>> http://wiki2.dovecot.org/Namespaces >>>> >>> >>> I've read that but it doesn't tell me what a prefix is or what a >>> namespace is. I'm having a hard time grasping the overall concept. >> >> Try reading the defining RFC document itself: >> >> http://tools.ietf.org/html/rfc2342 >> >> michael >> > > I went there and maybe I'm just stupid. I don't understand what a prefix > is. I'm trying to grasp the whole process. > > try http://wiki.dovecot.org/Design/Storage/MailNamespace -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:52:45 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:52:45 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731EC8.6010101@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> <50731EC8.6010101@schetterer.org> Message-ID: <507320FD.2000201@perkel.com> On 10/8/2012 11:43 AM, Robert Schetterer wrote: > Am 08.10.2012 20:32, schrieb Marc Perkel: >> On 10/8/2012 11:26 AM, Robert Schetterer wrote: >>> Am 08.10.2012 20:09, schrieb Marc Perkel: >>>> OK - I'm trying to do weird stuff so rather not answer why I'm doing >>>> this. >>>> >>>> Trying to log in using a master user: >>>> >>>> user at example.com*master at master.com >>>> >>>> When debugging the master authenticates - but then it tries to >>>> authenticate user at example.com and it's not found. And - it is true that >>>> the user doesn't actually exist. >>>> >>>> What I want to do is allow it to log in without checking if the user >>>> exists, just on the credentials of the master. >>> i dont understand why login should work with masteruser to user , when >>> user does not exist, would you like to get the user created on the fly >>> by masteruser login etc ? >> The directory structure for user at example.com exists. Just not in the >> regular mysql database. If I create a fake passdb system it works: >> >> passdb { >> driver = static >> args = nopassword=y >> } >> >> However - it leaves the system wide open for anyone. Basically - only >> master users are going to log in and manage stored spam for regular >> users. The master user will be able to spoof being the regular user to >> review stored spam. >> >> > whats the problem in simply create the user mailbox? > i.e lda etc may do it on the fly be recieving a mail > > http://wiki2.dovecot.org/LDA > -m : Destination mailbox (default is INBOX). If the mailbox > doesn't exist, it will not be created (unless the lda_mailbox_autocreate > setting is set to yes). If message couldn't be saved to the mailbox for > any reason, it's delivered to INBOX instead. Maybe there's some way to have more than one mail_location? I have this - and it's confusing: maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs the /fakedir directory is externally built and it has symlinks pointing onto the /email directory where the email is stored. If I could set up a passdb and specify a different mail_location that might solve my problem, at least one of them. So close - yet so far away. From marc at perkel.com Mon Oct 8 22:07:28 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 12:07:28 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731EC8.6010101@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> <50731EC8.6010101@schetterer.org> Message-ID: <50732470.300@perkel.com> Making some progress = this seems to work. passdb { driver = static args = mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } From calestyo at scientia.net Tue Oct 9 00:37:19 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 08 Oct 2012 23:37:19 +0200 Subject: [Dovecot] some questions on dovecot or rather a mail system setup Message-ID: <1349732239.3344.62.camel@fermat.scientia.net> Hi folks. Perhaps you find some time to look into this,... if you think I should better direct this do some dovecot mailing list,.. just tell :) I'm trying to plan my mail system and would have some questions. The overall idea is about the following: - There is a (internet) server, which is the MTA (which will be postfix) and imap and/or pop3 server (which shall be, guess, dovecot!). - Any spam filtering, virus-scanning, mail filtering happens on that server. - I have not yet decided on whether to use maildrop for this or Sieve. Maildrop is nice, but has one major deficiency which I don't know how to work around. - A few clients (that means I won't serve 100 of users) connect to that server via imap and should see all mails, etc. already in some fancy sorted hierarchy (that means filtered into different directories). So far nothing complicated. But now... - Apart from spam, I never delete mail; and because I'm subscribed to many lists, I get a lot of mail. - Storage on my server is limited and it's located somewhere at my ISP, so I generally do not trust it with respect to safety... For both reasons, I want the canonical archive of all mail to be at home at some local server. - The local server should also be an imap server, so that I can access the archive from may computers at home. - The local server won't be available from the internet. - The local archive should have the same folder hierarchy as the internet server (I'd prefer not to filter twice). Mail Flow: - I generally want to have _all_ mail (which is not sorted out because of being spam) to be archived at the local server. - But(!) I want to selectively keep (in addition) mail at the internet server. For example I may want to select the folder that contains all mail form some friend to be kept online completely. But I may want to decide that mailinglists keep only the last 10 days and/or 1000 messages of mail. - The idea is, that the local server regularly (when it is online/running) catches new mail from the internet server... and stores it in the archive. - So apart from new mail that has not yet been read, that local archive contains always all mails that are also on the internet server... the later may contain (for specific directories) the same, or just parts of. - The MUAs will then have two imap accounts, one to the internet server and one to the local archive,... each one being usable, depending on where I am. 1) This is where my first problem arises: How can I implement that mail flow, especially: - How do I secure that all mail is read from the internet server (i.e. that nothing is "forgotten"? - How do I make sure that no mails are retrieved twice (or more)? A problem which I often had with pop, when the mail client crashed during sync? - Further it must be secured, that when I delete something on the internet server, it is NOT deleted on the local server (on the next mail-fetching).... this is why I don't use the word "sync". a) One stupid solution would be, that I duplicate all mail on the online server,... one part is for staying online, one part is for being fetched to the local archive. As soon as it was fetched... that copy gets removed (always). That solution would give a clean and secured separation of both? b) I don't think offlineimap or any other caching-like solution is the right thing... especially as one must always fear that such a cache may be accidentally wiped. Are there better solutions than (a)? 2) Problem would be already a refinement of a working solution for (1) (but obviously not when using (1).(a) ). When e.g. reply to or forward a mail using the online server,... and that mail had already been fetched,... can I make the flag synced? 3) Is dovecot suitable for the local server? - Ideally of course, I would use dovecot there, too... because that would mean one piece of software less to understand. - I couldn't sue maildir locally, because I loose just to much space to the block fragmentation. - I'd prefer not to use dbox (the thing that the indices are crucial scares me a bit off). a) When using mbox... is dovecot able to manage a really big folder hierarchy that basically ever keeps growing... with easily several 100k mails per folder... and that is in total already over 100GB? - I would prefer to have fast full text search. Does dovecot provide this? I was looking into database backed mail systems (again,... just for the local archive)... namely dbmail and archiveopteryx (are there other open source solutions?)... Not sure which of the two... or whether it's a good idea at all. I remember some dovecot wiki page that showed a comparison which said that both do not perfectly implement imap. Any suggestions with respect to that? Or is there even some SQL backend planned for dovecot? Thanks a lot, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From robert at schetterer.org Tue Oct 9 09:51:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 09 Oct 2012 08:51:18 +0200 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <1349732239.3344.62.camel@fermat.scientia.net> References: <1349732239.3344.62.camel@fermat.scientia.net> Message-ID: <5073C966.6030609@schetterer.org> Am 08.10.2012 23:37, schrieb Christoph Anton Mitterer: > > > - Apart from spam, I never delete mail; and because I'm subscribed to > many lists, I get a lot of mail. > > - Storage on my server is limited and it's located somewhere at my ISP, > so I generally do not trust it with respect to safety... > For both reasons, I want the canonical archive of all mail to be at home > at some local server. sorry you questioned very complex, try to ask more simple there are many tools which may help you bcc_copy with postfix imapsync rsync dsync getmail you may use filters too like sieve, maildrop , procmail etc at the end that should solve nearly all what you might goal its not that much a dovecot question, it more depends if you find that general layout which fits best to your ideas however there is no magical imap/pop3 server more flexibel to configure then dovecot, if your ideas dont work with it, your ideas are broken -- Best Regards MfG Robert Schetterer From stan at hardwarefreak.com Tue Oct 9 10:57:43 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 09 Oct 2012 02:57:43 -0500 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <1349732239.3344.62.camel@fermat.scientia.net> References: <1349732239.3344.62.camel@fermat.scientia.net> Message-ID: <5073D8F7.1060802@hardwarefreak.com> On 10/8/2012 4:37 PM, Christoph Anton Mitterer wrote: The proper way to accomplish your goals, or at least the big ones. > - I generally want to have _all_ mail (which is not sorted out because > of being spam) to be archived at the local server. http://www.postfix.org/postconf.5.html#always_bcc > - But(!) I want to selectively keep (in addition) mail at the internet > server. > For example I may want to select the folder that contains all mail form > some friend to be kept online completely. See above. > But I may want to decide that mailinglists keep only the last 10 days > and/or 1000 messages of mail. http://wiki2.dovecot.org/Plugins/Expire Does age based deletion, but not folder message count based. You must use your MUA, TBird, for the latter. It's far easier to configure this in TBird than in Dovecot config files. You seem like the type who wants flexibility so you can change things often, so use TBird to be happy here. > - The idea is, that the local server regularly (when it is > online/running) catches new mail from the internet server... and stores > it in the archive. This is not an option. The system must be up and connected to the internet 24x7x365. It must have an MX record associated and a valid domain, or a VPN tunnel and entries in both systems hosts files, along with a Postfix transport table, and other tweaks. http://www.postfix.org/transport.5.html If you refuse to run this "local server" 24x7x365 then you will have to use a fetchmail based solution, which will not work well, and whose configuration will prompt you to kill yourself. I cannot help you with any of that. > - So apart from new mail that has not yet been read, that local archive > contains always all mails that are also on the internet server... the > later may contain (for specific directories) the same, or just parts of. No. Mail arriving at the colo/VPS host is immediately sent to the always_bcc address, an address and mailbox on your home server. You will create a duplicate IMAP folder structure on the home server by hand in your MUA. Once this is completed you will write individual user sieve scripts that sort the mail into folder just as it is sorted on the colo/VPS server. Basically, home server Dovecot IMAP config is identical in structure to colo/VPS setup, only the mailbox account names differ. Folder tree, folders, sieve scripts identical, retention policy different. > - The MUAs will then have two imap accounts, one to the internet server > and one to the local archive,... each one being usable, depending on > where I am. Yep. > > > 1) This is where my first problem arises: > How can I implement that mail flow, especially: > - How do I secure that all mail is read from the internet server (i.e. > that nothing is "forgotten"? Done: always_bcc > - How do I make sure that no mails are retrieved twice (or more)? A > problem which I often had with pop, when the mail client crashed during > sync? Done: always_bcc > - Further it must be secured, that when I delete something on the > internet server, it is NOT deleted on the local server (on the next > mail-fetching).... this is why I don't use the word "sync". Done: always_bcc > a) One stupid solution would be, that I duplicate all mail on the online > server,... one part is for staying online, one part is for being fetched > to the local archive. Done: always_bcc And yes that is stupid. > As soon as it was fetched... that copy gets removed (always). > That solution would give a clean and secured separation of both? > b) I don't think offlineimap or any other caching-like solution is the > right thing... especially as one must always fear that such a cache may > be accidentally wiped. > > Are there better solutions than (a)? Yes. Already done: always_bcc > 2) Problem would be already a refinement of a working solution for (1) > (but obviously not when using (1).(a) ). > When e.g. reply to or forward a mail using the online server,... and > that mail had already been fetched,... can I make the flag synced? No. Your stated goal is that the local server is a mail archive put into service due to limited space on your colo/VPS server. An archive is an archive, not a secondary online server. It should only be accessed, read only, when you want to search and read an old message. And in fact, since this is an archive, you should implement the zlib plugin with dbox so all this archived mail is compressed in real time. Make up your mind. You can't have it both ways. I hear the iPhone5 can do anything automatically, no setup. Get one of those, problem solved. ;) > 3) Is dovecot suitable for the local server? Yes. Probably more than any other IMAP server. > - I couldn't sue maildir locally, because I loose just to much space to > the block fragmentation. Maildir causes the least filesystem fragmentation. You must be thinking of mbox, which causes heavy fragmentation due to constant appends past EOF. As I said you need dbox. One email per file, similar to maildir, but better integration and performance with Dovecot. > - I'd prefer not to use dbox (the thing that the indices are crucial > scares me a bit off). Are you designing/building this home server to be unreliable? Does it crash often? If so fix that problem and dbox is fine. If can't make it reliable use maildir which has expendable indexes. > a) When using mbox... is dovecot able to manage a really big folder > hierarchy that basically ever keeps growing... with easily several 100k > mails per folder... and that is in total already over 100GB? You have 100K emails in a single Dovecot mbox file? Or are you talking about an IMAP folder in TB that has no email in it, but many more IMAP folders whose combined email total is 100K? If you're worried about dbox index corruption, then you should be far more worried about mbox file corruption. With mbox files that large I'm surprised you've not hit it already. This would suggest that system is pretty stable. > - I would prefer to have fast full text search. Does dovecot provide > this? Yes. The problem with speed is two fold: 1. You must FTS often to keep the search indexes up to date. Wait a week between searches, after many new emails have been added to the IMAP folder, and your search crawls, as the file contents must be reindexed before the search starts. So you need to have a cron'd script that searches daily to keep the indexes up to date. 2. The mailbox file formats that best avoid fragmentation also have the slowest FTS times as the OS much open every file, 100K of them. If you use mbox or mdbox, you have far fewer files to open. mbox has the fastest FTS times of any format when indexes aren't fully up to date. It's also the fastest when updating the indexes. Your home server probably has a single SATA disk. mbox wins hands down for FTS due to very low IOPS load on the disk. The downside here is lack of good compression support--once you compress an mbox file you can't add new mail to it. This is where mdbox with compression comes in handy. With you 100K emails declaration, I think you're best served by mdbox with zlib compression. > I was looking into database backed mail systems (again,... just for the > local archive)... namely dbmail and archiveopteryx (are there other open > source solutions?)... > Not sure which of the two... or whether it's a good idea at all. > I remember some dovecot wiki page that showed a comparison which said > that both do not perfectly implement imap. > > Any suggestions with respect to that? If you're worried about fragmentation, or performance, I'd steer clear of a database driven mail store. Please, please, do not reply to each of my points here, and do not make this thread 100 replies. I'm not here to hold your hand. I don't have the time (nor patience) to engage in these lengthy emails. I gave you the architectural overview to build the correct solution to your problem. It's up to you to choose to use it or not, and if so, to do your own homework and self education, asking here only if something is unclear to you. In closing, you need real time bcc delivery which solves a ton of your mentioned problems. I'm not open to debating the merits of this. If you're not willing to meet the requirements for always_bcc, and you're determined to power the home server down most of the time, then you need assistance from someone else, as I simply have never used fetchmail, period, and have no idea if it can meet your needs. My guess is no, simply because, AFAIK, it doesn't work with LDA, which means you can't use sieve scripts and Dovecot's automatic sorting and indexing. Good luck. -- Stan From stan at hardwarefreak.com Tue Oct 9 11:02:46 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 09 Oct 2012 03:02:46 -0500 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <5073D8F7.1060802@hardwarefreak.com> References: <1349732239.3344.62.camel@fermat.scientia.net> <5073D8F7.1060802@hardwarefreak.com> Message-ID: <5073DA26.5060809@hardwarefreak.com> On 10/9/2012 2:57 AM, Stan Hoeppner wrote: > http://www.postfix.org/postconf.5.html#always_bcc Correction. In your case you'll need to use: http://www.postfix.org/postconf.5.html#recipient_bcc_maps Because you said you only want to archive email for some users, not simply all mail received by the colo/VPS server. -- Stan From wamp at promax.media.pl Tue Oct 9 14:36:00 2012 From: wamp at promax.media.pl (wamp at promax.media.pl) Date: Tue, 9 Oct 2012 13:36:00 +0200 Subject: [Dovecot] Quota - usage counting. Message-ID: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> Hello, I use dovecot 1.2 version with postfix virtual users and mysql. All information about quota for every user is in mysql table. How dovecot compare if quota in database is over quota in /var/vmail/exampleuser directory ? It uses something like du command ? regards, Wamp From tibby at tibby.hu Tue Oct 9 14:57:45 2012 From: tibby at tibby.hu (Tibby) Date: Tue, 9 Oct 2012 13:57:45 +0200 (CEST) Subject: [Dovecot] Quota - usage counting. In-Reply-To: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> References: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> Message-ID: <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> Do you actually have a Guide? How did you set up quota form MySql? I'm having an issu getting it working. Can you share your dovecot.conf and dovecot-sql.conf? What's the user_query in your dovecot-sql.conf ? Thank you! ----- Original Message ----- From: wamp at promax.media.pl To: dovecot at dovecot.org Subject: [Dovecot] Quota - usage counting. Hello, I use dovecot 1.2 version with postfix virtual users and mysql. All information about quota for every user is in mysql table. How dovecot compare if quota in database is over quota in /var/vmail/exampleuser directory ? It uses something like du command ? regards, Wamp From wamp at promax.media.pl Tue Oct 9 16:12:41 2012 From: wamp at promax.media.pl (wamp at promax.media.pl) Date: Tue, 9 Oct 2012 15:12:41 +0200 Subject: [Dovecot] Quota - usage counting. In-Reply-To: <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> References: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> Message-ID: <7066b6528ee587362d87aa76ba75e7f4.squirrel@poczta.promax.media.pl> Hi > Do you actually have a Guide? No, I read some docs like http://www.serverubuntu.it/postfix-dovecot-guide > How did you set up quota form MySql? > I'm having an issu getting it working. > Can you share your dovecot.conf and dovecot-sql.conf? > > What's the user_query in your dovecot-sql.conf ? I can't make it working - so need information about general idea how this values should be compared. Where is info about actual size of maildir. regards, > > Thank you! > > ----- Original Message ----- > From: wamp at promax.media.pl > To: dovecot at dovecot.org > Subject: [Dovecot] Quota - usage counting. > > Hello, > > I use dovecot 1.2 version with postfix virtual users and mysql. All > information about quota for every user is in mysql table. > How dovecot compare if quota in database is over quota in > /var/vmail/exampleuser directory ? > > It uses something like du command ? > > > regards, > Wamp > > > > From fxmulder at gmail.com Tue Oct 9 21:41:08 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 12:41:08 -0600 Subject: [Dovecot] LDAP encryption Message-ID: We have an LDAP server that contains AES encrypted passwords. So far I've been able to use this by adding a passdb module that encrypts the user's password prior to ldap comparison. Now I am looking at supporting client-side encrypted passwords. To do this I need to decrypt the password returned by LDAP. Is there a way to insert a module to do this decryption between ldap returning and the auth mechanism? From fxmulder at gmail.com Tue Oct 9 22:16:15 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 13:16:15 -0600 Subject: [Dovecot] LDAP encryption In-Reply-To: <5074750D.4070302@bitrate.net> References: <5074750D.4070302@bitrate.net> Message-ID: I don't think I understand. Right now the problem is the password retrieved from LDAP cannot be hashed to compare against what the user sent because it is encrypted. I have to perform my AES decryption before it can be hashed and compared. On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: > On 2012.10.09 14.41, James Devine wrote: > >> We have an LDAP server that contains AES encrypted passwords. So far I've >> been able to use this by adding a passdb module that encrypts the user's >> password prior to ldap comparison. Now I am looking at supporting >> client-side encrypted passwords. To do this I need to decrypt the >> password >> returned by LDAP. Is there a way to insert a module to do this decryption >> between ldap returning and the auth mechanism? >> >> that would be unwise, generally speaking. as a rule of thumb, in terms > of security fundamentals, only the rootdn [or equiv] should be able to read > the values in an ldap entry's password attribute. certainly the service > account used by dovecot should not. > > in the context of ldap, authentication should be accomplished by binding > as the user, not by retrieving attribute values and performing string > comparisons. among other things, this decouples the two components and > allows applications [e.g. dovecot] to be unconcerned with whatever password > hashing scheme the directory server might be using. > > -ben > From fxmulder at gmail.com Tue Oct 9 22:24:21 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 13:24:21 -0600 Subject: [Dovecot] LDAP encryption In-Reply-To: References: <5074750D.4070302@bitrate.net> Message-ID: Here is an example of the problem: Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: password(user at domain.tld,192.168.160.49): Generating NTLM from user 'user at domain.tld@', password 'IfBG6G3jykirE5r5vienC4w==' Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: password(user at domain.tld,192.168.160.49): Credentials: f124dc24328ed3d90db035f0d5284636 The listed password is a base64 representation of its encrypted form which I need to somehow decrypt between the time LDAP returns it and these credentials are generated. On Tue, Oct 9, 2012 at 1:16 PM, James Devine wrote: > I don't think I understand. Right now the problem is the password > retrieved from LDAP cannot be hashed to compare against what the user sent > because it is encrypted. I have to perform my AES decryption before it can > be hashed and compared. > > > On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: > >> On 2012.10.09 14.41, James Devine wrote: >> >>> We have an LDAP server that contains AES encrypted passwords. So far >>> I've >>> been able to use this by adding a passdb module that encrypts the user's >>> password prior to ldap comparison. Now I am looking at supporting >>> client-side encrypted passwords. To do this I need to decrypt the >>> password >>> returned by LDAP. Is there a way to insert a module to do this >>> decryption >>> between ldap returning and the auth mechanism? >>> >>> that would be unwise, generally speaking. as a rule of thumb, in terms >> of security fundamentals, only the rootdn [or equiv] should be able to read >> the values in an ldap entry's password attribute. certainly the service >> account used by dovecot should not. >> >> in the context of ldap, authentication should be accomplished by binding >> as the user, not by retrieving attribute values and performing string >> comparisons. among other things, this decouples the two components and >> allows applications [e.g. dovecot] to be unconcerned with whatever password >> hashing scheme the directory server might be using. >> >> -ben >> > > From tss at iki.fi Tue Oct 9 22:30:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 9 Oct 2012 22:30:19 +0300 Subject: [Dovecot] LDAP encryption In-Reply-To: References: <5074750D.4070302@bitrate.net> Message-ID: <97988640-DCC0-4068-A88F-D0201DBBE8EF@iki.fi> I don't think you can do that with a plugin without core Dovecot modifications. Unless you replace the whole passdb ldap. For example you could use passdb checkpassword if performance isn't a big issue. On 9.10.2012, at 22.24, James Devine wrote: > Here is an example of the problem: > > Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: > password(user at domain.tld,192.168.160.49): Generating NTLM from user > 'user at domain.tld@', password 'IfBG6G3jykirE5r5vienC4w==' > Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: > password(user at domain.tld,192.168.160.49): Credentials: > f124dc24328ed3d90db035f0d5284636 > > The listed password is a base64 representation of its encrypted form which > I need to somehow decrypt between the time LDAP returns it and these > credentials are generated. > > > > On Tue, Oct 9, 2012 at 1:16 PM, James Devine wrote: > >> I don't think I understand. Right now the problem is the password >> retrieved from LDAP cannot be hashed to compare against what the user sent >> because it is encrypted. I have to perform my AES decryption before it can >> be hashed and compared. >> >> >> On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: >> >>> On 2012.10.09 14.41, James Devine wrote: >>> >>>> We have an LDAP server that contains AES encrypted passwords. So far >>>> I've >>>> been able to use this by adding a passdb module that encrypts the user's >>>> password prior to ldap comparison. Now I am looking at supporting >>>> client-side encrypted passwords. To do this I need to decrypt the >>>> password >>>> returned by LDAP. Is there a way to insert a module to do this >>>> decryption >>>> between ldap returning and the auth mechanism? >>>> >>>> that would be unwise, generally speaking. as a rule of thumb, in terms >>> of security fundamentals, only the rootdn [or equiv] should be able to read >>> the values in an ldap entry's password attribute. certainly the service >>> account used by dovecot should not. >>> >>> in the context of ldap, authentication should be accomplished by binding >>> as the user, not by retrieving attribute values and performing string >>> comparisons. among other things, this decouples the two components and >>> allows applications [e.g. dovecot] to be unconcerned with whatever password >>> hashing scheme the directory server might be using. >>> >>> -ben >>> >> >> From marc at perkel.com Wed Oct 10 01:16:39 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 15:16:39 -0700 Subject: [Dovecot] Multiple Maildir? Message-ID: <5074A247.6080307@perkel.com> OK - Getting close to everything working in my weird configuration. Here's a problem I'm still having. I'm authenticating with this: userdb { driver = sql args = /etc/dovecot/master-combined-sql.conf } Default mail location: mail_location = maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs However - what I'd like to do is if the mail location doesn't exist then I want to try a second mail location: mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs How do I do that? Thanks in advance From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 10 01:42:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 10 Oct 2012 00:42:18 +0200 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074A247.6080307@perkel.com> References: <5074A247.6080307@perkel.com> Message-ID: <20121009224218.GA11401@daniel.localdomain> Marc Perkel wrote: > if the mail location doesn't exist > then I want to try a second mail location: > mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs You might do this with a script which exports the MAIL environment variable and then executes the service binary: http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection Regards Daniel -- https://plus.google.com/103021802792276734820 From kgc at corp.sonic.net Wed Oct 10 03:12:47 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 09 Oct 2012 17:12:47 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <20121009224218.GA11401@daniel.localdomain> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> Message-ID: <5074BD7F.50206@corp.sonic.net> On 10/09/12 15:42, Daniel Parthey wrote: > Marc Perkel wrote: >> if the mail location doesn't exist >> then I want to try a second mail location: >> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > > You might do this with a script which exports the MAIL environment > variable and then executes the service binary: It will work, we do this to set the maildir location to a custom hashed directory and muck around with the namespaces a bit. The script is in perl - the relevant parts look like this. #set user's maildir location for dovecot $ENV{'MAIL'} = 'maildir:' . getmaildir($ENV{'USER'}); $ENV{'USERDB_KEYS'} .= 'MAIL'; #pass along to dovecot's next process exec { $ARGV[0] } @ARGV; -K From marc at perkel.com Wed Oct 10 04:09:22 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:09:22 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <20121009224218.GA11401@daniel.localdomain> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> Message-ID: <5074CAC2.2030507@perkel.com> On 10/9/2012 3:42 PM, Daniel Parthey wrote: > Marc Perkel wrote: >> if the mail location doesn't exist >> then I want to try a second mail location: >> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > You might do this with a script which exports the MAIL environment > variable and then executes the service binary: > > http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection > > Regards > Daniel I see the idea. But how do I pick up the name and domain parameters to test the directory? From marc at perkel.com Wed Oct 10 04:12:11 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:12:11 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074BD7F.50206@corp.sonic.net> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> <5074BD7F.50206@corp.sonic.net> Message-ID: <5074CB6B.4090202@perkel.com> On 10/9/2012 5:12 PM, Kelsey Cummings wrote: > On 10/09/12 15:42, Daniel Parthey wrote: >> Marc Perkel wrote: >>> if the mail location doesn't exist >>> then I want to try a second mail location: >>> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> >> You might do this with a script which exports the MAIL environment >> variable and then executes the service binary: > > It will work, we do this to set the maildir location to a custom > hashed directory and muck around with the namespaces a bit. > > The script is in perl - the relevant parts look like this. > > #set user's maildir location for dovecot > $ENV{'MAIL'} = 'maildir:' . getmaildir($ENV{'USER'}); > $ENV{'USERDB_KEYS'} .= 'MAIL'; > > #pass along to dovecot's next process > exec { $ARGV[0] } @ARGV; > > -K > > Namespaces is something I don't understand. Still wondering what environment variables I can pick up in this script. What I want to do is first try /fakedir/%d/%n and if that doesn't exist I want to go to /email/%d/%n From marc at perkel.com Wed Oct 10 04:34:08 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:34:08 -0700 Subject: [Dovecot] Feature Request Message-ID: <5074D090.3010909@perkel.com> It would be handy (for me) if there were a userdb where a directory structure defined the db. userdb stat { mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } userdb stat { mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } The idea being that if the first directory doesn't exist then it will try the second one. From tss at iki.fi Wed Oct 10 05:29:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 05:29:05 +0300 Subject: [Dovecot] Feature Request In-Reply-To: <5074D090.3010909@perkel.com> References: <5074D090.3010909@perkel.com> Message-ID: <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> On 10.10.2012, at 4.34, Marc Perkel wrote: > It would be handy (for me) if there were a userdb where a directory structure defined the db. > > userdb stat { > mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > } > > userdb stat { > mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > } > > The idea being that if the first directory doesn't exist then it will try the second one. You could already implement this as userdb checkpassword script. From marc at perkel.com Wed Oct 10 05:40:27 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 19:40:27 -0700 Subject: [Dovecot] Feature Request In-Reply-To: <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> Message-ID: <5074E01B.8030001@perkel.com> On 10/9/2012 7:29 PM, Timo Sirainen wrote: > On 10.10.2012, at 4.34, Marc Perkel wrote: > >> It would be handy (for me) if there were a userdb where a directory structure defined the db. >> >> userdb stat { >> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> } >> >> userdb stat { >> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> } >> >> The idea being that if the first directory doesn't exist then it will try the second one. > You could already implement this as userdb checkpassword script. Can you give me an example? From tss at iki.fi Wed Oct 10 05:50:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 05:50:15 +0300 Subject: [Dovecot] Feature Request In-Reply-To: <5074E01B.8030001@perkel.com> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> <5074E01B.8030001@perkel.com> Message-ID: <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> On 10.10.2012, at 5.40, Marc Perkel wrote: >>> It would be handy (for me) if there were a userdb where a directory structure defined the db. >>> >>> userdb stat { >>> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >>> } >>> >>> userdb stat { >>> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >>> } >>> >>> The idea being that if the first directory doesn't exist then it will try the second one. >> You could already implement this as userdb checkpassword script. > > Can you give me an example? Something like this: userdb { driver = checkpasword args = /usr/local/bin/userdb.sh } /usr/local/bin/userdb.sh: #!/bin/sh path=/fakedir/$AUTH_DOMAIN/$AUTH_USERNAME if [ -d $path ]; then MAIL=maildir:$path:INBOX=/email/$AUTH_DOMAIN/$AUTH_USERNAME:LAYOUT=fs EXTRA=mail AUTHORIZED=2 exec "$@" fi exit 1 (I'm not sure if the MAIL/mail should be USERDB_MAIL/userdb_mail instead. Probably not.) From raabe at froglogic.com Wed Oct 10 10:29:00 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 09:29:00 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes Message-ID: <507523BC.9050004@froglogic.com> Hi, I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for providing access to a mailing list archive. The archive is implemented as a public read-only mailbox with per-user index files (i.e. the \Seen flags are per-user). I recently enbled the Squat plugin to accelerate searches in the message bodies and noticed that every user (I'm using a virtual user setup) gets his own dovecot.index.search and dovecot.index.search.uids copies. Is it possible to share those files among all users of the system? The squat plugin appears to store the search indices among the other index files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no matter what; I considered storing a central copy of the index files somewhere and then creating symlinks for all users. It should be ok as far as file-permissions go since all mail is owned by a single vmail system user, but I wonder whether the indices are really the same (I noticed their md5 checksums differ) and whether there may be file locking issues in case two users search message bodies simultaneously. Can anybody shed some light? -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From robert at schetterer.org Wed Oct 10 10:49:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 09:49:33 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <507523BC.9050004@froglogic.com> References: <507523BC.9050004@froglogic.com> Message-ID: <5075288D.9080304@schetterer.org> Am 10.10.2012 09:29, schrieb Frerich Raabe: > Hi, > > I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for > providing access to a mailing list archive. The archive is implemented > as a public read-only mailbox with per-user index files (i.e. the \Seen > flags are per-user). i guess better upgrade to 2.1.x first > > I recently enbled the Squat plugin to accelerate searches in the message > bodies and noticed that every user (I'm using a virtual user setup) gets > his own dovecot.index.search and dovecot.index.search.uids copies. > > Is it possible to share those files among all users of the system? The > squat plugin appears to store the search indices among the other index > files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no > matter what; I considered storing a central copy of the index files > somewhere and then creating symlinks for all users. It should be ok as > far as file-permissions go since all mail is owned by a single vmail > system user, but I wonder whether the indices are really the same (I > noticed their md5 checksums differ) and whether there may be file > locking issues in case two users search message bodies simultaneously. > > Can anybody shed some light? > after upgrade http://wiki2.dovecot.org/Plugins/FTS/Lucene may be better choice this info might help http://wiki2.dovecot.org/MailLocation ---snip Index files Index files are by default stored under the same directory as mails. With maildir they are stored in the actual maildirs, with mbox they are stored under .imap/ directory. You may want to change the index file location if you're using NFS or if you're setting up shared mailboxes. You can change the index file location by adding :INDEX= to mail_location. For example: mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u --snip after upgrade come back, ask again, or meanwhile Timo gives better advice -- Best Regards MfG Robert Schetterer From raabe at froglogic.com Wed Oct 10 11:06:47 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 10:06:47 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <5075288D.9080304@schetterer.org> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> Message-ID: <50752C97.1010209@froglogic.com> Am 10.10.2012 09:49, schrieb Robert Schetterer: > Am 10.10.2012 09:29, schrieb Frerich Raabe: >> I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for >> providing access to a mailing list archive. The archive is implemented >> as a public read-only mailbox with per-user index files (i.e. the \Seen >> flags are per-user). > > i guess better upgrade to 2.1.x > first Given that Dovecot 1.2.17 works fine for me, I actually didn't see the need to upgrade yet. >> I recently enbled the Squat plugin to accelerate searches in the message >> bodies and noticed that every user (I'm using a virtual user setup) gets >> his own dovecot.index.search and dovecot.index.search.uids copies. >> >> Is it possible to share those files among all users of the system? The >> squat plugin appears to store the search indices among the other index >> files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no >> matter what; I considered storing a central copy of the index files >> somewhere and then creating symlinks for all users. It should be ok as >> far as file-permissions go since all mail is owned by a single vmail >> system user, but I wonder whether the indices are really the same (I >> noticed their md5 checksums differ) and whether there may be file >> locking issues in case two users search message bodies simultaneously. >> >> Can anybody shed some light? >> > > after upgrade > http://wiki2.dovecot.org/Plugins/FTS/Lucene > may be better choice Why? > this info might help > > http://wiki2.dovecot.org/MailLocation > > ---snip > Index files > > Index files are by default stored under the same directory as mails. > With maildir they are stored in the actual maildirs, with mbox they are > stored under .imap/ directory. You may want to change the index file > location if you're using NFS or if you're setting up shared mailboxes. > > You can change the index file location by adding :INDEX= to > mail_location. For example: > > mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u > --snip I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: namespace public { separator = / prefix = Lists/ location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists subscriptions = no } Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) > after upgrade come back, ask again, or meanwhile Timo gives better advice Does this imply that questions regarding Dovecot 1.2.17 are considered offtopic on this list? If so, I apologize - I'll look for another forum then. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From robert at schetterer.org Wed Oct 10 11:24:49 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 10:24:49 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <50752C97.1010209@froglogic.com> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> Message-ID: <507530D1.8070202@schetterer.org> Am 10.10.2012 10:06, schrieb Frerich Raabe: > I already use this; as I mentioned, the index files of the public > readonly mailbox is stored per-user so that each user has his own set of > \Seen flags. Here's my public namespace: > > namespace public { > separator = / > prefix = Lists/ > location = > maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > > subscriptions = no > } > > Alas, this means that *all* index files (including the Squat index) is > stored per-user whereas I'd just to have just *some* of them per-user. :-) > >> after upgrade come back, ask again, or meanwhile Timo gives better advice > > Does this imply that questions regarding Dovecot 1.2.17 are considered > offtopic on this list? If so, I apologize - I'll look for another forum > then. no wait till others will reply, indexing questions might be more complex to answer, then i know recent in short time perhaps meanwhile this helps -snip http://wiki2.dovecot.org/SharedMailboxes/Public With Maildir a dovecot-shared file controls if the \Seen flags are shared or private. The file must be created separately inside each Maildir, although if the file already exists in the Maildir root it's automatically copied for newly created mailboxes. If dovecot-shared file doesn't exist in Maildir, the \Seen flags are shared. If it exists, the \Seen flag state is stored only in the user's index files. By making each user have their own private index files, you can make the \Seen flag private for the users. For example: namespace { type = public separator = / prefix = Public/ location = maildir:/var/mail/public:INDEX=~/Maildir/public subscriptions = no } Now when accessing e.g. "Public/lkml" mailbox, Dovecot keeps its index files in ~/Maildir/public/lkml/ directory. If it ever gets deleted, the \Seen flags are lost. If you want to change what flags are shared when dovecot-shared file exists, currently you'll have to modify the source code: src/lib-storage/index/maildir/maildir-storage.c maildir_open() has mbox->ibox.box.private_flags_mask = MAIL_SEEN; Change the MAIL_SEEN to any flag combination you want. See src/lib-mail/mail-types.h for list of valid flags. --snip however if there is a need for some new/debugged/better features relate to what you asked you have to upgrade to latest dovecot -- Best Regards MfG Robert Schetterer From sandro.tosi at dada.eu Wed Oct 10 12:23:17 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Wed, 10 Oct 2012 11:23:17 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups Message-ID: <50753E85.5060904@dada.eu> Hello, we're scouting if it's possible to use Pigeonhole (currently v0.3.1, as this will be provided with an upcoming Debian package) with MySQL dict lookups with the mail setup we're designing. Our (main) goals are: 1. store the filters on the database 2. allow each user to enable/disable any of the filters set we provide (it's a static set of some general filters, available to all the users; we're currently not providing the possibility to users to write their own filters) For point 1) we already see[1] that's possible, but it uses the map construct that might not fit with our current database structure: we have a domain table (storing the domain info) and a mailbox table (storing the mailbox info, but the username is composed by the local part, stored in this table, and the domain part is a FK to the domain table, using an id). Do you think it's possible to run a join query on domain+mailbox to retrieve the mailbox_id needed to query the table for the filters? Or do we have to create the filter table and store the local at domain.ext info there ("relaxing" the integrity relationships between tables)? How do we specify which filters are enabled for any given user? We originally thought of an "Enabled" field on the filter table, but in the example in the doc[1] I hadn't seen a way to do that: it seems like the filter list is specified in the proxy definition - am I wrong? How can we do that? [1] http://hg.rename-it.nl/dovecot-2.1-pigeonhole/file/e9ed5d5cef4b/doc/script-location-dict.txt I think it's enough for now, maybe additional questions will arise going deeper in details :) Thanks in advance, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From raabe at froglogic.com Wed Oct 10 12:49:27 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 11:49:27 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <507530D1.8070202@schetterer.org> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <507530D1.8070202@schetterer.org> Message-ID: <507544A7.9050306@froglogic.com> Am 10.10.2012 10:24, schrieb Robert Schetterer: > Am 10.10.2012 10:06, schrieb Frerich Raabe: >> I already use this; as I mentioned, the index files of the public >> readonly mailbox is stored per-user so that each user has his own set of >> \Seen flags. Here's my public namespace: >> >> namespace public { >> separator = / >> prefix = Lists/ >> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >> subscriptions = no >> } [..] > perhaps meanwhile this helps > > -snip > > http://wiki2.dovecot.org/SharedMailboxes/Public [..] > namespace { > type = public > separator = / > prefix = Public/ > location = maildir:/var/mail/public:INDEX=~/Maildir/public > subscriptions = no > } Note how this is basically exactly the same as what I posted, except that it uses the Dovecot 2 configuration file format ('type = public') and that it calls the prefix/location "public" instead of "lists". > If you want to change what flags are shared when dovecot-shared file > exists, currently you'll have to modify the source code: > src/lib-storage/index/maildir/maildir-storage.c maildir_open() has > mbox->ibox.box.private_flags_mask = MAIL_SEEN; Change the MAIL_SEEN to > any flag combination you want. See src/lib-mail/mail-types.h for list of > valid flags. I don't think this is applicable to my case, and a check of the source code seems to confirm that: I'm not trying to change the set of flags stored for a given mail but rather the index file of the Squat plugin. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From natanael.copa at gmail.com Wed Oct 10 13:00:51 2012 From: natanael.copa at gmail.com (Natanael Copa) Date: Wed, 10 Oct 2012 12:00:51 +0200 Subject: [Dovecot] [PATCH] dovadm plugins underlinking Message-ID: Hi, Running doveadm on Alpine Linux will show various underlinking errors: /usr/bin/doveadm: symbol 'acl_user_module': can't resolve symbol in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. /usr/bin/doveadm: symbol 'acl_identifier_parse': can't resolve symbol in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. ... (complete list is found here: http://bugs.alpinelinux.org/issues/1274) I understand that those are "harmless" (at least as long as you dont use those plugins), and even if some dynamic linkers are forgiving, the doveadm plugin modules are technically underlinked. The lib10_doveadm_acl_plugin.so uses symbol acl_user_module: $ nm -D /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep acl_user_module U acl_user_module That symbol is defined in lib01_acl_plugin.so: $ nm -D /usr/lib/dovecot/lib01_acl_plugin.so | grep acl_user_module 000000000020fb80 D acl_user_module Which means that lib10_doveadm_acl_plugin.so should be directly linked to lib01_acl_plugin.so. But it is not: $ readelf -d /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep NEEDED 0x0000000000000001 (NEEDED) Shared library: [librt.so.0.9.32] 0x0000000000000001 (NEEDED) Shared library: [libc.so.0.9.32] Below is a patch that should solve it. --- diff -r 4d268e810c15 src/plugins/acl/Makefile.am --- a/src/plugins/acl/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/acl/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -10,7 +10,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_acl_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_acl_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_acl_plugin_la_LIBADD = $(module_LTLIBRARIES) lib01_acl_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/expire/Makefile.am --- a/src/plugins/expire/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/expire/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -14,7 +14,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_expire_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_expire_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_expire_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_expire_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/fts/Makefile.am --- a/src/plugins/fts/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/fts/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib20_doveadm_fts_plugin_la_LDFLAGS = -module -avoid-version +lib20_doveadm_fts_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib20_doveadm_fts_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_fts_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/quota/Makefile.am --- a/src/plugins/quota/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/quota/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_quota_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_quota_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_quota_plugin_la_LIBADD = $(module_LTLIBRARIES) lib10_quota_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/zlib/Makefile.am --- a/src/plugins/zlib/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/zlib/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_zlib_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_zlib_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_zlib_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_zlib_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ From tss at iki.fi Wed Oct 10 13:22:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 13:22:53 +0300 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: References: Message-ID: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> On 10.10.2012, at 13.00, Natanael Copa wrote: > Running doveadm on Alpine Linux will show various underlinking errors: > /usr/bin/doveadm: symbol 'acl_user_module': can't resolve symbol in > lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. > /usr/bin/doveadm: symbol 'acl_identifier_parse': can't resolve symbol > in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. .. > Which means that lib10_doveadm_acl_plugin.so should be directly linked > to lib01_acl_plugin.so. But it is not: .. > Below is a patch that should solve it. I think I once did that with imap_quota plugin for quota plugin. The problem was that then it was possible to set "mail_plugins=imap_quota", which automatically loaded the quota plugin, but didn't call its quota_plugin_init() function, which caused some strange behavior (crashed probably). With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. Maybe there are some other ways to make this work more nicely. There would still be time to change it for v2.2 if you have good ideas. :) From natanael.copa at gmail.com Wed Oct 10 15:17:58 2012 From: natanael.copa at gmail.com (Natanael Copa) Date: Wed, 10 Oct 2012 14:17:58 +0200 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> References: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> Message-ID: On Wed, Oct 10, 2012 at 12:22 PM, Timo Sirainen wrote: > On 10.10.2012, at 13.00, Natanael Copa wrote: > >> Running doveadm on Alpine Linux will show various underlinking errors: > With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. Why does it need to load all the plugins? Why not only try to load those who are enabled? > Maybe there are some other ways to make this work more nicely. There would still be time to change it for v2.2 if you have good ideas. :) Link statically? Or only link in the .o files that has the needed symbols? I'm not familiar with the code, but how about adding an char* adm_module to the module struct, and then instead of trying to dlopen all in doveadm dir, loop through all the already loaded modules and dlopen(loaded_module->adm_module)? -- Natanael Copa From listas at adminlinux.com.br Wed Oct 10 15:46:56 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Wed, 10 Oct 2012 09:46:56 -0300 Subject: [Dovecot] About ManageSieve and TLS In-Reply-To: <506B40F9.1000905@rename-it.nl> References: <506B3653.5020804@adminlinux.com.br> <506B40F9.1000905@rename-it.nl> Message-ID: <50756E40.2040408@adminlinux.com.br> It works ! Thanks Stephan! On 02-10-2012 16:31, Stephan Bosch wrote: > If you really want to, you can disable TLS for ManageSieve > specifically by putting a ssl=no inside the protocol sieve {} section. -- Thiago Henrique adminlinux.com.br From listas at adminlinux.com.br Wed Oct 10 16:39:27 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Wed, 10 Oct 2012 10:39:27 -0300 Subject: [Dovecot] Irrelevant information filling logs Message-ID: <50757A8F.4030508@adminlinux.com.br> Hi, I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. There is a lot of logs of "quota exceeded" like this: Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. Is there a way to send specific Dovecot errors on specific files or just discard them? Thanks -- Thiago Henrique adminlinux.com.br From robert at schetterer.org Wed Oct 10 17:14:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 16:14:15 +0200 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <507582B7.5040400@schetterer.org> Am 10.10.2012 15:39, schrieb 3.listas at adminlinux.com.br: > Hi, > > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My > mailbox server is shared by ~ 10k domains. It works fine with ~50k > accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, > user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: > msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded > (mailbox for user is full) > > These messages are not important to me. But these messages fill the log > files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or just > discard them? > > Thanks > -- > Thiago Henrique > adminlinux.com.br no sure if you can disable this notice without loosing other wanted ones by dovecot log settings if using rsyslog you can do it with discard action http://www.rsyslog.com/discarding-unwanted-messages/ -- Best Regards MfG Robert Schetterer From tibby at tibby.hu Wed Oct 10 17:29:08 2012 From: tibby at tibby.hu (Tibby) Date: Wed, 10 Oct 2012 16:29:08 +0200 (CEST) Subject: [Dovecot] I need a quota expert In-Reply-To: <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> References: <20121010142523.871B11AE881D@dovecot.org> <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> Message-ID: <648980957.4179.1349879348102.JavaMail.root@tibby.hu> I'm having problems with dovecot quota configuration. If I Include the driver mysql in dovecot-sql.conf I'm getting this error: dovecot: dict: Error in configuration file /etc/dovecot/dovecot-sql.conf line 1: Unknown setting: driver dovecot: dict: Failed to initialize dictionary 'quotadict' If I Don't include it: dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf Now What??? How Should I configure quota ? My SQL has a mail db with a user table. The table has email, password, quota_kb fields. my dovecot.conf: ---------------- protocols = imap imaps pop3 pop3s log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir disable_plaintext_auth = no ssl_cert_file = /etc/ssl/certs/dovecot.pem ssl_key_file = /etc/ssl/private/dovecot.pem namespace private { separator = . prefix = INBOX. inbox = yes } protocol imap { mail_plugins = quota imap_quota } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster at domain.tld mail_plugins = sieve quota global_script_path = /home/vmail/globalsieverc mail_plugin_dir = /usr/lib/dovecot/modules/lda } protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } dict { quotadict = mysql:/etc/dovecot/dovecot-sql.conf } plugin { quota = dict:user::proxy::quotadict quota_rule = *:storage=10M:messages=1000 } auth default { user = root passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb sql { args = /etc/dovecot/dovecot-sql.conf } socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } mail_uid = 5000 mail_gid = 5000 my dovecot-sql.conf: ---------------- driver = mysql connect = host=127.0.0.1 dbname=mail user=****** password=****** default_pass_scheme = CRYPT password_query = SELECT email as user, password, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u'; user_query = SELECT CONCAT('/home/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u' map { pattern = priv/quota/storage table = user username_field = user value_field = quota_kb } SOMEBODY PLEASE HELP ME!!! :) From jbates at brightok.net Wed Oct 10 17:37:16 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:37:16 -0500 Subject: [Dovecot] lmtp proxy logging Message-ID: <5075881C.4060905@brightok.net> The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. Jack From roundcube222 at alaadin.org Wed Oct 10 17:40:12 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 10 Oct 2012 17:40:12 +0300 Subject: [Dovecot] Dovecot doesnot disconnect at end of pop3 session Message-ID: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> Hello, I have 3 users checking same email account using pop3 (outlook express) suddenly after some hours , users are unable to disconnect from pop3 after checking mail So i telnet to the pop3 and found that the issue is true but i do not know the reason telnet mymail.com 110 user username pass password list quit all commands work except the quit command, Dovecot hang at this point !!! Why Dovecot hang on receiving the Quit command !!! and doesnot disconnect !! Any help is greatly appreciated. Thanks. From jbates at brightok.net Wed Oct 10 17:43:02 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:43:02 -0500 Subject: [Dovecot] I need a quota expert In-Reply-To: <648980957.4179.1349879348102.JavaMail.root@tibby.hu> References: <20121010142523.871B11AE881D@dovecot.org> <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> <648980957.4179.1349879348102.JavaMail.root@tibby.hu> Message-ID: <50758976.8060302@brightok.net> Looking at the wiki: http://wiki2.dovecot.org/Quota/Dict It appears that the conf files for dict are different than the sql files for auth. This is why you are getting error messages. The auth config needs the driver option, the dict config does not. Create a file specifically for dict and point to that. Use the above wiki's examples for format. Jack On 10/10/2012 9:29 AM, Tibby wrote: > I'm having problems with dovecot quota configuration. > If I Include the driver mysql in dovecot-sql.conf I'm getting this error: > dovecot: dict: Error in configuration file /etc/dovecot/dovecot-sql.conf line 1: Unknown setting: driver > dovecot: dict: Failed to initialize dictionary 'quotadict' > > If I Don't include it: > dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf > dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf > > Now What??? > How Should I configure quota ? > > My SQL has a mail db with a user table. > The table has email, password, quota_kb fields. > > my dovecot.conf: > ---------------- > protocols = imap imaps pop3 pop3s > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/home/vmail/%d/%n/Maildir > disable_plaintext_auth = no > ssl_cert_file = /etc/ssl/certs/dovecot.pem > ssl_key_file = /etc/ssl/private/dovecot.pem > namespace private { > separator = . > prefix = INBOX. > inbox = yes > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol lda { > log_path = /home/vmail/dovecot-deliver.log > auth_socket_path = /var/run/dovecot/auth-master > postmaster_address = postmaster at domain.tld > mail_plugins = sieve quota > global_script_path = /home/vmail/globalsieverc > mail_plugin_dir = /usr/lib/dovecot/modules/lda > } > protocol pop3 { > pop3_uidl_format = %08Xu%08Xv > mail_plugins = quota > } > dict { > quotadict = mysql:/etc/dovecot/dovecot-sql.conf > } > plugin { > quota = dict:user::proxy::quotadict > quota_rule = *:storage=10M:messages=1000 > } > auth default { > user = root > passdb sql { > args = /etc/dovecot/dovecot-sql.conf > } > userdb sql { > args = /etc/dovecot/dovecot-sql.conf > } > socket listen { > master { > path = /var/run/dovecot/auth-master > mode = 0600 > user = vmail > } > client { > path = /var/spool/postfix/private/auth > mode = 0660 > user = postfix > group = postfix > } > } > } > mail_uid = 5000 > mail_gid = 5000 > > > > > > my dovecot-sql.conf: > ---------------- > driver = mysql > connect = host=127.0.0.1 dbname=mail user=****** password=****** > default_pass_scheme = CRYPT > password_query = SELECT email as user, password, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u'; > > user_query = SELECT CONCAT('/home/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u' > > map { > pattern = priv/quota/storage > table = user > username_field = user > value_field = quota_kb > } > > > SOMEBODY PLEASE HELP ME!!! :) From jbates at brightok.net Wed Oct 10 17:47:24 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:47:24 -0500 Subject: [Dovecot] Dovecot doesnot disconnect at end of pop3 session In-Reply-To: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> References: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> Message-ID: <50758A7C.1050501@brightok.net> On 10/10/2012 9:40 AM, Robert JR wrote: > > telnet mymail.com 110 > user username > pass password > list > quit > > all commands work except the quit command, Dovecot hang at this point !!! > > Why Dovecot hang on receiving the Quit command !!! and doesnot > disconnect !! > Please include your dovecot version. Also, does the list command end with a line that contains a period? You might want to include dovecot -n as well, which will show everyone what your mail storage type is and other relevant information. Jack From roundcube222 at alaadin.org Wed Oct 10 18:07:57 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 10 Oct 2012 18:07:57 +0300 Subject: [Dovecot] Fwd: Re: Dovecot doesnot disconnect at end of pop3 session Message-ID: On 2012-10-10 17:47, Jack Bates wrote: > On 10/10/2012 9:40 AM, Robert JR wrote: > >> telnet mymail.com 110 user username pass password list quit all >> commands work except the quit command, Dovecot hang at this point >> !!! >> Why Dovecot hang on receiving the Quit command !!! and doesnot >> disconnect !! > Please include your dovecot version. Also, does the list command end > with a line that contains a period? You might want to include dovecot > -n > as well, which will show everyone what your mail storage type is and > other relevant information. Jack # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.9.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login auth_username_format = %Lu disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mal:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = References: Message-ID: <50759753.5040301@brightok.net> On 10/10/2012 10:07 AM, Robert JR wrote: > > list command return msg number and size only. > Thanks Below is the proper format you should see. The line with a period at the end is required to show that the list command has completed. If you do not get that, then the server hasn't successfully completed the list command to accept new commands. Check your logs for any mention of the connection and errors that might have generated. If you are using NFS, there are several issues with locking that can be problematic. If you are getting the '.' line, then try other commands besides quit. It is important to determine if the code is ignoring input or if it has a problem with the actual quit code. Also, I notice you mentioned several hours. It would be useful to know if this occurs on startup or if it fails over time. It is also useful to know if it effects all accounts or just some of the active accounts. It is also useful to know if there are people currently logged in on those accounts when you perform your test. telnet localhost 110 Trying ::1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user **** +OK pass **** +OK Logged in. list +OK 3 messages: 1 2821 2 5907 3 11171 . quit +OK Logging out. Connection closed by foreign host. From slitt at troubleshooters.com Wed Oct 10 20:35:08 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Wed, 10 Oct 2012 13:35:08 -0400 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <20121010133508.6f836584@mydesk> On Wed, 10 Oct 2012 10:39:27 -0300, 3.listas at adminlinux.com.br said: > Hi, > > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. > My mailbox server is shared by ~ 10k domains. It works fine with ~50k > accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, > user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: > msgid=unspecified: failed to store into mailbox 'INBOX': Quota > exceeded (mailbox for user is full) > > These messages are not important to me. But these messages fill the > log files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or > just discard them? If all else fails, you could cat the log through a grep -v to filter out the quota exceeded messages, and then pipe it to less for viewing. SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From jbates at brightok.net Wed Oct 10 21:09:25 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 13:09:25 -0500 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <5075B9D5.8030007@brightok.net> On 10/10/2012 8:39 AM, 3.listas at adminlinux.com.br wrote: > > Is there a way to send specific Dovecot errors on specific files or > just discard them? syslog/rsyslog filters are the best method, in my opinion. This would allow you to filter to another file or discard things you aren't interested in all together. I personally like quota messages in another logfile. Then I can tail the file with a script and issue changes so that the MTA servers reject emails to that user instead of trying to deliver, then periodically check quotas for users we currently block to determine when it is okay to accept emails again. It really lowers the bounces on the MTA servers. Jack From tss at iki.fi Wed Oct 10 22:37:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 22:37:25 +0300 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <5B596E8E-0721-4EE9-81E2-FC7F308B3C5C@iki.fi> On 10.10.2012, at 16.39, 3.listas at adminlinux.com.br wrote: > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) > > These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or just discard them? Upgrade to v2.1, they are logged with info level there. (I think v2.0 also logs them with info level if you don't use Sieve.) From dovecot at freakout.de Thu Oct 11 09:43:18 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Thu, 11 Oct 2012 08:43:18 +0200 (CEST) Subject: [Dovecot] dovecot cores Message-ID: <201210110643.q9B6hIF6003121@bongo.freakout.de> Hi dovecot-community, i have set up a new dovecot server - everything is self-compiled and with newest versions. dovecot daemon dies every night: Bugreport Mail - dovecot cores with sig11 my dovecot daemon cores every night - no message in log file. Please help. It runs in an OpenVZ container - therefore reiserfs is reported as filesystem. root at glen ~]# cat /etc/dovecot/dovecot.conf # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.8.2.el5.028stab101.1PAE i686 CentOS release 5.8 (Final) reiserfs auth_mechanisms = plain login cram-md5 apop default_login_user = nobody disable_plaintext_auth = no first_valid_uid = 300 listen = * mail_gid = 332 mail_home = /var/dovecot mail_location = maildir:/var/spool/mail/virtual/%d/%n mail_plugins = quota mail_uid = 332 passdb { driver = sql args = /etc/dovecot/sql.conf } plugin { quota = maildir:User quota } protocols = imap pop3 ssl_cert = This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /opt/dovecot-2.1.10-3/sbin/dovecot...(no debugging symbols found)...done. [New Thread 20049] Reading symbols from /opt/mysql/lib/libmysqlclient.so.18...done. Loaded symbols for /opt/mysql/lib/libmysqlclient.so.18 Reading symbols from /opt/dovecot/lib/libdovecot.so.0...done. Loaded symbols for /opt/dovecot/lib/libdovecot.so.0 Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /opt/mysql/lib/libstrings.so...done. Loaded symbols for /opt/mysql/lib/libstrings.so Reading symbols from /opt/zlib/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /opt/zlib/lib/libz.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /opt/ssp/lib/libssp.so.0...done. Loaded symbols for /opt/ssp/lib/libssp.so.0 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/opt/dovecot/sbin/dovecot'. Program terminated with signal 11, Segmentation fault. #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 (gdb) bt #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 #1 0xb7f131c0 in master_instance_list_add_line (list=0x9d48880, line=0x9d540c8 "1349762052\tdovecot\t/var/dovecot/run\t") at master-instance.c:85 #2 0xb7f1331f in master_instance_list_refresh (list=0x9d48880) at master-instance.c:115 #3 0xb7f13620 in master_instance_write_init (list=0x9d48880, dotlock_r=0xbff6fb60) at master-instance.c:173 #4 0xb7f1390d in master_instance_list_set_name (list=0x9d48880, base_dir=0x8055300 "/var/dovecot/run", name=0x8055323 "dovecot") at master-instance.c:252 #5 0x0804c347 in net_accept () at network.c:502 #6 0xb7f3255b in io_loop_handle_timeouts_real (ioloop=0x9d465b0) at ioloop.c:354 #7 0xb7f325de in io_loop_handle_timeouts (ioloop=0x9d465b0) at ioloop.c:367 #8 0xb7f338b6 in io_loop_handler_run (ioloop=0x9d465b0) at ioloop-poll.c:171 #9 0xb7f326d3 in io_loop_run (ioloop=0x9d465b0) at ioloop.c:398 #10 0xb7f17304 in master_service_run (service=0x9d464e0, callback=0) at master-service.c:543 #11 0x0804d375 in net_accept () at network.c:502 #12 0xb7d84e9c in __libc_start_main () from /lib/libc.so.6 #13 0x0804b501 in net_accept () at network.c:502 (gdb) q Regards Axel From calestyo at scientia.net Thu Oct 11 05:18:34 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 04:18:34 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup (ignore) In-Reply-To: <1349921426.3341.175.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> Message-ID: <1349921914.3341.183.camel@fermat.scientia.net> Oops... that was obivously not intended for dovecot but AOX mailing list,... where I ask around similar questions. Sorry for the noise =) Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From talanchor at mail.ru Thu Oct 11 13:21:09 2012 From: talanchor at mail.ru (=?UTF-8?B?LiAu?=) Date: Thu, 11 Oct 2012 14:21:09 +0400 Subject: [Dovecot] =?utf-8?q?memory_allocation_in_new_thread?= Message-ID: <1349950869.22650094@f123.mail.ru> Hi! I have some problems with memory allocation. I create new thread in cidir storage and call malloc(), and it fails to allocate even 1 byte. What can cause this problem? Dovecot vesrion is: 2.1.10 (130563b592c9+) Sample code looks like this (I also link to pthread with: -pthread): #define TEST_MALLOC() \ { \ ??? void *p; \ ??? p = malloc(1); \ ??? if (!p) { \ ??? ??? i_info("%s: malloc() failed", __FUNCTION__); \ ??? } else { \ ??? ??? i_info("%s: malloc() succeeded", __FUNCTION__); \ ??? ??? free(p); \ ??? } \ } #include static void *test_pthread_malloc_func(void *data_) { ??? TEST_MALLOC(); ??? return NULL; } static void test_pthread_malloc() { ??? int ret; ??? pthread_t tid; ??? TEST_MALLOC(); ??? ret = pthread_create(&tid, NULL, test_pthread_malloc_func, NULL); ??? if (ret) { ??????? i_info("failed to start thread"); ??? } else { ??????? pthread_join(tid, NULL); ??? } } I call test_pthread_malloc() function. It produces following output: Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc: malloc() succeeded Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc_func: malloc() failed From calestyo at scientia.net Thu Oct 11 05:10:26 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 04:10:26 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup Message-ID: <1349921426.3341.175.camel@fermat.scientia.net> Hi folks. Perhaps you find some time to look into this,... I'm trying to plan my mail system and would have some questions. The overall idea is about the following: - There is a (internet) server, which is the MTA (which will be postfix) and imap and/or pop3 server (which shall be, guess, dovecot ... AOX comes in later). - Any spam filtering, virus-scanning, mail filtering happens on that server. - I have not yet decided on whether to use maildrop for this or Sieve. Maildrop is nice, but has one major deficiency which I don't know how to work around. - A few clients (that means I won't serve 100 of users) connect to that server via imap and should see all mails, etc. already in some fancy sorted hierarchy (that means filtered into different directories). So far nothing complicated. But now... - Apart from spam, I never delete mail; and because I'm subscribed to many lists, I get a lot of mail. - Storage on my server is limited and it's located somewhere at my ISP, so I generally do not trust it with respect to safety... For both reasons, I want the canonical archive of all mail to be at home at some local server. - The local server should also be an imap server, so that I can access the archive from may computers at home. - The local server won't be available from the internet. - The local archive should have the same folder hierarchy as the internet server (I'd prefer not to filter twice). - The local archive might be AOX... Mail Flow: - I generally want to have _all_ mail (which is not sorted out because of being spam) to be archived at the local server. - But(!) I want to selectively keep (in addition) mail at the internet server. For example I may want to select the folder that contains all mail form some friend to be kept online completely. But I may want to decide that mailinglists keep only the last 10 days and/or 1000 messages of mail. - The idea is, that the local server regularly (when it is online/running) catches new mail from the internet server... and stores it in the archive. - So apart from new mail that has not yet been read, that local archive contains always all mails that are also on the internet server... the later may contain (for specific directories) the same, or just parts of. - The MUAs will then have two imap accounts, one to the internet server and one to the local archive,... each one being usable, depending on where I am. 1) This is where my first problem arises: How can I implement that mail flow, especially: - How do I secure that all mail is read from the internet server (i.e. that nothing is "forgotten"? - How do I make sure that no mails are retrieved twice (or more)? A problem which I often had with pop, when the mail client crashed during sync? - Further it must be secured, that when I delete something on the internet server, it is NOT deleted on the local server (on the next mail-fetching).... this is why I don't use the word "sync". a) One stupid solution would be, that I duplicate all mail on the online server,... one part is for staying online, one part is for being fetched to the local archive. As soon as it was fetched... that copy gets removed (always). That solution would give a clean and secured separation of both? b) I don't think offlineimap or any other caching-like solution is the right thing... especially as one must always fear that such a cache may be accidentally wiped. Are there better solutions than (a)? 2) Problem would be already a refinement of a working solution for (1) (but obviously not when using (1).(a) ). When e.g. reply to or forward a mail using the online server,... and that mail had already been fetched,... can I make the flag synced? 3) Is AOX suitable for the local server? - I couldn't use maildir locally, because I loose just to much space to the block fragmentation. But I guess I won't have this problem with the DB backend. a) Is AOX able to manage a really big folder hierarchy that basically ever keeps growing... with easily several 100k mails per folder... and that is in total already over 100GB? I read that e.g. dovecot would have more performance problems with that. b) I would prefer to have fast full text search. Does AOX provide this? I read that IMAP has limitations which make full text search not really usable via it? Further, I remember some dovecot wiki page that showed a comparison which said that both do not perfectly implement imap. Thanks a lot, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From robert at schetterer.org Thu Oct 11 15:56:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 11 Oct 2012 14:56:34 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <1349921426.3341.175.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> Message-ID: <5076C202.7080202@schetterer.org> Am 11.10.2012 04:10, schrieb Christoph Anton Mitterer: > 3) Is AOX suitable for the local server? > - I couldn't use maildir locally, because I loose just to much space to > the block fragmentation. But I guess I won't have this problem with the DB backend. > > > a) Is AOX able to manage a really big folder hierarchy that basically > ever keeps growing... with easily several 100k > mails per folder... and that is in total already over 100GB? > > I read that e.g. dovecot would have more performance problems with that. > > > b) I would prefer to have fast full text search. Does AOX provide > this? > > I read that IMAP has limitations which make full text search not really usable via it? > > > Further, I remember some dovecot wiki page that showed a comparison which said > that both do not perfectly implement imap. Christoph, sorry, what exact is AOX, and what is its relation to the dovecot list.... youre looking to an overall mail setup so split your questions up to software you wanna use and ask the related mail list, perhaps hire some mail consultant advice you -- Best Regards MfG Robert Schetterer From robert at schetterer.org Thu Oct 11 15:57:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 11 Oct 2012 14:57:11 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup (ignore) In-Reply-To: <1349921914.3341.183.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> <1349921914.3341.183.camel@fermat.scientia.net> Message-ID: <5076C227.9040303@schetterer.org> Am 11.10.2012 04:18, schrieb Christoph Anton Mitterer: > Oops... that was obivously not intended for dovecot but AOX mailing > list,... where I ask around similar questions. > > Sorry for the noise =) > > > Cheers, > Chris. > ups answered exact about this *g -- Best Regards MfG Robert Schetterer From raabe at froglogic.com Thu Oct 11 16:03:55 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 11 Oct 2012 15:03:55 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <5076C202.7080202@schetterer.org> References: <1349921426.3341.175.camel@fermat.scientia.net> <5076C202.7080202@schetterer.org> Message-ID: <5076C3BB.1090307@froglogic.com> Am 11.10.2012 14:56, schrieb Robert Schetterer: > Am 11.10.2012 04:10, schrieb Christoph Anton Mitterer: >> 3) Is AOX suitable for the local server? [..] > Christoph, sorry, what exact is AOX, and what is its relation to the > dovecot list.... I suppose he meant Archiveopteryx (another IMAP server). -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dovecot at freakout.de Thu Oct 11 16:19:27 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Thu, 11 Oct 2012 15:19:27 +0200 (CEST) Subject: [Dovecot] iterate_query does not use userdb - mail_location not found Message-ID: <201210111319.q9BDJRV3005614@bongo.freakout.de> Hi dovecot-comminity, can't get iterate_query working. doveadm cannot find mail_location which comes from userdb query. dovecot itself works fine with sql. Whats wrong? please help: [root at glen exim]# /opt/dovecot/bin/doveadm search -A mailbox Trash savedbefore 90d doveadm(uwe at mitmachnet.de): Error: user uwe at mitmachnet.de: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=(not set) doveadm(uwe at mitmachnet.de): Error: User init failed dovecot.conf: passdb { driver = sql args = /etc/dovecot/sql.conf } userdb { driver = sql args = /etc/dovecot/sql.conf } sql.conf: driver = mysql connect = host=much dbname=toarx user=exim password=xxxxxxxx default_pass_scheme = PLAIN password_query = select user as username, password, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' user_query = select user as username, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' iterate_query = select email as user from vusers Cheers Axel From jbates at brightok.net Thu Oct 11 17:28:40 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 11 Oct 2012 09:28:40 -0500 Subject: [Dovecot] iterate_query does not use userdb - mail_location not found In-Reply-To: <201210111319.q9BDJRV3005614@bongo.freakout.de> References: <201210111319.q9BDJRV3005614@bongo.freakout.de> Message-ID: <5076D798.80102@brightok.net> On 10/11/2012 8:19 AM, dovecot at freakout.de wrote: > sql.conf: > driver = mysql > connect = host=much dbname=toarx user=exim password=xxxxxxxx > default_pass_scheme = PLAIN > password_query = select user as username, password, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' > user_query = select user as username, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' > iterate_query = select email as user from vusers > Only prefetch in the password_query prefixes with userdb_. Some things such as doveadm commands and lmtp delivery don't use passdb but userdb directly. The user_query should not have the userdb_ prefix. Also, according to the wiki, iterate_query returns username, not user. http://wiki2.dovecot.org/AuthDatabase/SQL Jack From calestyo at scientia.net Thu Oct 11 19:39:49 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 18:39:49 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <5076C3BB.1090307@froglogic.com> References: <1349921426.3341.175.camel@fermat.scientia.net> <5076C202.7080202@schetterer.org> <5076C3BB.1090307@froglogic.com> Message-ID: <1349973589.3370.4.camel@fermat.scientia.net> Hi. Sorry folks for the stupid postings... At first I posted what should go to the AOX list accidentally here and then our institute's MTA hat some issues yesterday, so the mail[0] where I already tried to explain the wrong posting, came much earlier than the wrong post itself. Guess you see why I need a better mail system ;) Sorry, Chris. [0] http://dovecot.org/pipermail/dovecot/2012-October/068740.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From btb at bitrate.net Thu Oct 11 20:35:00 2012 From: btb at bitrate.net (btb) Date: Thu, 11 Oct 2012 13:35:00 -0400 Subject: [Dovecot] imap proxy setup - "killed with signal 11" Message-ID: <50770344.70905@bitrate.net> hi- i'm setting up an imap proxy in front of a novell groupwise server. it seems to so far be partially working, but dovecot is having trouble in certain cases. i expect that it's ultimately due to what i believe is a very poor implementation of imap provided by groupwise [at least based on other experiences in the past] - but that's a big part of why i'd like to have dovecot in between it and clients. below is information collected during starting of dovecot, opening/initial connection from a client [os x mail.app], closing of the client, and stopping of dovecot. os is ubuntu 12.10 development/beta, dovecot is 2.1.7 courtesy of ubuntu's packages. log entries: Oct 11 13:24:33 halo dovecot: master: Dovecot v2.1.7 starting up Oct 11 13:24:49 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14171, TLS, session= Oct 11 13:24:50 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14174, TLS, session= Oct 11 13:24:51 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14176, TLS, session= Oct 11 13:24:51 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14178, TLS, session= Oct 11 13:24:51 halo dovecot: imap(jdoe): Connection closed in=16 out=350 Oct 11 13:24:52 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14180, TLS, session= Oct 11 13:24:52 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:52 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14176 killed with signal 11 (core dumped) Oct 11 13:24:57 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:57 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14178 killed with signal 11 (core dumped) Oct 11 13:24:57 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14182, TLS, session= Oct 11 13:24:58 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:58 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14180 killed with signal 11 (core dumped) Oct 11 13:25:03 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:03 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14182 killed with signal 11 (core dumped) Oct 11 13:25:03 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14184, TLS, session= Oct 11 13:25:03 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:03 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14184 killed with signal 11 (core dumped) Oct 11 13:25:09 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:09 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14174 killed with signal 11 (core dumped) [repeats] Oct 11 13:25:27 halo dovecot: dns-client: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: dns-client: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: master: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: auth: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: config: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: ssl-params: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: anvil: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: log: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) gdb backtrace: ~ >gdb /usr/lib/dovecot/imap /var/cache/imapproxy/jdoe/core GNU gdb (GDB) 7.5-ubuntu Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug/usr/lib/dovecot/imap...done. done. [New LWP 13939] warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `dovecot/imap'. Program terminated with signal 11, Segmentation fault. #0 0x0000000000000000 in ?? () (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00007fc7f6cb611e in imap_parser_reset (parser=0x7fc7f8a0f3a0) at imap-parser.c:93 No locals. #2 0x00007fc7f6f7ada7 in imapc_connection_input_reset (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:664 No locals. #3 0x00007fc7f6f7c6f4 in imapc_connection_input_untagged (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:908 imap_args = 0x7fc7f8a0f4f8 name = 0x7fc7f8a0f5d0 "" value = parser = 0x0 reply = {name = 0x7fc7f8a0f5d0 "", num = 11, args = 0x7fc7f8a0f4f8, file_args = 0x7fc7f8a0d5d0, file_args_count = 0, resp_text_key = 0x0, resp_text_value = 0x0, untagged_box_context = 0x7fc7f8a1ad70} ret = #4 0x00007fc7f6f7d25e in imapc_connection_input_one (conn=0x7fc7f8a0d270) at imapc-connection.c:1061 tag = 0x7fc7f8a0f5c0 "" ret = -1 #5 imapc_connection_input_pending (conn=0x7fc7f8a0d270) at imapc-connection.c:1407 _data_stack_cur_id = 6 ret = #6 0x00007fc7f6f7d2c2 in imapc_connection_input (conn=0x7fc7f6c8f798) at imapc-connection.c:1100 errstr = ret = #7 0x00007fc7f6cdf006 in io_loop_call_io (io=0x7fc7f8a23800) at ioloop.c:379 ioloop = 0x7fc7f8a23630 t_id = 5 #8 0x00007fc7f6cdfcb7 in io_loop_handler_run (ioloop=ioloop at entry=0x7fc7f8a23630) at ioloop-epoll.c:213 ctx = 0x7fc7f8a23850 events = 0x7fc7f8a245e0 event = 0x7fc7f8a238c0 list = 0x7fc7f8a24320 io = tv = {tv_sec = 299, tv_usec = 999402} events_count = msecs = ret = 1 i = call = #9 0x00007fc7f6cdea18 in io_loop_run (ioloop=0x7fc7f8a23630) at ioloop.c:398 No locals. #10 0x00007fc7f6f7a0f7 in imapc_client_run_pre (client=) at imapc-client.c:142 connp = 0x7fc7f8a0cfe0 prev_ioloop = 0x7fc7f89e3670 #11 imapc_client_run (client=0x7fc7f8a0ce80) at imapc-client.c:161 No locals. #12 0x00007fc7f6f79254 in imapc_storage_run (storage=0x7fc7f8a0be60) at imapc-storage.c:118 No locals. #13 0x00007fc7f6f78311 in imapc_sync_index (ctx=0x7fc7f8a1fc70) at imapc-sync.c:351 mbox = 0x7fc7f8a1ad70 sync_rec = {uid1 = 47, uid2 = 0, type = (unknown: 4171272512), add_flags = 199 '\307', remove_flags = 127 '\177', keyword_idx = 0, guid_128 = "\000\000\000\000)\235\312\366\307\177\000\000p\255\241", } seq1 = 32767 seq2 = 0 #14 imapc_sync_begin (force=, ctx_r=, mbox=0x7fc7f8a1ad70) at imapc-sync.c:422 ctx = 0x7fc7f8a1fc70 sync_flags = ret = #15 imapc_sync (mbox=0x7fc7f8a1ad70) at imapc-sync.c:464 sync_ctx = force = #16 imapc_mailbox_sync_init (box=0x7fc7f8a1ad70, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at imapc-sync.c:498 mbox = 0x7fc7f8a1ad70 capabilities = changes = false ret = #17 0x00007fc7f6f8bd43 in mailbox_sync_init (box=0x7fc7f8a1ad70, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at mail-storage.c:1320 _data_stack_cur_id = 4 ctx = #18 0x00007fc7f6f8be67 in mailbox_sync (box=, flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at mail-storage.c:1368 ctx = status = {sync_delayed_expunges = 0} #19 0x00007fc7f74475d2 in select_open (readonly=false, mailbox=, ctx=0x7fc7f8a0da98) at cmd-select.c:296 client = 0x7fc7f8a13d30 status = {messages = 4171084000, recent = 32711, unseen = 0, uidvalidity = 0, uidnext = 13, first_unseen_seq = 0, first_recent_uid = 4137782496, last_cached_seq = 32711, highest_modseq = 13, keywords = 0x7fc7f6cd06fb , permanent_flags = 4171118192, nonpermanent_modseqs = 1, permanent_keywords = 1, allow_new_keywords = 1} flags = MAILBOX_FLAG_DROP_RECENT ret = #20 cmd_select_full (cmd=, readonly=) at cmd-select.c:419 ---Type to continue, or q to quit--- client = ctx = args = 0x7fc7f8a18598 list_args = 0x7fc7f89db0e0 mailbox = 0x7fc7f89db310 "Trash" ret = 1 __FUNCTION__ = "cmd_select_full" #21 0x00007fc7f744b29c in command_exec (cmd=cmd at entry=0x7fc7f8a0d9a0) at imap-commands.c:148 hook = 0x7fc7f89e4cd0 ret = #22 0x00007fc7f744a2ee in client_command_input (cmd=0x7fc7f6c8f798) at imap-client.c:682 client = 0x7fc7f8a13d30 command = __FUNCTION__ = "client_command_input" #23 0x00007fc7f744a39a in client_command_input (cmd=0x7fc7f8a0d9a0) at imap-client.c:733 client = 0x7fc7f8a13d30 command = __FUNCTION__ = "client_command_input" #24 0x00007fc7f744a5fd in client_handle_next_command (remove_io_r=, client=0x7fc7f8a13d30) at imap-client.c:774 size = 19 #25 client_handle_input (client=client at entry=0x7fc7f8a13d30) at imap-client.c:786 _data_stack_cur_id = 3 ret = 112 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #26 0x00007fc7f744aef5 in client_input (client=0x7fc7f8a13d30) at imap-client.c:825 cmd = output = 0x7fc7f8a0d868 bytes = 19 __FUNCTION__ = "client_input" #27 0x00007fc7f6cdf006 in io_loop_call_io (io=0x7fc7f8a11c90) at ioloop.c:379 ioloop = 0x7fc7f89e3670 t_id = 2 #28 0x00007fc7f6cdfcb7 in io_loop_handler_run (ioloop=ioloop at entry=0x7fc7f89e3670) at ioloop-epoll.c:213 ctx = 0x7fc7f89e39e0 events = 0x7fc7f8a245e0 event = 0x7fc7f89e3a50 list = 0x7fc7f8a0f380 io = tv = {tv_sec = 1739, tv_usec = 996790} events_count = msecs = ret = 1 i = call = #29 0x00007fc7f6cdea18 in io_loop_run (ioloop=0x7fc7f89e3670) at ioloop.c:398 No locals. #30 0x00007fc7f6ccb463 in master_service_run (service=0x7fc7f89e3520, callback=callback at entry=0x7fc7f7452f70 ) at master-service.c:544 No locals. #31 0x00007fc7f74428c7 in main (argc=1, argv=0x7fc7f89e3370) at main.c:389 set_roots = {0x7fc7f7658d20 , 0x0} login_set = {auth_socket_path = 0x7fc7f89db070 "/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x7fc7f7452e10 , failure_callback = 0x7fc7f7452b60 } service_flags = storage_service_flags = username = 0x0 c = (gdb) config: >doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-17-generic x86_64 Ubuntu quantal (development branch) first_valid_gid = 999 first_valid_uid = 999 imapc_host = backend.example.com last_valid_gid = 999 last_valid_uid = 999 log_timestamp = "%d.%m.%Y %H.%M.%S " login_greeting = dovecot ready mail_gid = imapproxy mail_home = /var/cache/imapproxy/%u mail_location = imapc:%h/%n/Maildir mail_uid = imapproxy passdb { args = host=backend.example.com default_fields = userdb_imapc_user=%u userdb_imapc_password=%w driver = imap } protocols = " imap" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } ssl = required ssl_cert = References: <50757A8F.4030508@adminlinux.com.br> <5B596E8E-0721-4EE9-81E2-FC7F308B3C5C@iki.fi> Message-ID: <50771D40.5000105@rename-it.nl> On 10/10/2012 9:37 PM, Timo Sirainen wrote: > On 10.10.2012, at 16.39, 3.listas at adminlinux.com.br wrote: > >> I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. >> >> There is a lot of logs of "quota exceeded" like this: >> >> Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) >> >> These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. >> >> Is there a way to send specific Dovecot errors on specific files or just discard them? > Upgrade to v2.1, they are logged with info level there. (I think v2.0 also logs them with info level if you don't use Sieve.) Nope, that is unfortunately not going to help right now. At least not until the following change I made yesterday is released: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/5c1ce25596ed Of course you can patch it if you're in a hurry. Regards, Stephan. From tss at iki.fi Thu Oct 11 22:48:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 22:48:34 +0300 Subject: [Dovecot] memory allocation in new thread In-Reply-To: <1349950869.22650094@f123.mail.ru> References: <1349950869.22650094@f123.mail.ru> Message-ID: <66C27C1C-8C0F-4835-9E73-CB5D22DFCB3F@iki.fi> On 11.10.2012, at 13.21, . . wrote: > Hi! I have some problems with memory allocation. > I create new thread in cidir storage and call malloc(), and it fails to allocate even 1 byte. > What can cause this problem? No idea. Dovecot in general isn't designed to work with threads. > Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc: malloc() succeeded > Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc_func: malloc() failed Well, or the one thing I can think of you to try: Set default_vsz_limit=0 Also straceing the process could show what exactly fails. From tss at iki.fi Thu Oct 11 22:52:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 22:52:01 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210110643.q9B6hIF6003121@bongo.freakout.de> References: <201210110643.q9B6hIF6003121@bongo.freakout.de> Message-ID: On 11.10.2012, at 9.43, dovecot at freakout.de wrote: > Core was generated by `/opt/dovecot/sbin/dovecot'. OK.. > #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 > #1 0xb7f131c0 in master_instance_list_add_line (list=0x9d48880, line=0x9d540c8 "1349762052\tdovecot\t/var/dovecot/run\t") > at master-instance.c:85 Dovecot code is calling str_to_time() from libmysqlclient, instead of from Dovecot's internal code. Not the first time mysql conflicted with Dovecot code. This could be worked around, but .. why is your dovecot binary linked with libmysqlclient? Only auth and dict binaries should be. From tss at iki.fi Thu Oct 11 23:10:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 23:10:56 +0300 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <50752C97.1010209@froglogic.com> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> Message-ID: <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> On 10.10.2012, at 11.06, Frerich Raabe wrote: > I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: > > namespace public { > separator = / > prefix = Lists/ > location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > subscriptions = no > } > > Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) You'll need v2.2 and its INDEXPVT setting. From stephan at rename-it.nl Thu Oct 11 23:35:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 11 Oct 2012 22:35:21 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50753E85.5060904@dada.eu> References: <50753E85.5060904@dada.eu> Message-ID: <50772D89.4050601@rename-it.nl> On 10/10/2012 11:23 AM, Sandro Tosi wrote: > Hello, > we're scouting if it's possible to use Pigeonhole (currently v0.3.1, > as this will be provided with an upcoming Debian package) with MySQL > dict lookups with the mail setup we're designing. > > Our (main) goals are: > > 1. store the filters on the database That is possible with some limitations. > 2. allow each user to enable/disable any of the filters set we provide > (it's a static set of some general filters, available to all the > users; we're currently not providing the possibility to users to write > their own filters) Will one or multiple scripts be active at the same time? > For point 1) we already see[1] that's possible, but it uses the map > construct that might not fit with our current database structure: we > have a domain table (storing the domain info) and a mailbox table > (storing the mailbox info, but the username is composed by the local > part, stored in this table, and the domain part is a FK to the domain > table, using an id). > > Do you think it's possible to run a join query on domain+mailbox to > retrieve the mailbox_id needed to query the table for the filters? Or > do we have to create the filter table and store the local at domain.ext > info there ("relaxing" the integrity relationships between tables)? My SQL is a bit rusty, but afaik this is possible with a JOIN or a nested query. > How do we specify which filters are enabled for any given user? We > originally thought of an "Enabled" field on the filter table, but in > the example in the doc[1] I hadn't seen a way to do that: it seems > like the filter list is specified in the proxy definition - am I > wrong? How can we do that? The above suggests that you would like to activate multiple Sieve scripts at the same time. That is currently not possible with the dict Script location. It is on my TODO list, but I am not sure when it will be ready (definitely not for coming Debian stable). For Dovecot v2.2 the new :optional tag for the Sieve include command could be used in - combination with the dict Sieve script location type - to provide some hackish solution. Unfortunately, in your case that is still not helpful, because v2.2 is not even in beta stage. :/ Regards, Stephan. From raabe at froglogic.com Thu Oct 11 23:38:51 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 11 Oct 2012 22:38:51 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> Message-ID: Am 11.10.2012 um 22:10 schrieb Timo Sirainen: > On 10.10.2012, at 11.06, Frerich Raabe wrote: >> I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: >> >> namespace public { >> separator = / >> prefix = Lists/ >> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >> subscriptions = no >> } >> >> Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) > > You'll need v2.2 and its INDEXPVT setting. Hm, you mean the feature introduced by http://hg.dovecot.org/dovecot-2.2/rev/dbd42f7198eb ? Is there some discussion of the feature somewhere? The commit log is a bit unclear to me, it says 'Per-user flags can now be stored in private index files.' however http://wiki2.dovecot.org/SharedMailboxes/Public says 'By making each user have their own private index files, you can make the \Seen flag private for the users.' (using the INDEX setting). Makes me wonder - the Wiki talks about 'private index files' when talking about 'INDEX' and the commit says 'private index files' talking about INDEXPVT - what is the difference? :-) -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dave.mehler at gmail.com Thu Oct 11 23:40:11 2012 From: dave.mehler at gmail.com (David Mehler) Date: Thu, 11 Oct 2012 16:40:11 -0400 Subject: [Dovecot] per-user quotas Message-ID: Hello, I've got quotas set up on an all-user basis on my system, it's a Postfix, Dovecot, Mysql virtual users setup. Currently I have each user getting a 1GB quota with these settings in 90-quota.conf: plugin { quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M } plugin { quota = maildir:User quota } While this works it's not what I want for all users. Say I add a virtual user called user1 to the mysql database but he's a test user and I only want that user to have 25 megabytes of storage, reading the wiki on quotas per-user items such as for example in a database overrides the global items above, is this right? If so, I'm hoping I'm not going to have to redo my entire user database, some users will have per-user quotas while I'll let others have the global quota. Thanks. Dave. From tss at iki.fi Thu Oct 11 23:46:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 23:46:01 +0300 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> Message-ID: <99F7FC09-E21F-4808-9796-E2AAC31CDED0@iki.fi> On 11.10.2012, at 23.38, Frerich Raabe wrote: > Am 11.10.2012 um 22:10 schrieb Timo Sirainen: >> On 10.10.2012, at 11.06, Frerich Raabe wrote: >>> I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: >>> >>> namespace public { >>> separator = / >>> prefix = Lists/ >>> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >>> subscriptions = no >>> } >>> >>> Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) >> >> You'll need v2.2 and its INDEXPVT setting. > > Hm, you mean the feature introduced by http://hg.dovecot.org/dovecot-2.2/rev/dbd42f7198eb ? Yes. > Is there some discussion of the feature somewhere? http://markmail.org/message/45jxf363ffrubonv has some. > The commit log is a bit unclear to me, it says 'Per-user flags can now be stored in private index files.' however http://wiki2.dovecot.org/SharedMailboxes/Public says 'By making each user have their own private index files, you can make the \Seen flag private for the users.' (using the INDEX setting). > > Makes me wonder - the Wiki talks about 'private index files' when talking about 'INDEX' and the commit says 'private index files' talking about INDEXPVT - what is the difference? :-) You can have both! Shared indexes having the shared stuff (including squat indexes), while the private indexes only have the per-user flags, nothing else. For example with sdbox/mdbox you couldn't even have set per-user INDEX location or it would have just broken. From tibby at tibby.hu Thu Oct 11 23:49:11 2012 From: tibby at tibby.hu (Tibby) Date: Thu, 11 Oct 2012 22:49:11 +0200 Subject: [Dovecot] per-user quotas In-Reply-To: References: Message-ID: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> Hello! What's the version of your dovecot? 1.2.X ? or 2.0 ? Tibby On Oct 11, 2012, at 10:40 PM, David Mehler wrote: > Hello, > > I've got quotas set up on an all-user basis on my system, it's a > Postfix, Dovecot, Mysql virtual users setup. Currently I have each > user getting a 1GB quota with these settings in 90-quota.conf: > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } > plugin { > quota = maildir:User quota > } > > While this works it's not what I want for all users. Say I add a > virtual user called user1 to the mysql database but he's a test user > and I only want that user to have 25 megabytes of storage, reading the > wiki on quotas per-user items such as for example in a database > overrides the global items above, is this right? > > If so, I'm hoping I'm not going to have to redo my entire user > database, some users will have per-user quotas while I'll let others > have the global quota. > > Thanks. > Dave. From dave.mehler at gmail.com Fri Oct 12 00:43:57 2012 From: dave.mehler at gmail.com (David Mehler) Date: Thu, 11 Oct 2012 17:43:57 -0400 Subject: [Dovecot] per-user quotas In-Reply-To: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> References: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> Message-ID: Hello, I'm running Dovecot 2.1.10. Thanks. Dave. On 10/11/12, Tibby wrote: > Hello! > > What's the version of your dovecot? 1.2.X ? or 2.0 ? > > Tibby > > On Oct 11, 2012, at 10:40 PM, David Mehler wrote: > >> Hello, >> >> I've got quotas set up on an all-user basis on my system, it's a >> Postfix, Dovecot, Mysql virtual users setup. Currently I have each >> user getting a 1GB quota with these settings in 90-quota.conf: >> >> plugin { >> quota_rule = *:storage=1G >> quota_rule2 = Trash:storage=+100M >> } >> plugin { >> quota = maildir:User quota >> } >> >> While this works it's not what I want for all users. Say I add a >> virtual user called user1 to the mysql database but he's a test user >> and I only want that user to have 25 megabytes of storage, reading the >> wiki on quotas per-user items such as for example in a database >> overrides the global items above, is this right? >> >> If so, I'm hoping I'm not going to have to redo my entire user >> database, some users will have per-user quotas while I'll let others >> have the global quota. >> >> Thanks. >> Dave. > > From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 02:01:36 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 01:01:36 +0200 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074CAC2.2030507@perkel.com> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> <5074CAC2.2030507@perkel.com> Message-ID: <20121011230136.GA9153@daniel.localdomain> Hi Marc, Marc Perkel wrote: > On 10/9/2012 3:42 PM, Daniel Parthey wrote: > >Marc Perkel wrote: > >>if the mail location doesn't exist > >>then I want to try a second mail location: > >>mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >You might do this with a script which exports the MAIL environment > >variable and then executes the service binary: > >http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection > But how do I pick up the name and domain parameters to test the directory? You can get the username and any other userdb value from the environment, have a look at the environment section: http://wiki2.dovecot.org/PostLoginScripting#Running_environment Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 02:24:08 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 01:24:08 +0200 Subject: [Dovecot] Feature Request In-Reply-To: <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> <5074E01B.8030001@perkel.com> <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> Message-ID: <20121011232408.GA9444@daniel.localdomain> Timo Sirainen wrote: > On 10.10.2012, at 5.40, Marc Perkel wrote: > > >>> It would be handy (for me) if there were a userdb where a directory structure defined the db. > >>> > >>> userdb stat { > >>> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >>> } > >>> > >>> userdb stat { > >>> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >>> } > >>> > >>> The idea being that if the first directory doesn't exist then it will try the second one. > >> You could already implement this as userdb checkpassword script. > > > > Can you give me an example? > > Something like this: > > userdb { > driver = checkpasword > args = /usr/local/bin/userdb.sh > } Here is the documentation about how to implement a checkpassword script: http://wiki2.dovecot.org/AuthDatabase/CheckPassword Dovecot sets some environment variables that the script may use. All of the AUTH_* variables are available as AUTH_ extra fields in the environment: http://wiki2.dovecot.org/Variables#line-30 Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 03:15:39 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 02:15:39 +0200 Subject: [Dovecot] per-user quotas In-Reply-To: References: Message-ID: <20121012001539.GA10473@daniel.localdomain> Hi Dave, David Mehler wrote: > I've got quotas set up on an all-user basis on my system, it's a > Postfix, Dovecot, Mysql virtual users setup. Currently I have each > user getting a 1GB quota with these settings in 90-quota.conf: > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } > plugin { > quota = maildir:User quota > } > > While this works it's not what I want for all users. Say I add a > virtual user called user1 to the mysql database but he's a test user > and I only want that user to have 25 megabytes of storage, reading the > wiki on quotas per-user items such as for example in a database > overrides the global items above, is this right? > > If so, I'm hoping I'm not going to have to redo my entire user > database, some users will have per-user quotas while I'll let others > have the global quota. You can just make your SQL query a bit more sophisticated in order to fit your needs. MySQL supports SQL CASE statement and default value with ELSE: http://dev.mysql.com/doc/refman/5.1/en/case.html This example sets quota to unlimited if mail comes in via port 20025, otherwise is uses the quota_bytes and quota_message columns: user_query = SELECT username AS user, \ home as home, \ uid as uid, \ gid as gid, \ CASE '%a' \ WHEN '20025' THEN '*:bytes=0:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota_bytes AS CHAR), ':messages=', CAST(quota_message AS CHAR)) \ END AS `quota_rule` \ FROM dovecot_users \ WHERE username='%u' Regards Daniel -- https://plus.google.com/103021802792276734820 From dovecot at freakout.de Fri Oct 12 09:50:16 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Fri, 12 Oct 2012 08:50:16 +0200 (CEST) Subject: [Dovecot] dovecot cores Message-ID: <201210120650.q9C6oGSG005182@bongo.freakout.de> According to Timo Sirainen: > On 11.10.2012, at 9.43, dovecot at freakout.de wrote: > > Dovecot code is calling str_to_time() from libmysqlclient, > instead of from Dovecot's internal code. > Not the first time mysql conflicted with Dovecot code. > This could be worked around, but .. why is your dovecot > binary linked with libmysqlclient? > Only auth and dict binaries should be. > but dovecot's configure script does not allow to specify the mysql libs and headers explictly - only by global CPPFLAGS and LDFLAGS extensions, which are used for all binaries - when i tried to specify: ./configure --prefix=/opt/dovecot --sysconfdir=/etc/dovecot --mandir=/opt/dovecot/man \ --docdir=/opt/dovecot/doc --libexecdir=/opt/dovecot/sbin --datadir=/opt/dovecot \ --with-rundir=/var/dovecot/run --with-statedir=/var/dovecot/state \ --with-mysql=/opt/mysql/bin/mysql_config checking for ... checking pkg-config is at least version 0.9.0... yes configure: error: --with-mysql=path not supported. You may want to use instead: CPPFLAGS=-I/opt/mysql/bin/mysql_config/include LDFLAGS=-L/opt/mysql/bin/mysql_config/lib ./configure --with-mysql i followed the hint from the configure script above and run into the core-dumps due to symbol clash str_to_time. How to work around with mysql in non-standard location? Thanks Axel From dovecot-list at mohtex.net Fri Oct 12 10:25:59 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 14:25:59 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120650.q9C6oGSG005182@bongo.freakout.de> References: <201210120650.q9C6oGSG005182@bongo.freakout.de> Message-ID: <5077C607.3010308@mohtex.net> dovecot at freakout.de wrote the following on 12.10.2012 13:50: > According to Timo Sirainen: >> On 11.10.2012, at 9.43, dovecot at freakout.de wrote: >> >> Dovecot code is calling str_to_time() from libmysqlclient, >> instead of from Dovecot's internal code. >> Not the first time mysql conflicted with Dovecot code. >> This could be worked around, but .. why is your dovecot >> binary linked with libmysqlclient? >> Only auth and dict binaries should be. >> > but dovecot's configure script does not allow to specify the > mysql libs and headers explictly - only by global CPPFLAGS and > LDFLAGS extensions, which are used for all binaries - when i > tried to specify: > > ./configure --prefix=/opt/dovecot --sysconfdir=/etc/dovecot --mandir=/opt/dovecot/man \ > --docdir=/opt/dovecot/doc --libexecdir=/opt/dovecot/sbin --datadir=/opt/dovecot \ > --with-rundir=/var/dovecot/run --with-statedir=/var/dovecot/state \ > --with-mysql=/opt/mysql/bin/mysql_config > checking for ... > checking pkg-config is at least version 0.9.0... yes > configure: error: --with-mysql=path not supported. You may want to use instead: > CPPFLAGS=-I/opt/mysql/bin/mysql_config/include LDFLAGS=-L/opt/mysql/bin/mysql_config/lib ./configure --with-mysql > > i followed the hint from the configure script above and run > into the core-dumps due to symbol clash str_to_time. > > How to work around with mysql in non-standard location? > > Thanks > Axel This one works for me for mysql in a non-standard location (my.cnf is in /etc): CPPFLAGS='-I/opt/mysql/include/mysql' LDFLAGS='-L/opt/mysql/lib/mysql -lmysqlclient -lz -lcrypt -lnsl -lm' ./configure --with-mysql ..... Rds Tamsy From tss at iki.fi Fri Oct 12 10:34:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:34:08 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120702.q9C72saS005717@bongo.freakout.de> References: <201210120702.q9C72saS005717@bongo.freakout.de> Message-ID: <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > According to Timo Sirainen: >> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? >> > > ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" > LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" -lmysqlclient shouldn't be in LDFLAGS. > if i omit "-lmysqlclient" (seems to be the reason for the hassle) i get: > > libtool: link: gcc4 -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include -o .libs/auth auth.o auth-cache.o auth-client-connection.o > auth-master-connection.o auth-postfix-connection.o mech-otp-skey-common.o mech-plain-common.o auth-penalty.o auth-request.o auth-request-handler.o auth-settings.o auth-stream.o auth-worker-client.o auth-worker-server.o db-checkpassword.o db-dict.o db-sql.o db-passwd-file.o main.o mech.o mech-anonymous.o mech-plain.o mech-login.o > mech-cram-md5.o mech-digest-md5.o mech-external.o mech-gssapi.o mech-ntlm.o mech-otp.o mech-scram-sha1.o mech-skey.o mech-rpa.o mech-apop.o mech-winbind.o passdb.o passdb-blocking.o passdb-bsdauth.o passdb-cache.o passdb-checkpassword.o passdb-dict.o passdb-passwd.o passdb-passwd-file.o passdb-pam.o passdb-shadow.o passdb-sia.o > passdb-vpopmail.o passdb-sql.o passdb-static.o passdb-template.o userdb.o userdb-blocking.o userdb-checkpassword.o userdb-dict.o userdb-nss.o userdb-passwd.o userdb-passwd-file.o userdb-prefetch.o userdb-static.o userdb-vpopmail.o userdb-sql.o userdb-template.o db-ldap.o passdb-ldap.o userdb-ldap.o -Wl,--export-dynamic -L/opt > /zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib libpassword.a ../lib-ntlm/libntlm.a ../lib-otp/libotp.a ../../src/lib-sql/.libs/libsql.a ../../src/lib-dovecot/.libs/libdovecot.so -lcrypt -ldl -Wl,-rpath -Wl,/opt/dovecot/lib > ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': > /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' >>>> ... tons of other undefined reference to mysqlclient > /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' > collect2: error: ld returned 1 exit status I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. From tss at iki.fi Fri Oct 12 10:39:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:39:07 +0300 Subject: [Dovecot] imap proxy setup - "killed with signal 11" In-Reply-To: <50770344.70905@bitrate.net> References: <50770344.70905@bitrate.net> Message-ID: <560FF304-5D04-44D1-AEC2-8DE8DC3F0943@iki.fi> On 11.10.2012, at 20.35, btb wrote: > i'm setting up an imap proxy in front of a novell groupwise server. it seems to so far be partially working, but dovecot is having trouble in certain cases. i expect that it's ultimately due to what i believe is a very poor implementation of imap provided by groupwise [at least based on other experiences in the past] - but that's a big part of why i'd like to have dovecot in between it and clients. > > os is ubuntu 12.10 development/beta, dovecot is 2.1.7 courtesy of ubuntu's packages. There have been a couple of imapc fixes since v2.1.7. It's possible that the crash is fixed by one of them. > Oct 11 13:24:52 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Could you get imapc rawlogs where this happens? Point imapc_rawlog_dir setting to some directory. > #0 0x0000000000000000 in ?? () > No symbol table info available. > #1 0x00007fc7f6cb611e in imap_parser_reset (parser=0x7fc7f8a0f3a0) at imap-parser.c:93 > No locals. > #2 0x00007fc7f6f7ada7 in imapc_connection_input_reset (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:664 > No locals. > #3 0x00007fc7f6f7c6f4 in imapc_connection_input_untagged (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:908 This backtrace unfortunately doesn't make it very clear what the problem is. I'd guess it's trying to use already freed memory (one such bug was already fixed). From tss at iki.fi Fri Oct 12 10:40:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:40:36 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5075881C.4060905@brightok.net> References: <5075881C.4060905@brightok.net> Message-ID: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> On 10.10.2012, at 17.37, Jack Bates wrote: > The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. > > My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. From tss at iki.fi Fri Oct 12 10:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:45:05 +0300 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: References: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> Message-ID: On 10.10.2012, at 15.17, Natanael Copa wrote: > On Wed, Oct 10, 2012 at 12:22 PM, Timo Sirainen wrote: >> On 10.10.2012, at 13.00, Natanael Copa wrote: >> >>> Running doveadm on Alpine Linux will show various underlinking errors: > >> With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. > > Why does it need to load all the plugins? Why not only try to load > those who are enabled? doveadm has two types of commands: mail commands and non-mail commands. The mail_plugins can add new mail features, and doveadm plugins can add more doveadm commands, which use the new mail features. But doveadm can also have plugins that add non-mail commands, which don't need anything in mail_plugins. So both cases would need to work.. From dovecot-list at mohtex.net Fri Oct 12 10:45:25 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 14:45:25 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> Message-ID: <5077CA95.1060506@mohtex.net> Timo Sirainen wrote the following on 12.10.2012 14:34: > On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > >> According to Timo Sirainen: >>> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? >>> >> ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" >> LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" > -lmysqlclient shouldn't be in LDFLAGS. > >> if i omit "-lmysqlclient" (seems to be the reason for the hassle) i get: >> >> libtool: link: gcc4 -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include -o .libs/auth auth.o auth-cache.o auth-client-connection.o >> auth-master-connection.o auth-postfix-connection.o mech-otp-skey-common.o mech-plain-common.o auth-penalty.o auth-request.o auth-request-handler.o auth-settings.o auth-stream.o auth-worker-client.o auth-worker-server.o db-checkpassword.o db-dict.o db-sql.o db-passwd-file.o main.o mech.o mech-anonymous.o mech-plain.o mech-login.o >> mech-cram-md5.o mech-digest-md5.o mech-external.o mech-gssapi.o mech-ntlm.o mech-otp.o mech-scram-sha1.o mech-skey.o mech-rpa.o mech-apop.o mech-winbind.o passdb.o passdb-blocking.o passdb-bsdauth.o passdb-cache.o passdb-checkpassword.o passdb-dict.o passdb-passwd.o passdb-passwd-file.o passdb-pam.o passdb-shadow.o passdb-sia.o >> passdb-vpopmail.o passdb-sql.o passdb-static.o passdb-template.o userdb.o userdb-blocking.o userdb-checkpassword.o userdb-dict.o userdb-nss.o userdb-passwd.o userdb-passwd-file.o userdb-prefetch.o userdb-static.o userdb-vpopmail.o userdb-sql.o userdb-template.o db-ldap.o passdb-ldap.o userdb-ldap.o -Wl,--export-dynamic -L/opt >> /zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib libpassword.a ../lib-ntlm/libntlm.a ../lib-otp/libotp.a ../../src/lib-sql/.libs/libsql.a ../../src/lib-dovecot/.libs/libdovecot.so -lcrypt -ldl -Wl,-rpath -Wl,/opt/dovecot/lib >> ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': >> /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' >>>>> ... tons of other undefined reference to mysqlclient >> /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' >> collect2: error: ld returned 1 exit status > I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). From tss at iki.fi Fri Oct 12 10:50:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:50:06 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <5077CA95.1060506@mohtex.net> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> <5077CA95.1060506@mohtex.net> Message-ID: <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> On 12.10.2012, at 10.45, Tamsy wrote: > Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". > > Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). If you run configure without adding the -lmysqlclient, what do you get with: egrep -i 'mysql|auth_libs|sql_libs' Makefile I guess the problem is that I shouldn't have copy&pasted the mysql detection code from php, and configure somehow passes successfully without actually setting any MYSQL_LIBS.. From dovecot-list at mohtex.net Fri Oct 12 11:04:46 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 15:04:46 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> <5077CA95.1060506@mohtex.net> <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> Message-ID: <5077CF1E.4060809@mohtex.net> Timo Sirainen wrote the following on 12.10.2012 14:50: > On 12.10.2012, at 10.45, Tamsy wrote: > >> Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". >> >> Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). > If you run configure without adding the -lmysqlclient, what do you get with: > > egrep -i 'mysql|auth_libs|sql_libs' Makefile > > I guess the problem is that I shouldn't have copy&pasted the mysql detection code from php, and configure somehow passes successfully without actually setting any MYSQL_LIBS.. Just ran configure without adding the -lmysqlclient (CPPFLAGS='-I/opt/mysql/include/mysql' LDFLAGS='-L/opt/mysql/lib/mysql -lz -lcrypt -lnsl -lm' ./configure --with-mysql.... egrep -i 'mysql|auth_libs|sql_libs' Makefile says: AUTH_LIBS = CPPFLAGS = -I/opt/mysql/include/mysql LDFLAGS = $(NOPLUGIN_LDFLAGS) -L/opt/mysql/lib/mysql -lz -lcrypt -lnsl -lm MYSQL_CFLAGS = MYSQL_CONFIG = NO MYSQL_LIBS = PGSQL_LIBS = SQL_LIBS = sql_drivers = mysql scan-build -o scan-reports ../configure --with-ldap=auto --with-pgsql=auto --with-mysql=auto --with-sqlite=auto --with-solr=auto --with-gssapi=auto --with-libwrap=auto; \ configure runs without error but make ends with: ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_get_error': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:469: undefined reference to `mysql_errno' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_get_field_value_binary': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:436: undefined reference to `mysql_fetch_lengths' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_fetch_fields': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:383: undefined reference to `mysql_num_fields' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:384: undefined reference to `mysql_fetch_fields' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_next_row': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:366: undefined reference to `mysql_fetch_row' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:370: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_free': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:351: undefined reference to `mysql_free_result' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_do_query': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:233: undefined reference to `mysql_query' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:237: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_query_s': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:315: undefined reference to `mysql_affected_rows' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:316: undefined reference to `mysql_store_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:321: undefined reference to `mysql_next_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:332: undefined reference to `mysql_free_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:327: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_exec': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:284: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_escape_string': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:274: undefined reference to `mysql_real_escape_string' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:268: undefined reference to `mysql_escape_string' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:87: undefined reference to `mysql_options' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:92: undefined reference to `mysql_ssl_set' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:110: undefined reference to `mysql_real_connect' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:123: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_deinit_v': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:226: undefined reference to `mysql_close' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_parse_connect_string': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:198: undefined reference to `mysql_init' collect2: ld returned 1 exit status make[3]: *** [auth] Error 1 make[3]: Leaving directory `/usr/local/src/dovecot-2.1.10/src/auth' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/dovecot-2.1.10/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/dovecot-2.1.10' make: *** [all] Error 2 From dovecot at freakout.de Fri Oct 12 11:10:20 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Fri, 12 Oct 2012 10:10:20 +0200 (CEST) Subject: [Dovecot] dovecot cores In-Reply-To: <5077CA95.1060506@mohtex.net> Message-ID: <201210120810.q9C8AK7V007314@bongo.freakout.de> According to Tamsy: > Timo Sirainen wrote the following on 12.10.2012 14:34: > > On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > > > >> According to Timo Sirainen: > >>> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? > >>> > >> ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" > >> LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" > > -lmysqlclient shouldn't be in LDFLAGS. > > > > I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. > Axel, please let us know whether one of these works: > "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or > MYSQL_LIBS=-lmysqlclient". > > Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and > only by putting -lmysqlclient in LDFLAGS as described before Dovecot > compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a > non-standart location). > SQL_LIBS=-lmysqlclient => not working AUTH_LIBS=-lmysqlclient => not working MYSQL_LIBS=-lmysqlclient \ LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib" \ ./configure --prefix=%{_prefix} \ --sysconfdir=%{_etcdir} --mandir=%{_mandir} --docdir=%{_docdir} --libexecdir=%{_sbindir} --datadir=%{_prefix} \ --with-rundir=/var/dovecot/run \ --with-statedir=/var/dovecot/state \ --with-mysql => WORKING BUT: [axel at joe rpm]$ ldd BUILD/dovecot-2.1.10-root/opt/dovecot-2.1.10-5/sbin/dovecot libdovecot.so.0 => /opt/dovecot/lib/libdovecot.so.0 (0x00993000) libgcc_s.so.1 => /opt/gcc4/lib/libgcc_s.so.1 (0x0092c000) libc.so.6 => /lib/libc.so.6 (0x00ebf000) >>>>! libmysqlclient.so.18 => /opt/mysql/lib/libmysqlclient.so.18 (0x001cc000) libdl.so.2 => /lib/libdl.so.2 (0x00ae3000) libssp.so.0 => /opt/ssp/lib/libssp.so.0 (0x0057b000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x002c6000) libstrings.so => /opt/mysql/lib/libstrings.so (0x0057e000) libz.so.1 => /opt/zlib/lib/libz.so.1 (0x00110000) libpthread.so.0 => /lib/libpthread.so.0 (0x00b08000) libm.so.6 => /lib/libm.so.6 (0x00135000) dovecot still seems to be linked with the mysqlclient! i have just compiled - not tried the binaries - the core dump occurs only in the night! > If you run configure without adding the -lmysqlclient, what do you get with: egrep -i 'mysql|auth_libs|sql_libs' Makefile [axel at joe dovecot-2.1.10]$ egrep -i 'mysql|auth_libs|sql_libs' Makefile AUTH_LIBS = -lcrypt -lmysqlclient CFLAGS = -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include LDFLAGS = $(NOPLUGIN_LDFLAGS) -L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib MYSQL_CFLAGS = MYSQL_CONFIG = NO MYSQL_LIBS = -lmysqlclient PGSQL_LIBS = SQL_LIBS = -lmysqlclient sql_drivers = mysql scan-build -o scan-reports ../configure --with-ldap=auto --with-pgsql=auto --with-mysql=auto --with-sqlite=auto --with-solr=auto --with-gssapi=auto --with-libwrap=auto; \ Cheers Axel From kjonca at o2.pl Fri Oct 12 11:01:49 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Fri, 12 Oct 2012 10:01:49 +0200 Subject: [Dovecot] [sieve] - counting headers Message-ID: <87wqyw6rv6.fsf@alfa.kjonca> In some of my maildrop filters I have rules with weighted scoring[1], but only to count headers (for example to count "Received:" header) ie. all these rules are of form "/pattern/:h,1" Can dovecot sieve do this? KJ [1] http://www.courier-mta.org/maildrop/maildropfilter.html -- http://blogdebart.pl/2012/06/24/hiena/ Wiesz, tryb tekstowy w Linuksie ma si? tak do DOSu jak F-117A do paralotni. (c) Dawid Kuroczko From busseniu at in.tum.de Fri Oct 12 17:10:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Fri, 12 Oct 2012 16:10:28 +0200 Subject: [Dovecot] INBOX.INBOX.Sent causes problems in 2.0 Message-ID: <507824D4.7080303@in.tum.de> Hi, we have our namespace rooted at "INBOX.". Sometimes, users have folders like "INBOX.INBOX.Sent" or "INBOX.INBOX.INBOX.Sent". I do not know why these folders are created; I suspect it is due to buggy MUAs. If a mailbox like INBOX.INBOX.Sent exists, then in some cases Dovecot counts all messages in INBOX twice. This behavior can be demonstrated as follows: 1) Create a folder named "INBOX.INBOX.Sent" (using the IMAP CREATE command) 2) Save a message to "INBOX" (in my example the message is 7 MB) 3) dovecot-quota contains: priv/quota/storage 7129025 priv/quota/messages 1 4) doveadm quota recalc -u user1 5) Now dovecot-quota contains priv/quota/storage 14258050 priv/quota/messages 2 I.e. the user will experience that his quota fills up very fast. "doveadm -f flow fetch -u user1 'mailbox guid' ALL" prints: mailbox=INBOX guid=040ce73645177850497d000040c59ffc mailbox=INBOX guid=040ce73645177850497d000040c59ffc However, "doveadm -f flow fetch -u user1 'mailbox guid' mailbox INBOX" prints the message only once: mailbox=INBOX guid=040ce73645177850497d000040c59ffc How can we prevent this kind of confusion? What I'd like most is prevent the creation of these weird folders. Most MUAs have problems listing them properly, especially if both "INBOX.Sent" and "INBOX.INBOX.Sent" exist. Dovecot 2.1 does not seem to count anything twice. Cheers, Christoph Config: # 2.0.21: /usr/local/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (6224) # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mail mail_plugins = quota mail_uid = vmail namespace default { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { quota = dict:ROOT::file:%h/dovecot-quota quota_rule = *:storage=5G } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } service imap { process_limit = 5000 } ssl_cert = Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From thefantaman at gmail.com Fri Oct 12 18:38:59 2012 From: thefantaman at gmail.com (thefantaman) Date: Fri, 12 Oct 2012 08:38:59 -0700 (PDT) Subject: [Dovecot] /var/run/dovecot/auth-userdb failed Message-ID: <1350056339814-38093.post@n4.nabble.com> Hi, I work on test server and if I send an email on log i read lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: /var/run/dovecot/auth-userdb, euid is not dir owner) This is my dovecot.conf: auth_mechanisms = plain login info_log_path = /var/log/dovecot listen = 0.0.0.0 log_path = /var/log/dovecot login_greeting = Dovecot IMAP Server ready. mail_location = maildir:/home/vmail/%d/%u mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-ldap.conf.ext.fabry driver = ldap } protocols = pop3 sieve imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = root group = root } } ssl_cert = HELP! I have installed sendmail, dovecot, and squirrel mail. The squirrel mail portion of it works just fine, but I would like to have Mozilla Thunderbird as a client. Whenever I try and connect to the server it says "Thunderbird failed to find the settings for your email account." We do have an MX record in DNS pointing to our server. We are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 We are doing this for a class project and it is due by Tuesday 10/16/2012.. Thanks, Justin From ben at indietorrent.org Fri Oct 12 20:12:57 2012 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 12 Oct 2012 13:12:57 -0400 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <50784F99.5080201@indietorrent.org> On 10/12/2012 1:09 PM, Justin Vore wrote: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email account." > We do have an MX record in DNS pointing to our server. We are using > Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 > We are doing this for a class project and it is due by Tuesday 10/16/2012.. > > > Thanks, > > > Justin > I have found Thunderbird's automatic setting detection mechanism to be rather unreliable. Try entering the settings manually. -Ben From arne at drlinux.no Fri Oct 12 20:20:46 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Fri, 12 Oct 2012 19:20:46 +0200 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <5078516E.2040505@drlinux.no> Den 12.10.2012 19:09, skrev Justin Vore: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email account." > We do have an MX record in DNS pointing to our server. We are using > Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 > We are doing this for a class project and it is due by Tuesday 10/16/2012.. This explains how to set up autoconfigure in TB. You need a DNS record and a little work on a webhost. https://developer.mozilla.org/en-US/docs/Thunderbird/Autoconfiguration Arne -- Arne K. Haaje http://www.drlinux.no/ ::: arne at drlinux.no LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From lists at kokelnet.de Fri Oct 12 22:48:03 2012 From: lists at kokelnet.de (Tobias Hachmer) Date: Fri, 12 Oct 2012 21:48:03 +0200 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: Am 12.10.2012 19:09, schrieb Justin Vore: > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email > account." > We do have an MX record in DNS pointing to our server. Enter settings manually or set up automx (http://www.automx.org/). Regards, Tobias Hachmer From tss at iki.fi Fri Oct 12 23:07:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 23:07:08 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120810.q9C8AK7V007314@bongo.freakout.de> References: <201210120810.q9C8AK7V007314@bongo.freakout.de> Message-ID: <371D7F4F-3534-4F52-B106-165A487E1828@iki.fi> On 12.10.2012, at 11.10, dovecot at freakout.de wrote: > MYSQL_LIBS=-lmysqlclient \ > LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib" \ > ./configure --prefix=%{_prefix} \ > --sysconfdir=%{_etcdir} --mandir=%{_mandir} --docdir=%{_docdir} --libexecdir=%{_sbindir} --datadir=%{_prefix} \ > --with-rundir=/var/dovecot/run \ > --with-statedir=/var/dovecot/state \ > --with-mysql > => WORKING I think this is the correct fix for this. Basically same as what I committed now: http://hg.dovecot.org/dovecot-2.1/rev/c8d55ba25f39 > BUT: > > [axel at joe rpm]$ ldd BUILD/dovecot-2.1.10-root/opt/dovecot-2.1.10-5/sbin/dovecot > libdovecot.so.0 => /opt/dovecot/lib/libdovecot.so.0 (0x00993000) > libgcc_s.so.1 => /opt/gcc4/lib/libgcc_s.so.1 (0x0092c000) > libc.so.6 => /lib/libc.so.6 (0x00ebf000) >>>>> ! libmysqlclient.so.18 => /opt/mysql/lib/libmysqlclient.so.18 (0x001cc000) > libdl.so.2 => /lib/libdl.so.2 (0x00ae3000) > libssp.so.0 => /opt/ssp/lib/libssp.so.0 (0x0057b000) > /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x002c6000) > libstrings.so => /opt/mysql/lib/libstrings.so (0x0057e000) > libz.so.1 => /opt/zlib/lib/libz.so.1 (0x00110000) > libpthread.so.0 => /lib/libpthread.so.0 (0x00b08000) > libm.so.6 => /lib/libm.so.6 (0x00135000) > > dovecot still seems to be linked with the mysqlclient! Maybe the old LDFLAGS was cached somehow? Or maybe the rpm build does something strange? I don't see how that could happen otherwise. From mark at xwax.org Sat Oct 13 00:20:03 2012 From: mark at xwax.org (Mark Hills) Date: Fri, 12 Oct 2012 22:20:03 +0100 (BST) Subject: [Dovecot] dsync ignores ssh-agent Message-ID: <1210122159360.19545@vega.localdomain> I use IMAP over SSH, in pre-auth. I wanted to use dsync to offline mail to my laptop. dsync v2.1.10 would always ask for my SSH key/passphrase, ignoring ssh-agent. The culprit is the env_clean() in the stack below. Reading the source, I saw DOVECOT_PRESERVE_ENVS. When used as follows in my script it enables dsync to find my ssh-agent: export DOVECOT_PRESERVE_ENVS="SSH_AGENT_PID SSH_AUTH_SOCK" dsync mirror ssh imap.example.com /home/mark/opt/dovecot/bin/dsync and it now works without asking for password every time. I'm posting here so that anyone else googling for the same problem will hopefully find this, as I couldn't find anything about this in the docs. Also I'm interested in why dsync so aggressively cleans the environment; I tried a naive removal of env_clean() but this breaks basic functions. With this dsync is working very well for offline mail -- combined with alpine and a local exim for the outbound queue :) Thanks -- Mark Breakpoint 2, env_clean () at env-util.c:59 59 if (clearenv() < 0) (gdb) bt #0 env_clean () at env-util.c:59 #1 0xb7df10fc in master_service_env_clean () at master-service.c:454 #2 0xb7df26d4 in master_service_exec_config (service=0x809e7d0, input=0xbffff7e4) at master-service-settings.c:103 #3 0xb7df29be in config_exec_fallback (service=0x809e7d0, input=0xbffff7e4) at master-service-settings.c:153 #4 0xb7df2b65 in master_service_open_config (service=0x809e7d0, input=0xbffff7e4, path_r=0xbffff780, error_r=0xbffff7d8) at master-service-settings.c:206 #5 0xb7df3130 in master_service_settings_read (service=0x809e7d0, input=0xbffff7e4, output_r=0xbffff7dc, error_r=0xbffff7d8) at master-service-settings.c:345 #6 0x0805c672 in doveadm_read_settings () at doveadm.c:275 #7 0x0805c7d6 in main (argc=5, argv=0x809e1c0) at doveadm.c:342 From gedalya at gedalya.net Sat Oct 13 03:45:29 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 12 Oct 2012 20:45:29 -0400 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <5078B9A9.6040707@gedalya.net> This has nothing to do with dovecot or with any server. Thunderbird tries to guess settings such as your IMAP and SMTP server addresses, ports, TLS, authentication scheme, etc. Sometimes none of the guess attempts matches your settings. That's not an actual problem. Just enter the settings manually. If your priority is to make Thunderbird quickly and automatically configure your email accounts, read here https://wiki.mozilla.org/Thunderbird:Autoconfiguration Note that this is Thunderbird-specific, Microsoft Outlook, Blackberry etc. each have their own autoconfiguration schemes. On 10/12/2012 01:09 PM, Justin Vore wrote: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email > account." We do have an MX record in DNS pointing to our server. We > are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version > 8.14.4-2ubuntu2 We are doing this for a class project and it is due by > Tuesday 10/16/2012.. > > > Thanks, > > > Justin From robert at schetterer.org Sat Oct 13 08:31:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 13 Oct 2012 07:31:54 +0200 Subject: [Dovecot] Help! In-Reply-To: <5078B9A9.6040707@gedalya.net> References: <5078B9A9.6040707@gedalya.net> Message-ID: <5078FCCA.2090606@schetterer.org> Am 13.10.2012 02:45, schrieb Gedalya: > This has nothing to do with dovecot or with any server. > Thunderbird tries to guess settings such as your IMAP and SMTP server > addresses, ports, TLS, authentication scheme, etc. Sometimes none of the > guess attempts matches your settings. That's not an actual problem. Just > enter the settings manually. > > If your priority is to make Thunderbird quickly and automatically > configure your email accounts, read here > https://wiki.mozilla.org/Thunderbird:Autoconfiguration > > Note that this is Thunderbird-specific, Microsoft Outlook, Blackberry > etc. each have their own autoconfiguration schemes. you may use http://www.automx.org/ for that > > > On 10/12/2012 01:09 PM, Justin Vore wrote: >> HELP! >> >> I have installed sendmail, dovecot, and squirrel mail. The squirrel >> mail portion of it works just fine, but I would like to have Mozilla >> Thunderbird as a client. Whenever I try and connect to the server it >> says "Thunderbird failed to find the settings for your email >> account." We do have an MX record in DNS pointing to our server. We >> are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version >> 8.14.4-2ubuntu2 We are doing this for a class project and it is due by >> Tuesday 10/16/2012.. >> >> >> Thanks, >> >> >> Justin > -- Best Regards MfG Robert Schetterer From alessio at skye.it Sat Oct 13 11:16:27 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 10:16:27 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= Message-ID: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> Hi, I'm running dovecot 2.1.10 on Debian 6. When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] If I add -D to doveadm I can see this: doveadm(myuser at mydomain.com): Debug: Added userdb setting: plugin/quota_rule=*:backend=524288000S doveadm(myuser at mydomain.com): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/2/mydomain.com/myuser doveadm(myuser at mydomain.com): Debug: Quota root: name=UserQuota backend=maildir args= doveadm(myuser at mydomain.com): Debug: Quota rule: root=UserQuota mailbox=* bytes=524288000 messages=0 doveadm(myuser at mydomain.com): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 doveadm(myuser at mydomain.com): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota doveadm(myuser at mydomain.com): Debug: dict quota: user=myuser at mydomain.com, uri=proxy::quota, noenforcing=1 doveadm(myuser at mydomain.com): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir doveadm(myuser at mydomain.com): Debug: maildir++: root=/home/vpopmail/domains/2/mydomain.com/myuser/Maildir, index=, control=, inbox=/home/vpopmail/domains/2/mydomain.com/myuser/Maildir, alt= doveadm(myuser at mydomain.com): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 doveadm(myuser at mydomain.com): Debug: acl: acl username = myuser at mydomain.com doveadm(myuser at mydomain.com): Debug: acl: owner = 1 doveadm(myuser at mydomain.com): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls doveadm(myuser at mydomain.com): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u doveadm(myuser at mydomain.com): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= doveadm(myuser at mydomain.com): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 doveadm(myuser at mydomain.com): Debug: acl: acl username = myuser at mydomain.com doveadm(myuser at mydomain.com): Debug: acl: owner = 0 doveadm(myuser at mydomain.com): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=276 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=277 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=278 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=279 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=280 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=281 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=282 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=283 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=284 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=285 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=286 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=287 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=288 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=289 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=290 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=291 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=292 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=293 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=294 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=295 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=296 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=297 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=298 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=299 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=300 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=301 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=302 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=303 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=304 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=305 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=306 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=307 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=308 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=309 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=310 doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found doveadm(myuser at mydomain.com): Debug: Namespace : Using permissions from /home/vpopmail/domains/2/mydomain.com/myuser/Maildir: mode=0700 gid=-1 doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Drafts/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Spam/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Sent/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Trash/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/dovecot-acl not found Segmentation fault If I disable "acl" plugin in dovecot all works fine. I think that this problem is connected with "Dovecot deliver Segmentation fault when arrive the first message" http://www.dovecot.org/list/dovecot/2012-September/068343.html that I'm still experiencing. How can get core dumps from "doveadm"? This is my dovecot config with acl enabled: # 2.1.10: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /usr/local/dovecot-2.1/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } ssl_cert = References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> Message-ID: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> On 13.10.2012, at 11.16, Alessio Cecchi wrote: > I'm running dovecot 2.1.10 on Debian 6. > > When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" > > [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] The most helpful way to get this fixed is to get a gdb backtrace: http://dovecot.org/bugreport.html From alessio at skye.it Sat Oct 13 13:48:46 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 12:48:46 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= In-Reply-To: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> Message-ID: <9a747f967d6b70da5a1551a82a017112@skye.it> Il 2012-10-13 10:42 Timo Sirainen ha scritto: > On 13.10.2012, at 11.16, Alessio Cecchi wrote: > >> I'm running dovecot 2.1.10 on Debian 6. >> >> When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it >> crash with "Segmentation fault" >> >> [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 >> sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] > > The most helpful way to get this fixed is to get a gdb backtrace: > http://dovecot.org/bugreport.html Hi Timo, I'm unable to get core dump from doveadm, I start dovecot after run "ulimit -c unlimited" and set echo "/tmp/%p" > /proc/sys/kernel/core_pattern, so core dumps is enable: Oct 13 12:38:02 master: Info: Dovecot v2.1.10 starting up Oct 13 12:38:18 auth-worker(5000): Info: mysql(localhost): Connected to database vpopmail Oct 13 12:38:18 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:38:26 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:38:29 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:39:51 dict: Info: mysql(109.168.113.139): Connected to database dovecot but when doveadm stops with "Segmentation fault" I'm unable to find any dump file and no information in dovecot.log. Can you help me? Thanks From c at roessner-network-solutions.com Sat Oct 13 14:22:30 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Sat, 13 Oct 2012 13:22:30 +0200 Subject: [Dovecot] Help! In-Reply-To: <50784F99.5080201@indietorrent.org> References: <50784F99.5080201@indietorrent.org> Message-ID: Hi, >> I have installed sendmail, dovecot, and squirrel mail. The squirrel >> mail portion of it works just fine, but I would like to have Mozilla >> Thunderbird as a client. Whenever I try and connect to the server it >> says "Thunderbird failed to find the settings for your email account." >> We do have an MX record in DNS pointing to our server. We are using >> Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 >> We are doing this for a class project and it is due by Tuesday 10/16/2012.. > > I have found Thunderbird's automatic setting detection mechanism to be > rather unreliable. > > Try entering the settings manually. I am one of the automx developers. Have a look at http://www.automx.org. It is open source. I also finished setting up a test server, so you can try with mail address automx at automx.org, pw: automx and see how it works. Kind regards -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From alessio at skye.it Sat Oct 13 15:38:41 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 14:38:41 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> Il 2012-10-02 21:28 Timo Sirainen ha scritto: > On 19.9.2012, at 16.07, Alessio Cecchi wrote: > >> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > > This backtrace is rather weird. Could you also do (instead of bt > full): > > fr 1 > p *ns > p *ns.user > p *auser > > It crashes because auser->dict = NULL, but it should never be NULL. Hi Timo, this is a new backtrace: root at demo-vpop ~ # /home/vpopmail/bin/vadduser test160 at qboxdns.it qweqweroot at demo-vpop ~ # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test160 at qboxdns.it Segmentation fault (core dumped) root at demo-vpop ~ # gdb /usr/local/dovecot-2.1/libexec/dovecot/deliver /tmp/10923 GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/local/dovecot-2.1/libexec/dovecot/deliver...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-lda.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-lda.so.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libbz2.so.1.0...(no debugging symbols found)...done. Loaded symbols for /lib/libbz2.so.1.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib90_sieve_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib90_sieve_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-sieve.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-sieve.so.0 Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver -d test160 at qboxdns.it'. Program terminated with signal 11, Segmentation fault. #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 221 if (dict->dict == NULL) (gdb) fr 1 #1 0x00007f9edac761b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x2496520) at acl-backend-vfile-acllist.c:297 297 (void)acl_lookup_dict_rebuild(auser->acl_lookup_dict); (gdb) p *ns $1 = {next = 0x2496860, refcount = 1, type = NAMESPACE_PRIVATE, flags = 8235, prefix = 0x24961b0 "", prefix_len = 0, alias_for = 0x0, alias_chain_next = 0x0, user = 0x249a700, owner = 0x249a700, list = 0x249ef40, storage = 0x2496210, set = 0x249b4f0, unexpanded_set = 0x249ab58, mail_set = 0x249b200, destroyed = 0} (gdb) p *ns.user $2 = {pool = 0x249a6e0, v = {deinit = 0x7f9edac7a280 }, vlast = 0x249bf38, refcount = 1, username = 0x249a7b8 "test160 at qboxdns.it", _home = 0x249bb60 "/home/vpopmail/domains/qboxdns.it/test160", uid = 89, gid = 89, service = 0x249bb90 "lda", local_ip = 0x0, remote_ip = 0x0, var_expand_table = 0x249bb98, error = 0x0, set_info = 0x2482ce8, unexpanded_set = 0x249a7d0, set = 0x249b168, namespaces = 0x2496130, storages = 0x24a1e20, hooks = {arr = {buffer = 0x249beb0, element_size = 8}, v = 0x249beb0, v_modifiable = 0x249beb0}, mountpoints = 0x0, module_contexts = {arr = {buffer = 0x249bb00, element_size = 8}, v = 0x249bb00, v_modifiable = 0x249bb00}, home_looked_up = 1, admin = 0, autocreated = 0, initialized = 1, mail_debug = 0, inbox_open_error_logged = 0, fuzzy_search = 0, dsyncing = 0} (gdb) p *auser $3 = {module_ctx = {super = {deinit = 0x7f9edaa68190 }, reg = 0x7f9edaa68190}, master_user = 0x0, acl_env = 0x249bd88 "vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300", groups = 0x0, acl_lookup_dict = 0x0} (gdb) and also but full for safety: (gdb) bt full #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 ns = ids_arr = {arr = {buffer = 0x0, element_size = 38363440}, v = 0x0, v_modifiable = 0x0} ids = 0x24787e0 i = dest = ret = -601327851 #1 0x00007f9edac761b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x2496520) at acl-backend-vfile-acllist.c:297 auser = 0x249bf10 iter = 0x0 acllist_path = 0x24787e0 "/home/vpopmail/domains/qboxdns.it/test160/Maildir/dovecot-acl-list" ret = ns = 0x2496130 output = 0x0 st = {st_dev = 2051, st_ino = 663856, st_nlink = 1, st_mode = 33152, st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1350131151, tv_nsec = 0}, st_mtim = {tv_sec = 1350131151, tv_nsec = 0}, st_ctim = {tv_sec = 1350131151, tv_nsec = 0}, __unused = {0, 0, 0}} path = 0x24783a8 ---Type to continue, or q to quit--- file_mode = 384 dir_mode = 448 gid = 4294967295 list = info = rootdir = 0x24787a0 "Sent" origin = 0x249f4c0 "/home/vpopmail/domains/qboxdns.it/test160/Maildir" fd = 8 #2 acl_backend_vfile_acllist_rebuild (backend=0x2496520) at acl-backend-vfile-acllist.c:311 acllist_path = #3 0x00007f9edac76563 in acl_backend_vfile_acllist_refresh (backend=0x2496520) at acl-backend-vfile-acllist.c:153 __FUNCTION__ = "acl_backend_vfile_acllist_refresh" #4 0x00007f9edac766d5 in acl_backend_vfile_acllist_verify (backend=0x0, name=0x2496800 "", mtime=0) at acl-backend-vfile-acllist.c:343 acllist = #5 0x00007f9edac750b8 in acl_backend_vfile_object_refresh_cache ( _aclobj=0x24967c0) at acl-backend-vfile.c:858 old_validity = validity = {global_validity = {last_check = 0, last_read_time = 1350131151, last_mtime = 0, last_size = 0}, local_validity = {last_check = 0, last_read_time = 0, ---Type to continue, or q to quit--- last_mtime = 0, last_size = 0}, mailbox_validity = { last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}} mtime = 0 ret = 38387472 #6 0x00007f9edac7325e in acl_backend_get_default_rights (backend=0x2496520, mask_r=0x28) at acl-backend.c:164 No locals. #7 0x00007f9edac795bd in acl_mailbox_try_list_fast (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:107 alist = nonowner_list_ctx = ret = backend = 0x2496520 acl_mask = 0x1 ns = 0x2496130 update_ctx = {iter_ctx = 0x7f9edc4bf2c8, tree_ctx = 0x7f9edcbdda88, glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, update_only = 0, match_parents = 0} name = #8 acl_mailbox_list_iter_init (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 _data_stack_cur_id = 2 ---Type to continue, or q to quit--- ctx = 0x2498e60 pool = i = inboxcase = #9 0x00007f9edc538d33 in mailbox_list_iter_init_multiple (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at mailbox-list-iter.c:158 ctx = ret = __FUNCTION__ = "mailbox_list_iter_init_multiple" #10 0x00007f9edc539459 in mailbox_list_iter_init (list=0x0, pattern=, flags=1350131151) at mailbox-list-iter.c:58 patterns = {0x7f9edaa696dc "*", 0x0} #11 0x00007f9edaa64370 in quota_count_namespace (root=0x2496cb0, bytes_r=, count_r=0x7fff890373d0) at quota-count.c:73 ctx = 0x7f9edc270ef3 info = #12 quota_count (root=0x2496cb0, bytes_r=, count_r=0x7fff890373d0) at quota-count.c:111 i = 0 ret = 0 #13 0x00007f9edaa657ce in dict_quota_count (root=0x0, want_bytes=true, value_r=0x7fff89037418) at quota-dict.c:113 ---Type to continue, or q to quit--- dt = bytes = 0 count = 0 #14 0x00007f9edaa6595a in dict_quota_update_callback ( ret=, context=0x249bf10) at quota-dict.c:178 value = 1 #15 0x00007f9edc244258 in client_dict_finish_transaction (dict=0x249eb30, line_r=) at dict-client.c:265 ctx = 0x24da1b0 #16 client_dict_read_one_line (dict=0x249eb30, line_r=) at dict-client.c:356 id = 1 line = ret = 0 __FUNCTION__ = "client_dict_read_one_line" #17 0x00007f9edc244565 in client_dict_wait (_dict=) at dict-client.c:520 dict = 0x249eb30 line = 0x0 ret = #18 0x00007f9edaa65ab5 in dict_quota_deinit (_root=) at quota-dict.c:90 root = 0x2496cb0 ---Type to continue, or q to quit--- #19 0x00007f9edaa61c72 in quota_root_deinit (root=0x0) at quota.c:240 pool = 0x249e900 #20 0x00007f9edaa636e1 in quota_deinit (_quota=0x249bf40) at quota.c:335 quota = 0x2496940 i = 2 #21 0x00007f9edaa681dd in quota_user_deinit (user=0x249a700) at quota-storage.c:412 quser = 0x249bf38 quota_set = 0x2499270 #22 0x00007f9edc53388e in mail_user_unref (_user=) at mail-user.c:153 user = 0x249a700 __FUNCTION__ = "mail_user_unref" #23 0x0000000000402de2 in main (argc=3, argv=0x247e370) at main.c:481 set_roots = {0x604640, 0x0} ctx = {pool = 0x247ef70, set = 0x24817e8, session = 0x247ef90, dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, src_envelope_sender = 0x0, dest_user = 0x0, dest_addr = 0x247e3c2 "test160 at qboxdns.it", final_dest_addr = 0x247e3c2 "test160 at qboxdns.it", dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = true, saved_mail = false, save_dest_mail = false, mailbox_full = false, ---Type to continue, or q to quit--- dsn = false} service_flags = user = 0x247e3c2 "test160 at qboxdns.it" errstr = 0x0 path = 0x7fff89037748 "\351\a" storage_service = 0x24803b0 service_user = 0x2480d58 service_input = {module = 0x4034d5 "lda", service = 0x4034d5 "lda", username = 0x247e3c2 "test160 at qboxdns.it", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = 0x2496210 user_source = destaddr_source = 0x403594 "user at hostname" process_euid = stderr_rejection = false ---Type to continue, or q to quit--- ret = c = error = MAIL_ERROR_NONE (gdb) and this the dovecot configuration: # dovecot -n # 2.1.9: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota zlib acl maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=109.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } ssl_cert = References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <83B37619-1CE8-4C5D-8147-A3C0E1C99CDC@iki.fi> Message-ID: <6730bf79a50779c9bd33311e50ccce9e@skye.it> Il 2012-10-02 22:15 Timo Sirainen ha scritto: > On 2.10.2012, at 22.28, Timo Sirainen wrote: > >> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >> >>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 >> >> This backtrace is rather weird. Could you also do (instead of bt >> full): > > Also, can you reproduce the crash always by running "doveadm quota > recalc -u user at domain"? If first add a news user, than run quota recalc and after deliver the first message "deliver" not crash: # vpopmail/bin/vadduser test10 at qboxdns.it # doveadm quota recalc -u test110 at qboxdns.it # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test10 at qboxdns.it When add a new user without quota recalc deliver crash: # /home/vpopmail/bin/vadduser test12 at qboxdns.it # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test12 at qboxdns.it Segmentation fault (core dumped) # Hope this will useful From simon.buongiorno at gmail.com Sun Oct 14 05:07:29 2012 From: simon.buongiorno at gmail.com (simon.buongiorno at gmail.com) Date: Sat, 13 Oct 2012 22:07:29 -0400 Subject: [Dovecot] POP UIDL Message-ID: Hi I run a small mail server with Dovecot and postfix. Several of the accounts are popped by an external email provide I use, mostly out habit and also for a bit of redundancy. I have no details on that external server except that it uses exim. Lately, it's been repopping mail from accounts (mail is left on the server so I can use IMAP) at fairly frequent, but undetermined, intervals. Before I take it to them, I want to be sure it's not an error on my side. How can I be sure Dovecot does not have a problem with the UIDL list causing this external server to repop the mail? For the record, I'm not inclined to think it's a Dovecot issue, but since that's the bit I can fix, I'd like to be sure. Cheers Simon From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 16:52:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 15:52:18 +0200 Subject: [Dovecot] [sieve] - counting headers In-Reply-To: <87wqyw6rv6.fsf@alfa.kjonca> References: <87wqyw6rv6.fsf@alfa.kjonca> Message-ID: <20121014135218.GA7602@daniel.localdomain> Kamil Jo?ca wrote: > In some of my maildrop filters I have rules with weighted scoring[1], > but only to count headers (for example to count "Received:" header) > ie. all these rules are of form "/pattern/:h,1" > > Can dovecot sieve do this? http://tools.ietf.org/rfc/rfc5231.txt To check the number of received fields in the header, the following test may be used: header :count "ge" :comparator "i;ascii-numeric" ["received"] ["3"] Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 17:24:22 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 16:24:22 +0200 Subject: [Dovecot] /var/run/dovecot/auth-userdb failed In-Reply-To: <1350056339814-38093.post@n4.nabble.com> References: <1350056339814-38093.post@n4.nabble.com> Message-ID: <20121014142422.GA8080@daniel.localdomain> thefantaman wrote: > I work on test server and if I send an email on log i read > > lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: > Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: > /var/run/dovecot/auth-userdb, euid is not dir owner) > > unix_listener auth-userdb { > mode = 0600 > user = root > group = root > } > } The problem is that LDA (local delivery agent or lmtp service) is not able to look up the destination mailbox in userdb. The socket /var/run/dovecot/auth-userdb is currently only readable or writable by user root since mode is set to 0600, not readable or writable by other groups. http://wiki2.dovecot.org/LDA#Virtual_users You'll need to set up a auth-userdb socket for dovecot-lda so it knows where to find mailboxes for the users. LDA is running under the virtual mailbox user and group "vmail", so you need to grant this user or group access to /var/run/dovecot/auth-userdb. You could do this by using group memberships and set mode = 0660 or simply make it world-readable-writable with mode = 0666: unix_listener auth-userdb { mode = 0666 user = root group = root } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 17:38:30 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 16:38:30 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: <9a747f967d6b70da5a1551a82a017112@skye.it> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> Message-ID: <20121014143830.GA8425@daniel.localdomain> Alessio Cecchi wrote: > I'm unable to get core dump from doveadm, I start dovecot after run > "ulimit -c unlimited" and set echo "/tmp/%p" > > /proc/sys/kernel/core_pattern, so core dumps is enable: > > but when doveadm stops with "Segmentation fault" I'm unable to find > any dump file and no information in dovecot.log. On Debian try to enable coredumps in /etc/default/dovecot and start dovecot as usual via init script. Also watch out for core dumps and segfaults in /var/log/kern.log Regards Daniel -- https://plus.google.com/103021802792276734820 From alessio at skye.it Sun Oct 14 18:58:40 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sun, 14 Oct 2012 17:58:40 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= In-Reply-To: <20121014143830.GA8425@daniel.localdomain> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> <20121014143830.GA8425@daniel.localdomain> Message-ID: Il 2012-10-14 16:38 Daniel Parthey ha scritto: > Alessio Cecchi wrote: >> I'm unable to get core dump from doveadm, I start dovecot after run >> "ulimit -c unlimited" and set echo "/tmp/%p" > >> /proc/sys/kernel/core_pattern, so core dumps is enable: >> >> but when doveadm stops with "Segmentation fault" I'm unable to find >> any dump file and no information in dovecot.log. > > On Debian try to enable coredumps in /etc/default/dovecot and start > dovecot as usual via init script. Also watch out for core dumps > and segfaults in /var/log/kern.log Thanks, my dovecot installation is build from source. Dovecot start fine with core dumps enabled but doveadm don't return "Core dumped" when crash. From dave at boostpro.com Sun Oct 14 21:30:10 2012 From: dave at boostpro.com (Dave Abrahams) Date: Sun, 14 Oct 2012 14:30:10 -0400 Subject: [Dovecot] Search for substring in header? Message-ID: Hi, According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), if I do a search for "TO isocpp.org" it should find all the messages whose To: field contains the string "isocpp.org", but dovecot is returning me an empty list. However, a search for "TO tm at isocpp.org" produces a long list of messages. What am I doing wrong? TIA, Dave -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From slusarz at curecanti.org Mon Oct 15 04:59:50 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Sun, 14 Oct 2012 19:59:50 -0600 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: Message-ID: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Quoting Dave Abrahams : > Hi, > > According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), > if I do a search for "TO isocpp.org" it should find all the messages > whose To: field contains the string "isocpp.org", but dovecot is > returning me an empty list. However, a search for "TO tm at isocpp.org" > produces a long list of messages. What am I doing wrong? First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. Second, your assumption is correct - TO should do a substring search. But this works fine for me (using version 2.1.10). michael From sandro.tosi at dada.eu Mon Oct 15 10:40:48 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Mon, 15 Oct 2012 09:40:48 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50772D89.4050601@rename-it.nl> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> Message-ID: <507BBE00.9010007@dada.eu> Hi Stephan, thanks a lot for your reply. On 10/11/2012 10:35 PM, Stephan Bosch wrote: > On 10/10/2012 11:23 AM, Sandro Tosi wrote: >> Hello, >> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >> as this will be provided with an upcoming Debian package) with MySQL >> dict lookups with the mail setup we're designing. >> >> Our (main) goals are: >> >> 1. store the filters on the database > That is possible with some limitations. Are the ones below the only limitatios (ie one script per user) or are there any other worth knowing? >> 2. allow each user to enable/disable any of the filters set we provide >> (it's a static set of some general filters, available to all the >> users; we're currently not providing the possibility to users to write >> their own filters) > Will one or multiple scripts be active at the same time? Yep, the idea is that any user could have multiple scripts active at the same time, and we'd like also to give them an ordering, so like managing a sort of priority (the lower the priority the sooner the script is executed, or the other way around, doesn't matter). Ideally, we have a set of several scripts and each user can select to enable only some of them, and choose the order of their executions. >> For point 1) we already see[1] that's possible, but it uses the map >> construct that might not fit with our current database structure: we >> have a domain table (storing the domain info) and a mailbox table >> (storing the mailbox info, but the username is composed by the local >> part, stored in this table, and the domain part is a FK to the domain >> table, using an id). >> >> Do you think it's possible to run a join query on domain+mailbox to >> retrieve the mailbox_id needed to query the table for the filters? Or >> do we have to create the filter table and store the local at domain.ext >> info there ("relaxing" the integrity relationships between tables)? > > My SQL is a bit rusty, but afaik this is possible with a JOIN or a > nested query. Ah no well, I mean, using map { } constructs :) The example for Sieve-MySQL only shows 2 maps, but given we've never used them, we'd want to know if a "map cascade" would work, so implementing the joins in multiple steps: selecting the ids with a map and the subsequent would use that id to exec the join and so on. >> How do we specify which filters are enabled for any given user? We >> originally thought of an "Enabled" field on the filter table, but in >> the example in the doc[1] I hadn't seen a way to do that: it seems >> like the filter list is specified in the proxy definition - am I >> wrong? How can we do that? > > The above suggests that you would like to activate multiple Sieve > scripts at the same time. That is currently not possible with the dict > Script location. It is on my TODO list, but I am not sure when it will > be ready (definitely not for coming Debian stable). I see, I think that some others would wonder the same, so you might also want to extend the doc to state that explicitly. Maybe you may want to include something in your TODO list to handle the ordering in case of multiple scripts. In our situation, what would you suggest? We're now thinking of keeping the scripts list on a separate table, and merge the "user selected ones" in a single script to write in the filters table. Is that what would you suggest? Is there a better solution? Cheers, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From stocton12 at yahoo.com Mon Oct 15 15:46:09 2012 From: stocton12 at yahoo.com (b m) Date: Mon, 15 Oct 2012 05:46:09 -0700 (PDT) Subject: [Dovecot] (no subject) Message-ID: <1350305169.43664.YahooMailNeo@web125703.mail.ne1.yahoo.com> Hi. I'm using dovecot 2.0.18 and I'm trying to authenticate through a CAS server (until now authentication was through MS Active Directory). I could not find anywhere some examples, so here is what i have done so far. -install phpcas and pam_cas -edit /etc/pam.d/dovecot ????????????????? auth??? sufficient????? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf -edit /etc.pam_cas.conf ????????????????? host mycas.mydomain.com ????????????????? port 443 ????????????????? uriValidate /cas/proxyValidate ????????????????? ssl on ????????????????? proxy ??????????????????????? ????????????????? trusted_ca /etc/cert/certificate.pem ????????????????? debug on - and finally dovecot.conf which I'm sure is complety wrong ????????????? userdb { ? ? ? ? ?? ?? args = /etc/dovecot/dovecot-ldap.conf ? ? ? ? ?? ?? driver = ldap ????????????? } ???????????? passdb { ???????????? driver = pam ? ? ? ? ? ?? args = cache_key=%u dovecot ???????????? } What I get in log is Oct 15 15:39:58 auth-worker: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): lookup service=dovecot Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): #1/1 style=1 msg=Password: Oct 15 15:39:58 auth-worker: Info: pam(user,127.0.0.1): pam_authenticate() failed: Permission denied Oct 15 15:40:00 auth: Debug: client out: FAIL??? 1??? user=user Oct 15 15:40:00 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 15 15:40:00 auth: Debug: auth client connected (pid=9019) Any ideas? Thanks. From linuxpencil at hotmail.com Mon Oct 15 16:01:04 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 09:01:04 -0400 Subject: [Dovecot] Can't Start Dovecot Message-ID: Hi; I just installed dovecot from yum on CentOS5. ps wax grep "dovecot" only brings up the grep The command "dovecot" is not recognized. # ls /usr/local/bin/dove* doveadm doveconf No dovecot. What up? TIA, John From s.lazzaris at interactive.eu Mon Oct 15 16:13:45 2012 From: s.lazzaris at interactive.eu (Simone Lazzaris) Date: Mon, 15 Oct 2012 15:13:45 +0200 Subject: [Dovecot] Plugin hooks in login process Message-ID: <1947528.35zxeZD9k1@orion> Hi all; I've setup dovecot (2.1.10) in a cluster configuration. We have two servers acting as frontend which authenticates users and proxy them to other two servers which handles the "real" work. Users credentials are on a mysql cluster; we have one master, in which read/write queries are processed, and many replicated slave, which process read-only queries. The frontend servers reads users credentials from the read-only mysql slaves. I'd like to execute a query once the client is verified to update the last login data. Right now, that query is executed on the backend servers, via a post-login service: protocols = imap service imap-postlogin { executable = script-login /usr/local/etc/dovecot/postlogin.sh unix_listener imap-postlogin { group = vchkpw mode = 0600 user = vpopmail } } service imap { executable = imap imap-postlogin process_limit = 2048 } Problem is, if I execute the update on the backend, I miss the information regarding the original IP, as I only see the IP of the proxies. I haven't been able to launch the postlogin service on the frontend, so I figured that I can try to write a plugin - that also seems to me the cleanest solution. Looking in the dovecot source code, I noticed that there aren't any hooks in the execution path used by the proxies; I am missing something ? I am the only one missing the presence of this hooks in the auth/proxy process ? I've also thought of a workaround for this problem. One way is to monitor the dovecot logs on the frontend and execute the update asyncronously. One other way is to query directly the main mysql server of the cluster and adjust the query making it call a stored procedure that updates the information in case of successful login. But I'd really prefer to create a plugin, that I'd be willing to share. I attach the configuration of the servers (front and back) generated via postfix -n. Thanks in advance for any help. -- Simone Lazzaris | Responsabile aree datacenter e VoIP Interactive Network srl | via Roggia Vignola 9, 24047 Treviglio (BG) Tel. 0363 1970352 | Fax 0363.1971971 | www.interactive.eu -------------- next part -------------- # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686-bigmem i686 Debian 6.0.2 auth_debug = yes auth_verbose = yes auth_verbose_passwords = plain base_dir = /var/run/dovecot/ default_login_user = nobody director_doveadm_port = 9091 director_mail_servers = AAA.BBB.CCC.DDD EEE.FFF.GGG.HHH director_servers = XXX.YYY.ZZZ.WWW disable_plaintext_auth = no listen = * log_path = /var/log/dovecot passdb { args = /usr/local/etc/dovecot/sql.conf driver = sql } protocols = imap service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director service_count = 0 } ssl_cert = From dave at boostpro.com Mon Oct 15 16:23:08 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 06:23:08 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: on Sun Oct 14 2012, Michael M Slusarz wrote: > Quoting Dave Abrahams : > >> Hi, >> >> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >> if I do a search for "TO isocpp.org" it should find all the messages >> whose To: field contains the string "isocpp.org", but dovecot is >> returning me an empty list. However, a search for "TO tm at isocpp.org" >> produces a long list of messages. What am I doing wrong? > > First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. Thanks for pointing me to the right one. > Second, your assumption is correct - TO should do a substring search. > But this works fine for me (using version 2.1.10). Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the same empty result with either of these two commands: UID SEARCH TO isocpp.org UID SEARCH TO "isocpp.org" Am I formatting the command wrongly? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From linuxpencil at hotmail.com Mon Oct 15 16:37:09 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 09:37:09 -0400 Subject: [Dovecot] Can't Start Dovecot In-Reply-To: References: Message-ID: Never mind. The command /etc/init.d/dovecot start would work; however, something else is using the port. Tracking it down. John > From: linuxpencil at hotmail.com > To: dovecot at dovecot.org > Date: Mon, 15 Oct 2012 09:01:04 -0400 > Subject: [Dovecot] Can't Start Dovecot > > > > Hi; > I just installed dovecot from yum on CentOS5. > ps wax grep "dovecot" only brings up the grep > The command "dovecot" is not recognized. > # ls /usr/local/bin/dove* > doveadm doveconf > No dovecot. What up? > TIA, > John > From h.reindl at thelounge.net Mon Oct 15 16:42:00 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 15 Oct 2012 15:42:00 +0200 Subject: [Dovecot] Can't Start Dovecot In-Reply-To: References: Message-ID: <507C12A8.1040702@thelounge.net> netstat --numeric-hosts --numeric-ports --programs -u -t -l will list all listening ports and as root also the exectueable /etc/init.d/dovecot star is they way to go never start a service by it's binary without a good reason without knowing exactly how it is supposed to work BTW: the dovecot binary lives in /sbin/ not /bin/ [root at srv:~]$ ps aux | grep dovecot root 1843 0.0 0.0 19548 1520 ? Ss 11:58 0:00 /usr/sbin/dovecot -F i am generally wonder about /usr/local as you said you installed with yum - typically distributions packages are using /usr/bin, /usr/sbin/ and not /usr/local Am 15.10.2012 15:37, schrieb John Reddy: > Never mind. The command > /etc/init.d/dovecot start > would work; however, something else is using the port. Tracking it down. > John > >> From: linuxpencil at hotmail.com >> To: dovecot at dovecot.org >> Date: Mon, 15 Oct 2012 09:01:04 -0400 >> Subject: [Dovecot] Can't Start Dovecot >> >> >> >> Hi; >> I just installed dovecot from yum on CentOS5. >> ps wax grep "dovecot" only brings up the grep >> The command "dovecot" is not recognized. >> # ls /usr/local/bin/dove* >> doveadm doveconf >> No dovecot. What up? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From list at airstreamcomm.net Mon Oct 15 17:36:11 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 15 Oct 2012 09:36:11 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> Message-ID: <507C1F5B.2060002@airstreamcomm.net> On 10/12/12 2:40 AM, Timo Sirainen wrote: > On 10.10.2012, at 17.37, Jack Bates wrote: > >> The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. >> >> My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. > I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. > > +1 for adding this detail to logging for LMTP. From dave at boostpro.com Mon Oct 15 18:08:59 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 08:08:59 -0700 Subject: [Dovecot] fts = squat solr Message-ID: I don't know if this was supposed to have changed with dovecot2, but http://wiki.dovecot.org/Plugins/FTS shows fts = squat solr so, since I have the lucene plugin?"fts = lucene" works by itself?I tried fts = squat lucene but: $ doveadm index '*' doveadm(dave): Error: fts: Failed to initialize backend 'squat lucene': Unknown backend So, is that syntax obsolete, is the wiki wrong, or am I doing something wrong? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Mon Oct 15 18:36:30 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 08:36:30 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: on Mon Oct 15 2012, Dave Abrahams wrote: > on Sun Oct 14 2012, Michael M Slusarz wrote: > >> Quoting Dave Abrahams : >> >>> Hi, >>> >>> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >>> if I do a search for "TO isocpp.org" it should find all the messages >>> whose To: field contains the string "isocpp.org", but dovecot is >>> returning me an empty list. However, a search for "TO tm at isocpp.org" >>> produces a long list of messages. What am I doing wrong? >> >> First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. > > Thanks for pointing me to the right one. > >> Second, your assumption is correct - TO should do a substring search. >> But this works fine for me (using version 2.1.10). > > Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the > same empty result with either of these two commands: > > UID SEARCH TO isocpp.org > > UID SEARCH TO "isocpp.org" > > Am I formatting the command wrongly? Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the same result. baffled-ly y'rs, -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From howellrepaja at gmail.com Mon Oct 15 18:43:25 2012 From: howellrepaja at gmail.com (Howell Repaja) Date: Mon, 15 Oct 2012 23:43:25 +0800 Subject: [Dovecot] Dovecot Authentication Problem Can't Make it Work Message-ID: Hi All, I am struggling for 2 weeks solving authentication problem in dovecot. logs from /etc/mail/maillog Oct 15 18:00:35 localhost dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 18:00:35 localhost dovecot: auth: Debug: auth client connected (pid=26723) Oct 15 18:00:35 localhost dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011lip=10.0.0.123#011rip=88.22.197.66#011lport=110#011rport=2358#011resp=AGhvd2VsbEB0b3VyZm9yeW91LmluZm8AanVtb25n Oct 15 18:00:35 localhost dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 18:00:35 localhost dovecot: auth: Debug: pam(howell at mydomain.info,88.22.33.66): lookup service=dovecot Oct 15 18:00:35 localhost dovecot: auth: Debug: pam(howell at mydomain.info,88.22.197.66): #1/1 style=1 msg=Password: Oct 15 18:00:37 localhost dovecot: auth: pam(howell at mydomain.info,88.22.197.66): unknown user Oct 15 18:00:39 localhost dovecot: auth: Debug: client out: FAIL#0111#011user=howell at mydomain.info#011reason=Password : Oct 15 18:00:39 localhost dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=>, method=PLAIN, rip=88.22.197.66, lip=10.0.0.123 Oct 15 18:01:05 localhost sendmail[26722]: q9FA15LB026722: [88.22.197.66] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.el6.x86_64 x86_64 CentOS release 6.3 (Final) ext4 auth_debug_passwords = yes auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb disable_plaintext_auth = no last_valid_gid = 10 last_valid_uid = 650 listen = * login_greeting = Dovecot ready for you. mail_debug = yes mail_location = mbox:/var/spool/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = setcred=yes failure_show_msg=yes cache_key=%u dovecot driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } service imap-login { inet_listener imap { port = 143 } } service pop3-login { inet_listener pop3 { port = 110 } } ssl_cert = Hi.I'm very sorry for the repost but I forgot the subject. So,? I'm using dovecot 2.0.18 and I'm trying to authenticate through a CAS server (until now authentication was through MS Active Directory). I could not find anywhere some examples, so here is what i have done so far. -install phpcas and pam_cas -edit /etc/pam.d/dovecot ????????????????? auth??? sufficient????? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf -edit /etc.pam_cas.conf ????????????????? host mycas.mydomain.com ????????????????? port 443 ????????????????? uriValidate /cas/proxyValidate ????????????????? ssl on ????????????????? proxy ??????????????????????? ????????????????? trusted_ca /etc/cert/certificate.pem ????????????????? debug on - and finally dovecot.conf which I'm sure is complety wrong ????????????? userdb { ? ? ? ? ?? ?? args = /etc/dovecot/dovecot-ldap.conf ? ? ? ? ?? ?? driver = ldap ????????????? } ???????????? passdb { ???????????? driver = pam ? ? ? ? ? ?? args = cache_key=%u dovecot ???????????? } What I get in log is Oct 15 15:39:58 auth-worker: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): lookup service=dovecot Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): #1/1 style=1 msg=Password: Oct 15 15:39:58 auth-worker: Info: pam(user,127.0.0.1): pam_authenticate() failed: Permission denied Oct 15 15:40:00 auth: Debug: client out: FAIL??? 1??? user=user Oct 15 15:40:00 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 15 15:40:00 auth: Debug: auth client connected (pid=9019) Any ideas? Thanks. From user+dovecot at localhost.localdomain.org Mon Oct 15 20:04:19 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 15 Oct 2012 19:04:19 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> <20121014143830.GA8425@daniel.localdomain> Message-ID: <507C4213.8000205@localhost.localdomain.org> On 10/14/2012 05:58 PM Alessio Cecchi wrote: > Thanks, my dovecot installation is build from source. Dovecot start > fine with core dumps enabled but doveadm don't return "Core dumped" when > crash. In your terminal emulator enter the following commands: ulimit -c unlimited doveadm ? Regards, Pascal -- The trapper recommends today: decade.1228919 at localdomain.org From linuxpencil at hotmail.com Mon Oct 15 21:10:12 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 14:10:12 -0400 Subject: [Dovecot] Where'd the Mail Go? Message-ID: Hi; I ran this: echo "Hello me" | mail -s "Dovecot test" $USER then created a bash script: for mbox in /var/mail/$USER /var/spool/mail/$USER ~/mbox ~/mail/* ~/*; do grep -q "Dovecot test" $mbox && echo "mbox: $mbox" done grep -q "Dovecot test" ~/Maildir/new/* 2>/dev/null && echo "Maildir: ~/Maildir" and ran it but couldn't find where the mail went. I checked /var/mail/my_user_name and it wasn't there, either. How do I find it? TIA, John From slusarz at curecanti.org Mon Oct 15 22:00:11 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 15 Oct 2012 13:00:11 -0600 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Quoting Dave Abrahams : > on Mon Oct 15 2012, Dave Abrahams wrote: > >> on Sun Oct 14 2012, Michael M Slusarz wrote: >> >>> Quoting Dave Abrahams : >>> >>>> Hi, >>>> >>>> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >>>> if I do a search for "TO isocpp.org" it should find all the messages >>>> whose To: field contains the string "isocpp.org", but dovecot is >>>> returning me an empty list. However, a search for "TO tm at isocpp.org" >>>> produces a long list of messages. What am I doing wrong? >>> >>> First, you referenced the wrong RFC - RFC 2060 has been obsoleted >>> by RFC 3501. >> >> Thanks for pointing me to the right one. >> >>> Second, your assumption is correct - TO should do a substring search. >>> But this works fine for me (using version 2.1.10). >> >> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >> same empty result with either of these two commands: >> >> UID SEARCH TO isocpp.org >> >> UID SEARCH TO "isocpp.org" >> >> Am I formatting the command wrongly? > > Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the > same result. Lucene for sure does not support subtext searching. Squat used to... but IIRC things may have changed for v2.1. Try the wiki. michael From jbates at brightok.net Mon Oct 15 22:07:07 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 15 Oct 2012 14:07:07 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> Message-ID: <507C5EDB.7050401@brightok.net> On 10/12/2012 2:40 AM, Timo Sirainen wrote: > would probably complicate the code. > I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. > > Commented logs below. I did 3 different types of connections. Let me know what you think. Because I'm logging the proxy host itself, it can be IP or name depending on the configuration. If you like it, want minor changes, additional logging, let me know and I'll adjust the code. As is, this is a one liner. Jack Oct 12 19:03:45 compiler dovecot: lmtp(18568): Connect from ::1 Connection succeeds using static proxy to lmtp.example.com (default in this config). Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from lmtp.example.com(test): 250 2.5.0 command succeeded Connection succeeds but user invalid using director mapping Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from 192.168.1.3(test2): 550 5.1.1 User doesn't exist: test2 Current error reporting Oct 12 19:04:14 compiler dovecot: lmtp(18568): Error: lmtp client: connect(192.168.1.4, 7025) failed: No route to host Connection failed and what we returned to client Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from 192.168.1.4(test42): 451 4.4.0 Remote server not answering (connect) Oct 12 19:04:16 compiler dovecot: lmtp(18568): Disconnect from ::1: Client quit (in reset) From tom at whyscream.net Mon Oct 15 22:56:19 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 15 Oct 2012 21:56:19 +0200 Subject: [Dovecot] Dovecot Authentication Problem Can't Make it Work In-Reply-To: References: Message-ID: <507C6A63.2000301@whyscream.net> On 15/10/12 17:43, Howell Repaja wrote: > Hi All, > > I am struggling for 2 weeks solving authentication problem in dovecot. > > logs from /etc/mail/maillog > Oct 15 18:00:37 localhost dovecot: auth: > pam(howell at mydomain.info,88.22.197.66): > unknown user > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-279.el6.x86_64 x86_64 CentOS release 6.3 (Final) ext4 > passdb { > args = setcred=yes failure_show_msg=yes cache_key=%u dovecot > driver = pam > } > userdb { > driver = passwd > } Pam says that you have no such user 'howell at tourforyou.info'. Pam most probably talks to /etc/passwd (and friends), which means that you either need to login with a valid valid system username listed in /etc/passwd, or you need to setup some other userdb/passdb that supports full email addresses as usernames. -- Tom From jbates at brightok.net Mon Oct 15 23:10:59 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 15 Oct 2012 15:10:59 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507C5EDB.7050401@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> Message-ID: <507C6DD3.2000309@brightok.net> On 10/15/2012 2:07 PM, Jack Bates wrote: > On 10/12/2012 2:40 AM, Timo Sirainen wrote: >> would probably complicate the code. >> I don't think this would be difficult to implement. Probably just a >> few lines of code. Yeah, could be useful. >> >> > If there's no argument over the last email, confirm and check this patch. It's not the overall logging I would like, but the lmtp code isn't as mature as pop3/imap and the proxy is a quick and dirty on the lmtp code. Both need a good revamp, preferably with x-session support and perhaps logging rip/lip similar to how we do pop3/imap logins. I think we should also work on adjusting all logging for services using x-session to also log the proxy ip. rip,lip,pip. As I get time I'll look at it. This patch is just to keep us from having no useful logging in lmtp proxy. Based on lmtp pid, one can at least follow the connect, the proxy replies, and the disconnect of a session. --- dovecot-2.1.10/src/lmtp/lmtp-proxy.c 2012-10-12 19:46:49.688952484 +0000 +++ dovecot-2.1.10/src/lmtp/lmtp-proxy.c-new 2012-10-12 19:48:51.751932325 +0000 @@ -160,6 +160,8 @@ static bool lmtp_proxy_send_data_replies break; o_stream_send_str(proxy->client_output, t_strconcat(rcpt[i]->reply, "\r\n", NULL)); + i_info("proxy(%s): proxy host=%s: status=%s",rcpt[i]->address, + rcpt[i]->conn->set.host,rcpt[i]->reply); } o_stream_uncork(proxy->client_output); proxy->next_data_reply_idx = i; From daniel.parthey at informatik.tu-chemnitz.de Tue Oct 16 00:27:46 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 15 Oct 2012 23:27:46 +0200 Subject: [Dovecot] Where'd the Mail Go? In-Reply-To: References: Message-ID: <20121015212746.GA8899@daniel.localdomain> Hi John, John Reddy wrote: > I ran this: > echo "Hello me" | mail -s "Dovecot test" $USER > and ran it but couldn't find where the mail went. I checked /var/mail/my_user_name and it wasn't there, either. How do I find it? I'm sorry to tell you this is rather off-topic on the dovecot list, since the mail is routed and delivered by your MTA. In the logfiles of your MTA (mail transport agent) you should find hints where the mail went. /var/log/postfix/... /var/log/exim/... Regards Daniel -- https://plus.google.com/103021802792276734820 From dave at boostpro.com Tue Oct 16 01:20:29 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 15:20:29 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Message-ID: on Mon Oct 15 2012, Michael M Slusarz wrote: > Quoting Dave Abrahams : > >> on Mon Oct 15 2012, Dave Abrahams wrote: >> >>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>> >>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>> same empty result with either of these two commands: >>> >>> UID SEARCH TO isocpp.org >>> >>> UID SEARCH TO "isocpp.org" >>> >>> Am I formatting the command wrongly? >> >> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >> same result. > > Lucene for sure does not support subtext searching. Squat used to... > but IIRC things may have changed for v2.1. Try the wiki. Sorry, but what does "try the wiki" mean? Which indexer are you using, that successfully finds the substring match? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 01:35:06 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 15:35:06 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search Message-ID: According to the IMAP spec if I do a search for "TO isocpp.org" it should find all the messages whose To: field contains the string "isocpp.org", but dovecot is returning me an empty list. However, a search for "TO tm at isocpp.org" produces a long list of messages. This behavior is present if I *even load* the lucene fts plugin. Note that lucene isn't in use (fts = squat); it's merely loaded. This behavior goes away if I don't load fts_lucene. Dovecot configuration with dovecot -n: --8<---------------cut here---------------start------------->8--- # 2.1.6: /usr/local/stow/dovecot-2.1.6/etc/dovecot/dovecot.conf # OS: Darwin 11.4.2 x86_64 hfs default_internal_user = _dovecot default_login_user = _dovenull mail_gid = 20 mail_location = mdbox:/Users/dave/Library/Data/LocalIMAP/mdbox mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = fts fts_squat fts_lucene zlib mail_uid = 501 maildir_very_dirty_syncs = yes namespace { inbox = yes location = prefix = separator = . subscriptions = yes type = private } passdb { args = uid=501 gid=20 home=/Users/dave nopassword=y driver = static } plugin { fts = squat zlib_save = gz zlib_save_level = 6 } protocols = imap ssl = no protocol imap { mail_plugins = fts fts_squat fts_lucene zlib } --8<---------------cut here---------------end--------------->8--- Dovecot version: 2.1.6 Operating system or Linux distribution name: MacOS X 10.7, 10.8 CPU architecture (x86 or something else?): x86_64 Filesystem you used (especially if you use NFS or not): Mac Some kind of description of what you were doing and with what IMAP client.: Searching -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From tss at iki.fi Tue Oct 16 03:09:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:09:03 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <20121005194847.GA15222@daniel.localdomain> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> <20121005194847.GA15222@daniel.localdomain> Message-ID: <5FF6D55F-3B53-4CAD-ACBA-FC334E09F159@iki.fi> On 5.10.2012, at 22.48, Daniel Parthey wrote: > Timo Sirainen wrote: >> -i changes to dovecot.conf used by the given instance name > > This does not seem to work, at least not with version 2.1.10: Fixed: http://hg.dovecot.org/dovecot-2.1/rev/0262ede193e5 From tss at iki.fi Tue Oct 16 03:12:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:12:14 +0300 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Message-ID: <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> On 4.10.2012, at 16.00, Benedetto Vassallo wrote: > All works fine, but with the new version it seems that dovecot don't do hardlinks when deliver a message to multiple users. The hard linking is done only when the directory permissions match. > mail_location = maildir:~/MailDir:LAYOUT=fs > > I tryed using lmtp directly issuing 'telnet localhost 24' and sending a test message to 3 recipients. > Then issuing a 'ls -il' in the "new" directory of that users, I saw the inode was not the same. What are the permissions of the MailDir directory for user1/user2? ls -ld /home/user1/MailDir ls -ld /home/user2/MailDir From tss at iki.fi Tue Oct 16 03:15:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:15:05 +0300 Subject: [Dovecot] Dovecot Hangs while mutile users download mail for same account using pop3 In-Reply-To: <793760c2702e89acc526a66c0b543293@Coptics.org> References: <793760c2702e89acc526a66c0b543293@Coptics.org> Message-ID: On 8.10.2012, at 18.12, Robert JR wrote: > I have a weird problem in dovecot, Dovecot Hangs while multiple users download mail for same account using pop3 > > Three persons use 1 same email , and three of them use outlook express to check > That specific mail .. Also some times one of the three users check the mail for this > Account using imap (squirrel mail) Make sure you have pop3_lock_session=no and.. > to=, orig_to=, relay=local, delay=357, delays=338/0.01/0/19, dsn=4.2.0, status=deferred (cannot update mailbox /var/mail/sales for user sales. unable to lock for exclusive access: Resource temporarily unavailable) The problem may simply be that you're using mbox format. POP3 protocol itself wasn't meant for simultaneous access (it's actually disallowed by the RFC) and with mbox format Dovecot optimizes it in a way that probably locks the mailbox exclusively for the whole duration of the session. From linuxpencil at hotmail.com Tue Oct 16 03:15:05 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 20:15:05 -0400 Subject: [Dovecot] Where'd the Mail Go? In-Reply-To: <20121015212746.GA8899@daniel.localdomain> References: , <20121015212746.GA8899@daniel.localdomain> Message-ID: > In the logfiles of your MTA (mail transport agent) you should find > hints where the mail went. Hmm. I guess I'll work on postfix then, and come back later ;) John From tss at iki.fi Tue Oct 16 03:35:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:35:57 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> Message-ID: <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> On 13.10.2012, at 15.38, Alessio Cecchi wrote: > Il 2012-10-02 21:28 Timo Sirainen ha scritto: >> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >> >>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 This should fix it: http://hg.dovecot.org/dovecot-2.1/rev/41aac09497ee From tss at iki.fi Tue Oct 16 03:37:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:37:38 +0300 Subject: [Dovecot] POP UIDL In-Reply-To: References: Message-ID: <30E57D11-39A4-43FC-9CF2-91644ADEF950@iki.fi> On 14.10.2012, at 5.07, simon.buongiorno at gmail.com wrote: > I run a small mail server with Dovecot and postfix. Several of the accounts are popped by an external email provide I use, mostly out habit and also for a bit of redundancy. I have no details on that external server except that it uses exim. > > Lately, it's been repopping mail from accounts (mail is left on the server so I can use IMAP) at fairly frequent, but undetermined, intervals. Before I take it to them, I want to be sure it's not an error on my side. How can I be sure Dovecot does not have a problem with the UIDL list causing this external server to repop the mail? > > For the record, I'm not inclined to think it's a Dovecot issue, but since that's the bit I can fix, I'd like to be sure. dovecot -n output would have helped. Anyway, some POP3 clients become confused if there are duplicate UIDLs. Recent v2.1 versions have a pop3_uidl_duplicates setting to avoid those. From tss at iki.fi Tue Oct 16 03:41:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:41:14 +0300 Subject: [Dovecot] Plugin hooks in login process In-Reply-To: <1947528.35zxeZD9k1@orion> References: <1947528.35zxeZD9k1@orion> Message-ID: <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> On 15.10.2012, at 16.13, Simone Lazzaris wrote: > Problem is, if I execute the update on the backend, I miss the information > regarding the original IP, as I only see the IP of the proxies. This is easy to solve: Set login_trusted_networks setting to point to your proxies, and you'll see the original IP. > Looking in the dovecot source code, I noticed that there aren't any hooks in > the execution path used by the proxies; I am missing something ? I am the only > one missing the presence of this hooks in the auth/proxy process ? The login processes aren't really meant to have any plugins. From tss at iki.fi Tue Oct 16 03:42:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:42:51 +0300 Subject: [Dovecot] fts = squat solr In-Reply-To: References: Message-ID: <933F50FC-B8F6-4A02-B738-9109B83D9D33@iki.fi> On 15.10.2012, at 18.08, Dave Abrahams wrote: > I don't know if this was supposed to have changed with dovecot2, but > http://wiki.dovecot.org/Plugins/FTS shows Read wiki2 for Dovecot v2. > fts = squat solr > > so, since I have the lucene plugin?"fts = lucene" works by itself?I > tried > > fts = squat lucene > > but: > > $ doveadm index '*' > doveadm(dave): Error: fts: Failed to initialize backend 'squat lucene': Unknown backend > > So, is that syntax obsolete, is the wiki wrong, or am I doing something > wrong? The syntax is obsolete. From tss at iki.fi Tue Oct 16 03:52:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:52:50 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 1.35, Dave Abrahams wrote: > According to the IMAP spec if I do a search for "TO isocpp.org" it > should find all the messages whose To: field contains the string > "isocpp.org", but dovecot is returning me an empty list. However, a > search for "TO tm at isocpp.org" produces a long list of messages. This specific problem can be solved by: plugin { fts_lucene = whitespace_chars=@. } > This > behavior is present if I *even load* the lucene fts plugin. > Note that lucene isn't in use (fts = squat); it's merely loaded. This > behavior goes away if I don't load fts_lucene. I don't really see how that's possible. Although a quick test shows me that fts_squat seems to be completely broken with me for some reason. From dave at boostpro.com Tue Oct 16 06:44:10 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:44:10 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } Wow; OK, Google tells me that's documented at http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now because I knew what to look for. This might be good enough for me, but still doesn't make it conforming to the IMAP spec, right? IIUC the spec says you can search for arbitrary strings without regard to word boundaries. >> This behavior is present if I *even load* the lucene fts plugin. >> Note that lucene isn't in use (fts = squat); it's merely loaded. >> This behavior goes away if I don't load fts_lucene. > > I don't really see how that's possible. Although a quick test shows me > that fts_squat seems to be completely broken with me for some reason. I don't know what to tell ya. Tests confirm it for me. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 06:45:53 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:45:53 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } OK, Google tells me that's documented at http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now because I knew what to look for. I suggest doing something to make that more discoverable. This might be good enough for me, but still doesn't make it conforming to the IMAP spec, right? IIUC the spec says you can search for arbitrary strings without regard to word boundaries. >> This behavior is present if I *even load* the lucene fts plugin. >> Note that lucene isn't in use (fts = squat); it's merely loaded. >> This behavior goes away if I don't load fts_lucene. > > I don't really see how that's possible. Although a quick test shows me > that fts_squat seems to be completely broken with me for some reason. I don't know what to tell ya. Tests confirm it for me. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 06:51:40 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:51:40 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } Do I also need plugin { fts = lucene } or are these mutually exclusive, or...? It's not clear from http://wiki2.dovecot.org/Plugins/FTS/Lucene -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave.mehler at gmail.com Tue Oct 16 09:30:34 2012 From: dave.mehler at gmail.com (David Mehler) Date: Tue, 16 Oct 2012 02:30:34 -0400 Subject: [Dovecot] per-user quotas In-Reply-To: <20121012040136.GA13561@daniel.localdomain> References: <20121012001539.GA10473@daniel.localdomain> <20121012040136.GA13561@daniel.localdomain> Message-ID: Hello, Thanks for your replies so far. Still having issues with per-user quotas. To my Mysql virtual_users table I've added a column quota_kb and for a test user I've added in a value of 250000 going for a 250 megabyte quota. I've tried various sql queries they're returning empty sets not pulling out the information needed. Thanks. Dave. On 10/12/12, Daniel Parthey wrote: > Hi Dave, > > David Mehler wrote: >> Thanks for your reply. I've written you directly as it is sounding >> like at least for now this isn't dovecot it's well a mysql issue with >> design. I don't have anything in my database setup with regards quota >> I've included it below. I've got one virtual mail user called vmail >> with UID/GID of 5000 who owns all the virtual mailboxes. If I'm >> understanding what I've read in the link, the dovecot wiki and your >> message since anything in a user section of Mysql will override the >> global configuration in 90-quota.conf, that being the case I should >> add an extra column to virtual users? Once that's done adjust the >> userdb query for dovecot to return quota information? > > Yes, the quota should be stored in an additional userdb column, > or you need at least an SQL statement which takes a username > and returns a quota rule. > > Quota of 0 is interpreted as "unlimited" by dovecot. > As already said, if the value in the quota column for the > user is 0, your SQL statement could also return a different > value (default quota) instead of 0 (using MySQL CASE/ELSE statement). > > Here is a short documentation on how it is done with Tine 2.0 Groupware: > http://www.tine20.org/wiki/index.php/Admins/Mailserver_integration > (it is a simple example which does not support a default quota fallback, > since Tine 2.0 writes the default quota as a value into the column > when the user is created) > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From tss at iki.fi Tue Oct 16 09:43:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 09:43:08 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 6.45, Dave Abrahams wrote: >>> According to the IMAP spec if I do a search for "TO isocpp.org" it >>> should find all the messages whose To: field contains the string >>> "isocpp.org", but dovecot is returning me an empty list. However, a >>> search for "TO tm at isocpp.org" produces a long list of messages. >> >> This specific problem can be solved by: >> >> plugin { >> fts_lucene = whitespace_chars=@. >> } > > OK, Google tells me that's documented at > http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now > because I knew what to look for. I suggest doing something to make that > more discoverable. That is the only page where there is any information about fts-lucene. I made it a bit clearer in that page now that whitespace_chars should be used as default. > This might be good enough for me, but still doesn't make it conforming > to the IMAP spec, right? IIUC the spec says you can search for > arbitrary strings without regard to word boundaries. It doesn't conform to the IMAP spec, correct. But nobody cares about that anymore. Everyone violates it. From tss at iki.fi Tue Oct 16 09:43:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 09:43:41 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 6.51, Dave Abrahams wrote: >> plugin { >> fts_lucene = whitespace_chars=@. >> } > > Do I also need > > plugin { > fts = lucene > } > > or are these mutually exclusive, or...? It's not clear from > http://wiki2.dovecot.org/Plugins/FTS/Lucene fts setting selects which backend to use. fts_lucene gives settings to that backend. From benedetto.vassallo at unipa.it Tue Oct 16 10:11:53 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Tue, 16 Oct 2012 09:11:53 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> Message-ID: <20121016091153.15601eysq5n040qh@webmail.unipa.it> Def. Quota Timo Sirainen : > On 4.10.2012, at 16.00, Benedetto Vassallo wrote: > >> All works fine, but with the new version it seems that dovecot >> don't do hardlinks when deliver a message to multiple users. > > The hard linking is done only when the directory permissions match. > >> mail_location = maildir:~/MailDir:LAYOUT=fs >> >> I tryed using lmtp directly issuing 'telnet localhost 24' and >> sending a test message to 3 recipients. >> Then issuing a 'ls -il' in the "new" directory of that users, I saw >> the inode was not the same. > > What are the permissions of the MailDir directory for user1/user2? > > ls -ld /home/user1/MailDir > ls -ld /home/user2/MailDir > > Thank you for your reply. They are different groups: drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ I tryed to issue: chgrp -R mail /home/user1/MailDir chgrp -R mail /home/user2/MailDir chgrp -R mail /home/user3/MailDir but nothing changed. Any idea? Thank you. -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From thefantaman at gmail.com Tue Oct 16 10:28:31 2012 From: thefantaman at gmail.com (Fabrizio Monti) Date: Tue, 16 Oct 2012 09:28:31 +0200 Subject: [Dovecot] /var/run/dovecot/auth-userdb failed In-Reply-To: <20121014142422.GA8080@daniel.localdomain> References: <1350056339814-38093.post@n4.nabble.com> <20121014142422.GA8080@daniel.localdomain> Message-ID: Thank you very much Daniel, I solved the problem. In this moment I have a problem with a maildir, I used a format "/home/vmail/%d/%u" and the first user make maildir he is owner "/home/vmail/%d" and second user don't make a maildir. Now, I look for the solution. Best regards. 2012/10/14 Daniel Parthey > thefantaman wrote: > > I work on test server and if I send an email on log i read > > > > lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: > > Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: > > /var/run/dovecot/auth-userdb, euid is not dir owner) > > > > unix_listener auth-userdb { > > mode = 0600 > > user = root > > group = root > > } > > } > > The problem is that LDA (local delivery agent or lmtp service) > is not able to look up the destination mailbox in userdb. > > The socket /var/run/dovecot/auth-userdb is currently only > readable or writable by user root since mode is set to 0600, > not readable or writable by other groups. > > http://wiki2.dovecot.org/LDA#Virtual_users > > You'll need to set up a auth-userdb socket for dovecot-lda so it > knows where to find mailboxes for the users. LDA is running under > the virtual mailbox user and group "vmail", so you need to grant > this user or group access to /var/run/dovecot/auth-userdb. > > You could do this by using group memberships and set mode = 0660 > or simply make it world-readable-writable with mode = 0666: > > unix_listener auth-userdb { > mode = 0666 > user = root > group = root > } > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From raabe at froglogic.com Tue Oct 16 10:30:26 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Tue, 16 Oct 2012 09:30:26 +0200 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Message-ID: <507D0D12.6000205@froglogic.com> Am 10/16/2012 12:20 AM, schrieb Dave Abrahams: > > on Mon Oct 15 2012, Michael M Slusarz wrote: > >> Quoting Dave Abrahams : >> >>> on Mon Oct 15 2012, Dave Abrahams wrote: >>> >>>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>>> >>>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>>> same empty result with either of these two commands: >>>> >>>> UID SEARCH TO isocpp.org >>>> >>>> UID SEARCH TO "isocpp.org" >>>> >>>> Am I formatting the command wrongly? >>> >>> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >>> same result. >> >> Lucene for sure does not support subtext searching. Squat used to... >> but IIRC things may have changed for v2.1. Try the wiki. > > Sorry, but what does "try the wiki" mean? > Which indexer are you using, that successfully finds the substring match? I don't know what Michael had in mind, but I also seemed to recall that the 'Squat' plugin used to be the only FTS plugin which suppotred substring matches. http://wiki2.dovecot.org/Plugins/FTS/Squat explains: "The main difference between Squat indexes and the others is that Squat provides support for substring searches, while pretty much all other FTS indexes support only matching from the beginning of words. By strictly reading the IMAP RFC it requires substring matching, so to optimize regular TEXT and BODY searches you must use Squat with Dovecot v2.0. [..] However, almost all other commonly used IMAP servers no longer care about this requirement, so Dovecot v2.1 also no longer makes this distinction." I'm not sure how to read this, but I can imagine (and maybe that's what Michael was hinting at) that the Squat plugin for Dovecot >= 2.1 no longer supports substring matches as required by the IMAP RFC whereas previous versions do. P.S.: I wish this list would have a Reply-To configured. :-) -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dave at boostpro.com Tue Oct 16 12:35:42 2012 From: dave at boostpro.com (Dave Abrahams) Date: Tue, 16 Oct 2012 02:35:42 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> <507D0D12.6000205@froglogic.com> Message-ID: on Tue Oct 16 2012, Frerich Raabe wrote: > Am 10/16/2012 12:20 AM, schrieb Dave Abrahams: >> >> on Mon Oct 15 2012, Michael M Slusarz wrote: >> >>> Quoting Dave Abrahams : >>> > >>>> on Mon Oct 15 2012, Dave Abrahams wrote: >>>> >>>>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>>>> >>>>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>>>> same empty result with either of these two commands: >>>>> >>>>> UID SEARCH TO isocpp.org >>>>> >>>>> UID SEARCH TO "isocpp.org" >>>>> >>>>> Am I formatting the command wrongly? >>>> >>>> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >>>> same result. >>> >>> Lucene for sure does not support subtext searching. Squat used to... >>> but IIRC things may have changed for v2.1. Try the wiki. >> >> Sorry, but what does "try the wiki" mean? >> Which indexer are you using, that successfully finds the substring match? > > I don't know what Michael had in mind, but I also seemed to recall > that the 'Squat' plugin used to be the only FTS plugin which suppotred > substring matches. http://wiki2.dovecot.org/Plugins/FTS/Squat > explains: > > "The main difference between Squat indexes and the others is that > Squat provides support for substring searches, while pretty much all > other FTS indexes support only matching from the beginning of > words. By strictly reading the IMAP RFC it requires substring > matching, so to optimize regular TEXT and BODY searches you must use > Squat with Dovecot v2.0. [..] However, almost all other commonly used > IMAP servers no longer care about this requirement, so Dovecot v2.1 > also no longer makes this distinction." > > I'm not sure how to read this, but I can imagine (and maybe that's > what Michael was hinting at) that the Squat plugin for Dovecot >= 2.1 > no longer supports substring matches as required by the IMAP RFC > whereas previous versions do. Well, it worked for me in 2.1.6 and 2.1.9. However, http://wiki2.dovecot.org/Plugins/FTS claims squat is "obsolete" in 2.1.x, my colleague is reporting (to me) crashes with squat during indexing, and Timo just posted that squat "seems to be completely broken for some reason." -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From alessio at skye.it Tue Oct 16 16:36:37 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 16 Oct 2012 15:36:37 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> Message-ID: <507D62E5.60006@skye.it> Il 16/10/2012 02:35, Timo Sirainen ha scritto: > On 13.10.2012, at 15.38, Alessio Cecchi wrote: > >> Il 2012-10-02 21:28 Timo Sirainen ha scritto: >>> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >>> >>>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > This should fix it: http://hg.dovecot.org/dovecot-2.1/rev/41aac09497ee > Timo, with this patch works fine! Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From towern at gmail.com Tue Oct 16 16:39:13 2012 From: towern at gmail.com (tower) Date: Tue, 16 Oct 2012 15:39:13 +0200 Subject: [Dovecot] Problem with quota update in dovecot 1.2 Message-ID: <507D6381.2030703@gmail.com> Hi A have a problem with quota2 table, working under postfixadmin. When I login into imap server with thunderbird client everything works fine, quota is update when i receive a message, but when I move message to trash or any other folder value for messages field in table quota2 increases +1 instead decreases or have still this same value. Only if I close my imap client value descending -2. I wish to table quota2 will be updated immediately after any operation on inbox. Is there any mistake in my configuration? Thanks for any advice. ============================================================== dovecot -n print: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-686 i686 Debian 6.0.3 ext3 log_path: /var/log/dovecot/error.log info_log_path: /var/log/dovecot/info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login first_valid_uid: 100 mail_privileged_group: Debian-exim mail_uid: 101 mail_gid: 103 mail_location: maildir:/var/mail/virtual/%d/%n/Maildir mail_cache_min_mail_count: 100 mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): autocreate quota imap_quota trash mail_plugins(imap): autocreate quota imap_quota trash mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): tb-extra-mailbox-sep imap_client_workarounds(imap): tb-extra-mailbox-sep imap_client_workarounds(pop3): imap_id_log(default): * imap_id_log(imap): * imap_id_log(pop3): lda: postmaster_address: postmaster at mydomain.com mail_plugins: quota log_path: /var/log/dovecot/lda-errors.log info_log_path: /var/log/dovecot/deliver.log auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: Debian-exim plugin: autocreate: INBOX autocreate2: Sent autocreate3: Trash autocreate4: Drafts autocreate5: Junk autocreate6: Spam autosubscribe: INBOX autosubscribe2: Sent autosubscribe3: Trash autosubscribe4: Drafts autosubscribe5: Junk autosubscribe6: Spam quota: dict:user::proxy::quotadict quota_rule: Trash:storage=+20%% quota_rule2: Spam:ignore quota_rule3: Drafts:storage=+5%% quota_rule4: Sent:storage=+15%% quota_rule5: Junk:storage=+10%% quota_warning: storage=100%% quota-exceeded 100 %u quota_warning2: storage=95%% quota-warning 95 %u quota_warning3: storage=90%% quota-warning 90 %u quota_warning4: storage=85%% quota-warning 85 %u quota_warning5: storage=80%% quota-warning 80 %u quota_warning6: storage=75%% quota-warning 75 %u quota_warning7: storage=70%% quota-warning 70 %u quota_warning8: storage=65%% quota-warning 65 %u trash: /etc/dovecot/dovecot-trash.conf mail_log_events: delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_group_events: no mail_log_fields: uid box msgid size subject from dict: quotadict: mysql:/etc/dovecot/dovecot-dict-quota.conf ============================================= cat /etc/dovecot/dovecot-sql.conf driver = mysql connect = host=localhost dbname=eximdb user=eximdbadm password=************* default_pass_scheme = PLAIN password_query = \ SELECT username as user, domain, password \ FROM mailbox WHERE username= '%u' AND active = 1 user_query = \ SELECT CONCAT('/var/mail/virtual/', maildir) AS home, 101 AS uid, 103 AS gid, concat('dict:storage=', CAST(ROUND(quota / 1024) AS CHAR), '::proxy::quotadict') AS quota, CONCAT('*:storage=', CAST(quota AS CHAR), 'B') AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' ============================================= cat /etc/dovecot/dovecot-dict-quota.conf connect = host=localhost dbname=eximdb user=eximdbadm password=********* map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } ============================================== cat /etc/dovecot/dovecot-trash.conf 1 Trash 2 Spam 3 Sent 4 Draft From jbates at brightok.net Tue Oct 16 16:48:17 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 08:48:17 -0500 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D6381.2030703@gmail.com> References: <507D6381.2030703@gmail.com> Message-ID: <507D65A1.2060809@brightok.net> On 10/16/2012 8:39 AM, tower wrote: > Hi > > A have a problem with quota2 table, working under postfixadmin. When I > login into imap server with thunderbird client everything works fine, > quota is update when i receive a message, but when I move message to > trash or any other folder value for messages field in table quota2 > increases +1 instead decreases or have still this same value. Only if > I close my imap client value descending -2. I wish to table quota2 > will be updated immediately after any operation on inbox. Is there any > mistake in my configuration? > > Thanks for any advice. > Are you sure the client isn't just copying it to the other folder. When you exit, you may be expunging inbox (deleting the marked message) and emptying trash (deleting the copied message). Jack From gdelafond+dovecot at aquaray.com Tue Oct 16 16:56:00 2012 From: gdelafond+dovecot at aquaray.com (de Lafond Guillaume) Date: Tue, 16 Oct 2012 15:56:00 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> Message-ID: Hi, > Hi.I'm very sorry for the repost but I forgot the subject. > So, I'm > using dovecot 2.0.18 and I'm trying to authenticate through a CAS server > (until now authentication was through MS Active Directory). I could not > find anywhere some examples, so here is what i have done so far. > -install phpcas and pam_cas > -edit /etc/pam.d/dovecot > auth sufficient /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf > -edit > /etc.pam_cas.conf > host mycas.mydomain.com > port 443 > uriValidate /cas/proxyValidate > ssl on > proxy ??????????????????????? > trusted_ca /etc/cert/certificate.pem > debug > on > > - and finally dovecot.conf which I'm sure is complety wrong > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > passdb { > driver = pam > args = cache_key=%u dovecot > } Could you try with "failure_show_msg=yes" on passdb args ? You can try without the "proxy" line in pam_cas.conf and a static userdb with allow_all_users=yes (in place of ldap configuration). Hope this can help. You have some logs in /var/log/auth.log (depends of your distrib). -- Guillaume de Lafond Aqua Ray From towern at gmail.com Tue Oct 16 17:00:46 2012 From: towern at gmail.com (tower) Date: Tue, 16 Oct 2012 16:00:46 +0200 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D65A1.2060809@brightok.net> References: <507D6381.2030703@gmail.com> <507D65A1.2060809@brightok.net> Message-ID: <507D688E.6000903@gmail.com> On 10/16/2012 03:48 PM, Jack Bates wrote: > On 10/16/2012 8:39 AM, tower wrote: >> Hi >> >> A have a problem with quota2 table, working under postfixadmin. When >> I login into imap server with thunderbird client everything works >> fine, quota is update when i receive a message, but when I move >> message to trash or any other folder value for messages field in >> table quota2 increases +1 instead decreases or have still this same >> value. Only if I close my imap client value descending -2. I wish to >> table quota2 will be updated immediately after any operation on >> inbox. Is there any mistake in my configuration? >> >> Thanks for any advice. >> > Are you sure the client isn't just copying it to the other folder. > When you exit, you may be expunging inbox (deleting the marked > message) and emptying trash (deleting the copied message). > > > Jack Yes, now I see, that messages still is in folder from which I move it. In thunderbird I do drag and drop (FYI). Is there any switch in dovecot configuration which treats message as deleted from folder, from which I move that message? From s.lazzaris at interactive.eu Tue Oct 16 17:40:57 2012 From: s.lazzaris at interactive.eu (Simone Lazzaris) Date: Tue, 16 Oct 2012 16:40:57 +0200 Subject: [Dovecot] Plugin hooks in login process In-Reply-To: <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> References: <1947528.35zxeZD9k1@orion> <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> Message-ID: <2673453.lfcgdkNUMt@orion> In data marted? 16 ottobre 2012 03:41:14, Timo Sirainen ha scritto: > On 15.10.2012, at 16.13, Simone Lazzaris wrote: > > Problem is, if I execute the update on the backend, I miss the information > > regarding the original IP, as I only see the IP of the proxies. > > This is easy to solve: Set login_trusted_networks setting to point to your > proxies, and you'll see the original IP. Thanks, it works as a charm. Just one more thing. I wasn't be able to find this option in the wiki, and also now that I know the meaning, I can't successfully google for it. Maybe the docs needs some love ? I hate tamper the developers for something I should have found by myself on the first place. > > Looking in the dovecot source code, I noticed that there aren't any hooks > > in the execution path used by the proxies; I am missing something ? I am > > the only one missing the presence of this hooks in the auth/proxy process > > ? > The login processes aren't really meant to have any plugins. I see. Well, thanks anyway. -- Simone Lazzaris | Responsabile aree datacenter e VoIP Interactive Network srl | via Roggia Vignola 9, 24047 Treviglio (BG) Tel. 0363 1970352 | Fax 0363.1971971 | www.interactive.eu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From david.travers at cohenschemist.co.uk Tue Oct 16 19:39:54 2012 From: david.travers at cohenschemist.co.uk (David Travers) Date: Tue, 16 Oct 2012 16:39:54 +0000 Subject: [Dovecot] Per user quotas Message-ID: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> Hi All, I keep going round in circles with this. I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. I believe I need to add to a passwd file, but I have added to that and nothing has changed. # dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.5 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:~/mail:LAYOUT=fs:INBOX=~/mail/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 lda: mail_plugins: quota auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: quota: maildir:user quota quota_rule: *:storage=200M quota_rule2: Trash:storage=10%% quota_rule2: SPAM:ignore quota_warning: storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2: storage=80%% /usr/local/bin/quota-warning.sh 80 quota_exceeded_message: Quota exceeded, Please delete some emails Any help gratefully received Dave _________________________________________ This email has been scanned for malicious content. _________________________________________ From jbates at brightok.net Tue Oct 16 19:54:21 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 11:54:21 -0500 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D688E.6000903@gmail.com> References: <507D6381.2030703@gmail.com> <507D65A1.2060809@brightok.net> <507D688E.6000903@gmail.com> Message-ID: <507D913D.8070004@brightok.net> On 10/16/2012 9:00 AM, tower wrote: > Yes, now I see, that messages still is in folder from which I move it. > In thunderbird I do drag and drop (FYI). Is there any switch in > dovecot configuration which treats message as deleted from folder, > from which I move that message? Not to my knowledge, but someone else may know more. I setup the Trash folder with a +100MB quota per the wiki so that it could handle deleting emails when quota was near full. Jack From stocton12 at yahoo.com Tue Oct 16 21:12:37 2012 From: stocton12 at yahoo.com (b m) Date: Tue, 16 Oct 2012 11:12:37 -0700 (PDT) Subject: [Dovecot] CAS Authentication In-Reply-To: References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> Message-ID: <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login? with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. ________________________________ From: de Lafond Guillaume To: b m ; Dovecot Mailing List Sent: Tuesday, October 16, 2012 4:56 PM Subject: Re: [Dovecot] CAS Authentication Hi, > Hi.I'm very sorry for the repost but I forgot the subject. > So,? I'm > using dovecot 2.0.18 and I'm trying to authenticate through a CAS server > (until now authentication was through MS Active Directory). I could not > find anywhere some examples, so here is what i have done so far. > -install phpcas and pam_cas > -edit /etc/pam.d/dovecot >? ? ? ? ? ? ? ? ? auth? ? sufficient? ? ? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf > -edit > /etc.pam_cas.conf >? ? ? ? ? ? ? ? ? host mycas.mydomain.com >? ? ? ? ? ? ? ? ? port 443 >? ? ? ? ? ? ? ? ? uriValidate /cas/proxyValidate >? ? ? ? ? ? ? ? ? ssl on >? ? ? ? ? ? ? ? ? proxy ??????????????????????? >? ? ? ? ? ? ? ? ? trusted_ca /etc/cert/certificate.pem >? ? ? ? ? ? ? ? ? debug > on > > - and finally dovecot.conf which I'm sure is complety wrong >? ? ? ? ? ? ? userdb { >? ? ? ? ? ? ? args = /etc/dovecot/dovecot-ldap.conf >? ? ? ? ? ? ? driver = ldap >? ? ? ? ? ? ? } >? ? ? ? ? ? ? passdb { >? ? ? ? ? ? ? driver = pam >? ? ? ? ? ? ? args = cache_key=%u dovecot >? ? ? ? ? ? ? } Could you try with "failure_show_msg=yes" on passdb args ? You can try without the "proxy" line in pam_cas.conf and a static userdb with allow_all_users=yes (in place of ldap configuration). Hope this can help. You have some logs in /var/log/auth.log (depends of your distrib). -- Guillaume de Lafond Aqua Ray From jbates at brightok.net Tue Oct 16 23:43:00 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 15:43:00 -0500 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 Message-ID: <507DC6D4.7090902@brightok.net> Timo, Please check the code. I didn't add it, but a real_lip might be useful for some people as well. Also, I notice that pop3-proxy is doing a different xsession than the imap proxy. Is there an xsession standard that is different between the two, or just an oversight in the code? Both send the remote address/port, but only imap proxy sends the local address/port. This patch declares long variable %{real_rip} so that a backend server can declare a different login_log_format_elements login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c This is primarily useful for backend servers to log both the rip, lip, and in case of xsession, the real rip. I haven't done extensive testing yet, but as long as nothing does anything weird elsewhere in the code, it should be good. http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch dovecot: pop3-login: Login: user=, method=PLAIN, rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured Jack Bates From list at airstreamcomm.net Tue Oct 16 23:57:45 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 16 Oct 2012 15:57:45 -0500 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> Message-ID: <507DCA49.9010702@airstreamcomm.net> On 10/4/12 9:58 AM, Timo Sirainen wrote: > On 4.10.2012, at 17.41, list at airstreamcomm.net wrote: > >>> protocol lmtp { >>> userdb { >>> .. >>> } >>> } >>> protocol !lmtp { >>> userdb { >>> .. >>> } >>> } >>> >>> >> Forgot to mention I am running 2.0.17. > The above works in v2.1. > >> And I am getting the following error: >> >> auth: Fatal: No passdbs specified in configuration file. PLAIN >> mechanism needs one >> >> >> From a previous post it appears that Dovecot cannot run without a global lookups specified: >> >> http://www.dovecot.org/list/dovecot/2012-March/064407.html >> >> Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? > The !lmtp version avoids that fatal problem. So the solution is: upgrade. Timo, I upgraded to 2.1 and configured as recommended, however I am still getting an error: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one Doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_debug = yes auth_verbose = yes disable_plaintext_auth = no mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service pop3-login { inet_listener pop3 { port = 110 } } ssl_cert = References: <507DC6D4.7090902@brightok.net> Message-ID: <507DE508.8070705@brightok.net> Still have some problems with the patch. will have to research it more. Worked fine on localhost, but isn't logging at all for foreign hosts in a production environment. Jack On 10/16/2012 3:43 PM, Jack Bates wrote: > Timo, > > Please check the code. I didn't add it, but a real_lip might be useful > for some people as well. Also, I notice that pop3-proxy is doing a > different xsession than the imap proxy. Is there an xsession standard > that is different between the two, or just an oversight in the code? > Both send the remote address/port, but only imap proxy sends the local > address/port. > > This patch declares long variable %{real_rip} so that a backend server > can declare a different login_log_format_elements > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l > pip=%{real_rip} mpid=%e %c > > This is primarily useful for backend servers to log both the rip, lip, > and in case of xsession, the real rip. I haven't done extensive > testing yet, but as long as nothing does anything weird elsewhere in > the code, it should be good. > > http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch > > dovecot: pop3-login: Login: user=, method=PLAIN, > rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured > > > Jack Bates > From fred at fredk.com Wed Oct 17 02:14:04 2012 From: fred at fredk.com (Fred Kilbourn) Date: Tue, 16 Oct 2012 18:14:04 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS Message-ID: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> I've had squat running on dovecot 2.0 and have been updating all users mailbox indexes nighty via cron with this command: doveadm -v search -A text xyzzyx I've just updated to 2.1 and I'm migrating to lucene indexes, but reading the documentation I'm having a hard time understanding the semantic differences between indexing and rescanning. If I were to continue to run an all user all mailbox index every night, would that be appropriate? Would running this every night avoid the need to ever rescan? Should I run rescan instead of index? Should I run both rescan and index? In which order? Best Regards, FredK From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 17 02:18:56 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 17 Oct 2012 01:18:56 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users Message-ID: <20121016231856.GA10851@daniel.localdomain> # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-44-server x86_64 Ubuntu 10.04.4 LTS doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A shows the following message in the log when iterating the 49th user: Oct 17 00:47:17 10.129.3.233 dovecot: doveadm: Error: purge: invalid option -- 'e' Oct 17 00:47:17 10.129.3.233 dovecot: doveadm(someuser at example-ll.org): Error: doveadm purge: Client sent unknown parameter: ? Any ideas on how this error gets triggered? Regards Daniel -- https://plus.google.com/103021802792276734820 From calestyo at scientia.net Wed Oct 17 02:21:14 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 01:21:14 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? Message-ID: <1350429674.3360.27.camel@fermat.scientia.net> Hi. I'm migrating all my mail archive (some 60 GB) from Evolution (which is really a broken piece of software) into dovecot. Now I face the problem how to do this best... Evolution (which is still a old 2.32.x version) itself uses mbox files, in a special hierarchical structure to allow subfolders and that like. It also stores it's own status info in X-Evolution and X-Evolution-Source mail headers. Unfortunately,... much of the mail was earlier from a Thunderbird installation, which uses it's own status headers (X-Mozilla*) that were not recognised by Evolution. I have no idea which mbox subformat was always used throughout the different programs and versions... 1) Any way to check for that? To make things worse... Thunder(burden) seems to have used a modified From_ line syntax... "^From -
$" Ideally I'd like migrate all mail into dovecot (for storage reasons again, mbox) retaining the different status flags (read, forwarded, etc.) and getting rid of the proprietary headers (of course only when they were. First thing I tried was to simply copy mail within Evolution (i.e. dragging&dropping it from the local folders to the IMAP folders from dovecot). - that preserves the status from Evolution, but doesn't restore that from Thunderbird - it clutters up the information of all From_ lines... "
" becomes "
" - neither does it handle the special Thunderbird From_ lines - neither does it remove the Thunderbird or the X-Evolution-Source headers - if Evolution has already had corrupted index files (and this is extremely likely... as it happens even immediately during recreating all of them)... so I may loose mail So my idea was that I need a program that: - can parse all the different mbox formats (those that use the quoted ">From" style and those that use Conent-Length) - can differentiate message headers from body (so that I can drop the proprietary headers and replace them by what dovecot uses as headers) - must of course understand multiline message headers 2) Any idea for a tool like that? The meaning of the X-Evolution and X-Mozilla* headers are easy to find on the web.... so I can convert them. So I basically "just" need a tool that parses all kinds of mbox formats... allow me to drop/add headers... and spits out the rest unmodified. 3) dovecot uses some special headers like X-UID and X-IMAPD... will it create these on it's own, the first time it processes the new mbox file? I mean these headers won't be there after creation. 4) Should I drop out (during) conversion... any other mail headers.. that dovecot uses as it's own? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 17 04:21:46 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 20:21:46 -0500 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 In-Reply-To: <507DE508.8070705@brightok.net> References: <507DC6D4.7090902@brightok.net> <507DE508.8070705@brightok.net> Message-ID: <507E082A.9050508@brightok.net> Umm, yeah. Setting your rsyslog to pipe certain IP matches to another file really screws things up when you change the log to include that IP. Hours of work before I figured out that it was logging just fine but to another file. lol Jack On 10/16/2012 5:51 PM, Jack Bates wrote: > Still have some problems with the patch. will have to research it > more. Worked fine on localhost, but isn't logging at all for foreign > hosts in a production environment. > > > Jack > > On 10/16/2012 3:43 PM, Jack Bates wrote: >> Timo, >> >> Please check the code. I didn't add it, but a real_lip might be >> useful for some people as well. Also, I notice that pop3-proxy is >> doing a different xsession than the imap proxy. Is there an xsession >> standard that is different between the two, or just an oversight in >> the code? Both send the remote address/port, but only imap proxy >> sends the local address/port. >> >> This patch declares long variable %{real_rip} so that a backend >> server can declare a different login_log_format_elements >> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l >> pip=%{real_rip} mpid=%e %c >> >> This is primarily useful for backend servers to log both the rip, >> lip, and in case of xsession, the real rip. I haven't done extensive >> testing yet, but as long as nothing does anything weird elsewhere in >> the code, it should be good. >> >> http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch >> >> dovecot: pop3-login: Login: user=, method=PLAIN, >> rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured >> >> >> Jack Bates >> > From calestyo at scientia.net Wed Oct 17 05:00:36 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 04:00:36 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350429674.3360.27.camel@fermat.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> Message-ID: <1350439236.18957.13.camel@fermat.scientia.net> Hi again... Things are even much much worse... (oh how I hate Evolution right now). I found a bug in Evolution, where it apparently corrupts all mail by incorrectly (not) quoting From_ lines in headers/bodies... It quotes lines matching "^From (.*)$" as ">From \1" but it does not quote at all already quoted From_ lines, i.e. "^>+From .*$". Now that means AFAICS, that it's not possible to repair that corruption (you'll see my "happiness" about this, when reading the offensive bug report): Details here: https://bugzilla.gnome.org/show_bug.cgi?id=686258 I'm not sure how this affects any of my migration/conversion plans... any ideas? Thanks, a desperate Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 17 05:44:19 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 21:44:19 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: <507E1B83.7070104@brightok.net> On 10/16/2012 6:14 PM, Fred Kilbourn wrote: > I've had squat running on dovecot 2.0 and have been updating all users > mailbox indexes nighty via cron with this command: > > doveadm -v search -A text xyzzyx > > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > the documentation I'm having a hard time understanding the semantic > differences between indexing and rescanning. > > > > If I were to continue to run an all user all mailbox index every night, > would that be appropriate? > > > > Would running this every night avoid the need to ever rescan? > > 2 sets of indexes. dovecot indexes FTS indexes Performing the cron search will update the FTS indexes, although you should read up on 2.1's doveadm index command. The dovecot indexes should stay in sync. However, if they do lose track of the FTS indexes, you can do a rescan to sync them back up. Except for corruption or index changes made outside of dovecot, the two sets of indexes should stay in sync. Jack From jbates at brightok.net Wed Oct 17 06:01:17 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 22:01:17 -0500 Subject: [Dovecot] Per user quotas In-Reply-To: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> Message-ID: <507E1F7D.4080107@brightok.net> On 10/16/2012 11:39 AM, David Travers wrote: > Hi All, > > I keep going round in circles with this. > I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! > > I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. > > I believe I need to add to a passwd file, but I have added to that and nothing has changed. > > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields Note that the extra fields are prefixed with userdb_ when placed in a passwd file. Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). Jack From fred at fredk.com Wed Oct 17 06:09:20 2012 From: fred at fredk.com (Fred Kilbourn) Date: Tue, 16 Oct 2012 22:09:20 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <507E1B83.7070104@brightok.net> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> <507E1B83.7070104@brightok.net> Message-ID: <00b301cdac14$cda1f880$68e5e980$@fredk.com> > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] > On Behalf Of Jack Bates > Sent: Tuesday, October 16, 2012 9:44 PM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Difference between Indexing and Rescan in FTS > > On 10/16/2012 6:14 PM, Fred Kilbourn wrote: > > I've had squat running on dovecot 2.0 and have been updating all users > > mailbox indexes nighty via cron with this command: > > > > doveadm -v search -A text xyzzyx > > > > > > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > > the documentation I'm having a hard time understanding the semantic > > differences between indexing and rescanning. > > > > > > > > If I were to continue to run an all user all mailbox index every night, > > would that be appropriate? > > > > > > > > Would running this every night avoid the need to ever rescan? > > > > > 2 sets of indexes. > > dovecot indexes > FTS indexes > > Performing the cron search will update the FTS indexes, although you > should read up on 2.1's doveadm index command. The dovecot indexes > should stay in sync. However, if they do lose track of the FTS indexes, > you can do a rescan to sync them back up. Except for corruption or index > changes made outside of dovecot, the two sets of indexes should stay in > sync. > > > Jack Thanks Jack, So here are my takeaways, let me know if I'm wrong: - The FTS index is the actual search data - The dovecot index holds, among other information, which messages are indexed by FTS - The FTS index still doesn't update automatically, so my nightly cronjob should keep it in order - The dovecot index should stay in order under normal circumstances, and issuing a resync command shouldn't be needed unless something bad happens Assuming my understanding above is correct, how about these questions, to further clarify my original questions: - As a system administrator, what signs should I look for that a resync is needed? (aside from user complaints) - What exact impact does running the resync command have? - Is it worthwhile to resync periodically as a maintenance task? - Or, does resyncing reset all FTS indexing that has been done, causing it to have to be done again from scratch? And, I did catch the revision in the user docs for updating indexes. I plan on updating my maintenance script accordingly. Thanks, Fred From tss at iki.fi Wed Oct 17 06:15:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 17 Oct 2012 06:15:54 +0300 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: On 17.10.2012, at 2.14, Fred Kilbourn wrote: > I've had squat running on dovecot 2.0 and have been updating all users > mailbox indexes nighty via cron with this command: > > doveadm -v search -A text xyzzyx doveadm index is a bit more efficient. > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > the documentation I'm having a hard time understanding the semantic > differences between indexing and rescanning. doveadm fts rescan makes sure that 1) all of the old messages are indexed and 2) there are no extra (already deleted) messages indexed. So it's basically repairing fts index. You probably shouldn't run it automatically, or at least not very often. From dmalolepszy at optusnet.com.au Wed Oct 17 09:11:13 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 17 Oct 2012 17:11:13 +1100 Subject: [Dovecot] Dovecot failed logins delay all logins Message-ID: <507E4C01.6010303@optusnet.com.au> Hi all, I have observed with my Dovecot setup that unique failed logins cause legitimate correct logins to be slowed. I am running two servers, each with two Dovecot instances, a Proxy with Director, and a backend Dovecot. I suspect that the backend instance is throttling connections from the same IP, and because I am running a Proxy, the backend will only see either of the two server IPs. I confirmed this by directly connecting to the backend, to bypass the proxy and rule it. I initiated dozens of unique failed logins from one IP and separately attempted to login from the same IP, and experienced an extended delay during login. At the same time a login from a different IP suceeded imediately. I see nothing in the logs suggesting some sort of process limits were exceeded, however I do see the following proc title for the backend auth process: "dovecot/auth [7 wait, 0 passdb, 0 userdb]" I have increased the mail_max_userip_connections to a very large value however I believe that setting is a per username/ip limit. Is there any sort of setting in Dovecot that I can configure that stops this authentication throttling per IP? Below is the configuration of the backend Dovecot instance. # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.3 (Santiago) auth_cache_negative_ttl = 3 secs auth_cache_size = 100 M auth_cache_ttl = 10 mins auth_default_realm = example.com auth_failure_delay = 5 secs auth_mechanisms = plain login auth_verbose_passwords = sha1 auth_worker_max_count = 25 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 last_valid_gid = 12 last_valid_uid = 8 login_greeting = Hello there. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_fsync = always mail_gid = mail mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " stats" mail_uid = mail mmap_disable = yes namespace { inbox = yes location = maildir:%h/Maildir prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocols = pop3 imap service auth { unix_listener auth-userdb { group = mail mode = 0660 user = mail } } service imap-login { inet_listener imap { address = 0.0.0.0 port = 9143 } process_min_avail = 5 service_count = 0 vsz_limit = 256 M } service imap { process_limit = 1000 vsz_limit = 256 M } service pop3-login { inet_listener pop3 { address = 0.0.0.0 port = 9110 } process_min_avail = 5 service_count = 0 vsz_limit = 256 M } service pop3 { process_limit = 1000 vsz_limit = 256 M } service stats { fifo_listener stats-mail { mode = 0600 user = mail } inet_listener { address = 127.0.0.1 port = 24242 } } ssl = no stats_memory_limit = 64 M userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol imap { imap_logout_format = bytes_read=%i bytes_send=%o mail_max_userip_connections = 1000 mail_plugins = " stats " } protocol pop3 { mail_max_userip_connections = 1000 } Dominic From pipefab at mweb.co.za Wed Oct 17 09:26:30 2012 From: pipefab at mweb.co.za (Hendrik) Date: Wed, 17 Oct 2012 08:26:30 +0200 Subject: [Dovecot] dovecot died Message-ID: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Hi All I have been trying to get this website running for months now. I get this emails from cpanel and don't know how to fix it. If anyone can help me please contact me at pipefab at mweb.co.za. Kind regards Hendrik imap failed @ Fri Oct 12 05:51:18 2012. A restart was attempted automagically. Service Check Method: [check command] Number of Restart Attempts: 1 Cmd Service Check Raw Output: dovecot is not running Startup Log: /etc/init.d/dovecot: line 15: 6043 Alarm clock /usr/sbin/dovecot > /dev/null 2>&1 Syslog Messages: Oct 12 04:01:01 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3 yj43fuh...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:01:02 scene dovecot: IMAP(__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3y j43fuhjgeiqomc3dhlkyjwdq): Disconnected: Logged out bytes=11/340 Oct 12 04:06:06 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0 jaovpdr...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:06:06 scene dovecot: IMAP(__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0j aovpdrgqjnf0_rxyi0wncetn): Disconnected: Logged out bytes=11/313 Oct 12 04:11:01 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4z k8h05nx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:11:01 scene dovecot: IMAP(__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4zk 8h05nx2p9n4yfxhrp3a2gjhl): Disconnected: Logged out bytes=11/313 Oct 12 04:16:08 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigs c75cnov...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:16:09 scene dovecot: IMAP(__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigsc 75cnovslbll4702ue2veu2n3): Disconnected: Logged out bytes=11/318 Oct 12 04:21:10 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsf ou8d5ky...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:21:11 scene dovecot: IMAP(__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsfo u8d5kysrfeaqvlmjgx6afvnb): Disconnected: Logged out bytes=11/340 Oct 12 04:26:35 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoe oga08za...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:26:36 scene dovecot: IMAP(__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoeo ga08zaq_nh6yzqsmveqpvnxi): Disconnected: Logged out bytes=11/313 Oct 12 04:31:05 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaus tpvzl1q...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:31:05 scene dovecot: IMAP(__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaust pvzl1qjjei5uuoi1c4g8tpea): Disconnected: Logged out bytes=11/313 Oct 12 04:36:02 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjj fzonng9...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:36:03 scene dovecot: IMAP(__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjjf zonng9eqrsw5l5hg7xoejer2): Disconnected: Logged out bytes=11/340 Oct 12 04:41:03 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gw eibvov1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:41:04 scene dovecot: IMAP(__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gwe ibvov1_minfh7j4_4ejaz7v2): Disconnected: Logged out bytes=11/340 Oct 12 04:46:04 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhd rhafjur...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:46:05 scene dovecot: IMAP(__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhdr hafjureydiuxbtbk2jkpkvlo): Disconnected: Logged out bytes=11/340 Oct 12 04:51:06 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrq h1iztqo...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:51:06 scene dovecot: IMAP(__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrqh 1iztqosnzfwt28kkzv4riyd9): Disconnected: Logged out bytes=11/340 Oct 12 04:56:07 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan 2pdmdvg...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:56:07 scene dovecot: IMAP(__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan2 pdmdvg6jofzylncdb3ytjaaz): Disconnected: Logged out bytes=11/340 Oct 12 05:01:08 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvir quwcv1z...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:01:08 scene dovecot: IMAP(__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvirq uwcv1zhloqt12rqni_o2pqcj): Disconnected: Logged out bytes=11/340 Oct 12 05:06:09 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgr cpi3usb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:06:09 scene dovecot: IMAP(__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgrc pi3usbdjkiy2n8zy2bbvhhny): Disconnected: Logged out bytes=11/340 Oct 12 05:11:10 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg 4vew3df...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:11:10 scene dovecot: IMAP(__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg4 vew3dfbyrglsasuldldaspck): Disconnected: Logged out bytes=11/340 Oct 12 05:16:11 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0 e2bwtas...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:16:11 scene dovecot: IMAP(__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0e 2bwtastu0kton3azlhmmuhwi): Disconnected: Logged out bytes=11/340 Oct 12 05:21:12 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszq mf4h8z1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:21:12 scene dovecot: IMAP(__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszqm f4h8z1shjl34q9zpid3g4gsp): Disconnected: Logged out bytes=11/340 Oct 12 05:26:13 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaog ha3yc0w...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:26:13 scene dovecot: IMAP(__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaogh a3yc0w6t7vvgyf1snz8vechf): Disconnected: Logged out bytes=11/340 Oct 12 05:31:14 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62 mucudsm...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:31:15 scene dovecot: IMAP(__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62m ucudsmjfmyolzcpm9shakkiw): Disconnected: Logged out bytes=11/340 Oct 12 05:36:15 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxc kmn7fkd...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:36:16 scene dovecot: IMAP(__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxck mn7fkdpsvbrjptqanfuljfv2): Disconnected: Logged out bytes=11/340 Oct 12 05:41:15 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqun f1neeqb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:41:16 scene dovecot: IMAP(__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqunf 1neeqbmzas00uqbzmmjsxiab): Disconnected: Logged out bytes=11/340 Oct 12 05:46:17 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfre yewqvxx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:46:17 scene dovecot: IMAP(__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfrey ewqvxxeoo7yhyiki7ghukxss): Disconnected: Logged out bytes=11/340 Oct 12 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 seconds. This might cause a lot of problems, so I'll just kill myself now. http://wiki.dovecot.org/TimeMovedBackwards From fred at fredk.com Wed Oct 17 09:26:32 2012 From: fred at fredk.com (Fred Kilbourn) Date: Wed, 17 Oct 2012 01:26:32 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: <00e101cdac30$5ab63270$10229750$@fredk.com> > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] > On Behalf Of Timo Sirainen > Sent: Tuesday, October 16, 2012 10:16 PM > To: Fred Kilbourn > Cc: dovecot at dovecot.org > Subject: Re: [Dovecot] Difference between Indexing and Rescan in FTS > > On 17.10.2012, at 2.14, Fred Kilbourn wrote: > > > I've had squat running on dovecot 2.0 and have been updating all users > > mailbox indexes nighty via cron with this command: > > > > doveadm -v search -A text xyzzyx > > doveadm index is a bit more efficient. > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but > reading > > the documentation I'm having a hard time understanding the semantic > > differences between indexing and rescanning. > > doveadm fts rescan makes sure that 1) all of the old messages are > indexed and 2) there are no extra (already deleted) messages indexed. So > it's basically repairing fts index. You probably shouldn't run it > automatically, or at least not very often. Okay, you've clarified that for me. I understand that rescan isn't a nightly task, but could be run every now and then periodically. How often might be appropriate if I wanted to do this as a maintenance task? Once a month? Lastly, I'm trying to use the index command instead of the search command, but I can't figure out how to make it index every mailbox for every user. Is there a wildcard that can be used for the mailbox? Or do I need to iterate all the mailboxes with one command and run index however many times for each inbox? Thanks for your help From dmalolepszy at optusnet.com.au Wed Oct 17 09:44:10 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 17 Oct 2012 17:44:10 +1100 Subject: [Dovecot] Dovecot failed logins delay all logins In-Reply-To: <507E4C01.6010303@optusnet.com.au> References: <507E4C01.6010303@optusnet.com.au> Message-ID: <507E53BA.7030001@optusnet.com.au> I think I found a solution to this thanks to a post by Timo here: http://dovecot.org/list/dovecot/2011-December/062631.html service anvil { unix_listener anvil-auth-penalty { mode = 0 } } On 17/10/12 17:11, Dominic Malolepszy wrote: > Hi all, > > I have observed with my Dovecot setup that unique failed logins cause > legitimate correct logins to be slowed. I am running two servers, each > with two Dovecot instances, a Proxy with Director, and a backend > Dovecot. I suspect that the backend instance is throttling > connections from the same IP, and because I am running a Proxy, the > backend will only see either of the two server IPs. I confirmed this > by directly connecting to the backend, to bypass the proxy and rule > it. I initiated dozens of unique failed logins from one IP and > separately attempted to login from the same IP, and experienced an > extended delay during login. At the same time a login from a different > IP suceeded imediately. I see nothing in the logs suggesting some sort > of process limits were exceeded, however I do see the following proc > title for the backend auth process: > "dovecot/auth [7 wait, 0 passdb, 0 userdb]" > > I have increased the mail_max_userip_connections to a very large value > however I believe that setting is a per username/ip limit. Is there > any sort of setting in Dovecot that I can configure that stops this > authentication throttling per IP? Below is the configuration of the > backend Dovecot instance. > > > # 2.1.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Red Hat Enterprise Linux > Server release 6.3 (Santiago) > auth_cache_negative_ttl = 3 secs > auth_cache_size = 100 M > auth_cache_ttl = 10 mins > auth_default_realm = example.com > auth_failure_delay = 5 secs > auth_mechanisms = plain login > auth_verbose_passwords = sha1 > auth_worker_max_count = 25 > base_dir = /var/run/dovecot/ > disable_plaintext_auth = no > first_valid_gid = 12 > first_valid_uid = 8 > last_valid_gid = 12 > last_valid_uid = 8 > login_greeting = Hello there. > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > mail_fsync = always > mail_gid = mail > mail_location = maildir:%h/Maildir > mail_nfs_index = yes > mail_nfs_storage = yes > mail_plugins = " stats" > mail_uid = mail > mmap_disable = yes > namespace { > inbox = yes > location = maildir:%h/Maildir > prefix = INBOX. > separator = . > } > passdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > protocols = pop3 imap > service auth { > unix_listener auth-userdb { > group = mail > mode = 0660 > user = mail > } > } > service imap-login { > inet_listener imap { > address = 0.0.0.0 > port = 9143 > } > process_min_avail = 5 > service_count = 0 > vsz_limit = 256 M > } > service imap { > process_limit = 1000 > vsz_limit = 256 M > } > service pop3-login { > inet_listener pop3 { > address = 0.0.0.0 > port = 9110 > } > process_min_avail = 5 > service_count = 0 > vsz_limit = 256 M > } > service pop3 { > process_limit = 1000 > vsz_limit = 256 M > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = mail > } > inet_listener { > address = 127.0.0.1 > port = 24242 > } > } > ssl = no > stats_memory_limit = 64 M > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > verbose_proctitle = yes > protocol imap { > imap_logout_format = bytes_read=%i bytes_send=%o > mail_max_userip_connections = 1000 > mail_plugins = " stats " > } > protocol pop3 { > mail_max_userip_connections = 1000 > } > > > Dominic From amateo at um.es Wed Oct 17 10:24:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 17 Oct 2012 09:24:42 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> Message-ID: <507E5D3A.5030900@um.es> El 16/10/12 20:12, b m escribi?: > Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. > This is my config. In /etc/pam.d/dovecot I have: auth sufficient pam_cas_ssh.so -simap://localhost -f/etc/pam_cas.conf account sufficient pam_permit.so session sufficient pam_permit.so and /etc/pam_cas.conf host port 443 uriValidate /cas/proxyValidate ssl on debug off proxy trusted_ca in dovecot, I have these users dbs: userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } With this, it works fine. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From david.travers at cohenschemist.co.uk Wed Oct 17 10:42:06 2012 From: david.travers at cohenschemist.co.uk (David Travers) Date: Wed, 17 Oct 2012 07:42:06 +0000 Subject: [Dovecot] Per user quotas In-Reply-To: <507E1F7D.4080107@brightok.net> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> Message-ID: <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> Hi Jake, Yep, similar to what I had been trying, but it doesn't seem to be working. In my /etc/passwd file I had the line:- dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash I have changed it to show:- dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange Also yes, I noticed the numbering and have corrected. Thanks for your quick reply. Dave -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates Sent: 17 October 2012 04:02 To: dovecot at dovecot.org Subject: Re: [Dovecot] Per user quotas On 10/16/2012 11:39 AM, David Travers wrote: > Hi All, > > I keep going round in circles with this. > I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! > > I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. > > I believe I need to add to a passwd file, but I have added to that and nothing has changed. > > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields Note that the extra fields are prefixed with userdb_ when placed in a passwd file. Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). Jack _________________________________________ This email has been scanned for malicious content. _________________________________________ From CMarcus at Media-Brokers.com Wed Oct 17 12:59:30 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 17 Oct 2012 05:59:30 -0400 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <507DCA49.9010702@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> <507DCA49.9010702@airstreamcomm.net> Message-ID: <507E8182.3040904@Media-Brokers.com> On 2012-10-16 4:57 PM, list at airstreamcomm.net wrote: > > Doveconf -n: > > # 2.1.1: /etc/dovecot/dovecot.conf Ummm... latest is 2.1.10... try upgrading again... ;) From CMarcus at Media-Brokers.com Wed Oct 17 13:13:34 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 17 Oct 2012 06:13:34 -0400 Subject: [Dovecot] dovecot died In-Reply-To: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> References: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Message-ID: <507E84CE.60708@Media-Brokers.com> On 2012-10-17 2:26 AM, Hendrik wrote: > 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 > seconds. This might cause a lot of problems, so I'll just kill myself now. > http://wiki.dovecot.org/TimeMovedBackwards Presumably this is on a shared hosting service, and they haven't gotten the VM to sync time properly. It is absolutely critical that time be kept in sync on servers, especially mail servers, so dovecot assumes a serious problem exists when time leaps like this happen and kills itself. Fix the time sync problem on your server (you may have to work with the service/hosting provider) and dovecot will stop committing seppuku... Charles From raabe at froglogic.com Wed Oct 17 15:44:12 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 17 Oct 2012 14:44:12 +0200 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read Message-ID: <507EA81C.5060806@froglogic.com> Hi, I'm running Dovecot 1.2.17 on FreeBSD 9 to serve an archive of a few internal mailinglists. The archive is implemented using a public namespace: namespace private { separator = / prefix = inbox = yes } namespace public { separator = / prefix = Lists/ location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists subscriptions = no } As you can see, the CONTROL/INDEX files are stored per-user to allow private \Seen flags. The different mailinglists are all sent to the 'lists' user which has a Sieve script to file them into different folders, so I have directories on my harddisk like /home/vmail/lists/Maildir/.somelist /home/vmail/lists/Maildir/.anotherlist Now, I'd like to mark the mail in *one* of those folders as \Seen by default. If the INDEX files weren't per-user, it would simply be a matter of using 'addflag "\Seen";' in the Sieve script of the lists user. Alas, this has no effect. Hence my question - how can I have the mail of just one mailinglist get marked as "read" for all users? So far, the only option I see is to add a second public namespace, with a different prefix - and this namespace doesn't use private CONTROL/INDEX files. However, I'd like to keep using the "Lists" prefix if possible to avoid too many changes to the clients. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From jbates at brightok.net Wed Oct 17 16:02:47 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:02:47 -0500 Subject: [Dovecot] Dovecot failed logins delay all logins In-Reply-To: <507E53BA.7030001@optusnet.com.au> References: <507E4C01.6010303@optusnet.com.au> <507E53BA.7030001@optusnet.com.au> Message-ID: <507EAC77.6060401@brightok.net> On 10/17/2012 1:44 AM, Dominic Malolepszy wrote: > I think I found a solution to this thanks to a post by Timo here: > http://dovecot.org/list/dovecot/2011-December/062631.html > > service anvil { unix_listener anvil-auth-penalty { mode = 0 } } > You can also leave IP based penalties and set your other servers such as proxy and webmail as trusted. Jack From jbates at brightok.net Wed Oct 17 16:07:06 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:07:06 -0500 Subject: [Dovecot] dovecot died In-Reply-To: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> References: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Message-ID: <507EAD7A.2080100@brightok.net> Read the wiki that was linked. It could be anything from ntpd/OS/hardware bug. It isn't uncommon for there to be TSC timing issues as well. I have a cutting edge server that has a bug that breaks TSC and causes timing issues. Luckily, my OS is relatively good at not stepping backwards in time. Jack On 10/17/2012 1:26 AM, Hendrik wrote: > Hi All > > I have been trying to get this website running for months now. I get this > emails from cpanel and don't know how to fix it. If anyone can help me > please contact me at pipefab at mweb.co.za. > > > > Kind regards > > Hendrik > > imap failed @ Fri Oct 12 05:51:18 2012. A restart was attempted > automagically. > > > > Service Check Method: [check command] > > > > Number of Restart Attempts: 1 > > > > Cmd Service Check Raw Output: dovecot is not running > > > > > > Startup Log: /etc/init.d/dovecot: line 15: 6043 Alarm clock > /usr/sbin/dovecot > /dev/null 2>&1 > > > > > > Syslog Messages: Oct 12 04:01:01 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3 > yj43fuh...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:01:02 scene dovecot: > IMAP(__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3y > j43fuhjgeiqomc3dhlkyjwdq): Disconnected: Logged out bytes=11/340 Oct 12 > 04:06:06 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0 > jaovpdr...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:06:06 scene dovecot: > IMAP(__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0j > aovpdrgqjnf0_rxyi0wncetn): Disconnected: Logged out bytes=11/313 Oct 12 > 04:11:01 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4z > k8h05nx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:11:01 scene dovecot: > IMAP(__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4zk > 8h05nx2p9n4yfxhrp3a2gjhl): Disconnected: Logged out bytes=11/313 Oct 12 > 04:16:08 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigs > c75cnov...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:16:09 scene dovecot: > IMAP(__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigsc > 75cnovslbll4702ue2veu2n3): Disconnected: Logged out bytes=11/318 Oct 12 > 04:21:10 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsf > ou8d5ky...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:21:11 scene dovecot: > IMAP(__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsfo > u8d5kysrfeaqvlmjgx6afvnb): Disconnected: Logged out bytes=11/340 Oct 12 > 04:26:35 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoe > oga08za...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:26:36 scene dovecot: > IMAP(__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoeo > ga08zaq_nh6yzqsmveqpvnxi): Disconnected: Logged out bytes=11/313 Oct 12 > 04:31:05 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaus > tpvzl1q...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:31:05 scene dovecot: > IMAP(__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaust > pvzl1qjjei5uuoi1c4g8tpea): Disconnected: Logged out bytes=11/313 Oct 12 > 04:36:02 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjj > fzonng9...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:36:03 scene dovecot: > IMAP(__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjjf > zonng9eqrsw5l5hg7xoejer2): Disconnected: Logged out bytes=11/340 Oct 12 > 04:41:03 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gw > eibvov1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:41:04 scene dovecot: > IMAP(__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gwe > ibvov1_minfh7j4_4ejaz7v2): Disconnected: Logged out bytes=11/340 Oct 12 > 04:46:04 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhd > rhafjur...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:46:05 scene dovecot: > IMAP(__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhdr > hafjureydiuxbtbk2jkpkvlo): Disconnected: Logged out bytes=11/340 Oct 12 > 04:51:06 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrq > h1iztqo...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:51:06 scene dovecot: > IMAP(__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrqh > 1iztqosnzfwt28kkzv4riyd9): Disconnected: Logged out bytes=11/340 Oct 12 > 04:56:07 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan > 2pdmdvg...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:56:07 scene dovecot: > IMAP(__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan2 > pdmdvg6jofzylncdb3ytjaaz): Disconnected: Logged out bytes=11/340 Oct 12 > 05:01:08 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvir > quwcv1z...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:01:08 scene dovecot: > IMAP(__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvirq > uwcv1zhloqt12rqni_o2pqcj): Disconnected: Logged out bytes=11/340 Oct 12 > 05:06:09 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgr > cpi3usb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:06:09 scene dovecot: > IMAP(__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgrc > pi3usbdjkiy2n8zy2bbvhhny): Disconnected: Logged out bytes=11/340 Oct 12 > 05:11:10 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg > 4vew3df...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:11:10 scene dovecot: > IMAP(__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg4 > vew3dfbyrglsasuldldaspck): Disconnected: Logged out bytes=11/340 Oct 12 > 05:16:11 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0 > e2bwtas...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:16:11 scene dovecot: > IMAP(__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0e > 2bwtastu0kton3azlhmmuhwi): Disconnected: Logged out bytes=11/340 Oct 12 > 05:21:12 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszq > mf4h8z1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:21:12 scene dovecot: > IMAP(__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszqm > f4h8z1shjl34q9zpid3g4gsp): Disconnected: Logged out bytes=11/340 Oct 12 > 05:26:13 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaog > ha3yc0w...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:26:13 scene dovecot: > IMAP(__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaogh > a3yc0w6t7vvgyf1snz8vechf): Disconnected: Logged out bytes=11/340 Oct 12 > 05:31:14 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62 > mucudsm...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:31:15 scene dovecot: > IMAP(__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62m > ucudsmjfmyolzcpm9shakkiw): Disconnected: Logged out bytes=11/340 Oct 12 > 05:36:15 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxc > kmn7fkd...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:36:16 scene dovecot: > IMAP(__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxck > mn7fkdpsvbrjptqanfuljfv2): Disconnected: Logged out bytes=11/340 Oct 12 > 05:41:15 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqun > f1neeqb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:41:16 scene dovecot: > IMAP(__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqunf > 1neeqbmzas00uqbzmmjsxiab): Disconnected: Logged out bytes=11/340 Oct 12 > 05:46:17 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfre > yewqvxx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:46:17 scene dovecot: > IMAP(__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfrey > ewqvxxeoo7yhyiki7ghukxss): Disconnected: Logged out bytes=11/340 Oct 12 > 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 > seconds. This might cause a lot of problems, so I'll just kill myself now. > http://wiki.dovecot.org/TimeMovedBackwards > > > > From jbates at brightok.net Wed Oct 17 16:44:38 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:44:38 -0500 Subject: [Dovecot] Per user quotas In-Reply-To: <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> Message-ID: <507EB646.5090702@brightok.net> My recommendation is that you use Passwd-file instead of Passwd and specify /etc/passwd. I mention this, as Passwd can use NSS and may not give you the results you want. Passwd-file will guarantee you use the /etc/passwd file. Also, I'm not as familiar with v1.x, but I know in v2.1.10, a userdb lookup doesn't use the userdb_ prefix. So you can try it with and without that prefix. userdb_ prefix on v2 is for cases where you do a prefetch on the passdb. I hope this helps. I've been using Passwd-file only for proxy settings and ldap for my backends to handle quota. Jack On 10/17/2012 2:42 AM, David Travers wrote: > Hi Jake, > > Yep, similar to what I had been trying, but it doesn't seem to be working. > > In my /etc/passwd file I had the line:- > dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash > > I have changed it to show:- > dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M > > Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? > > Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange > > Also yes, I noticed the numbering and have corrected. > > Thanks for your quick reply. > > Dave > > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates > Sent: 17 October 2012 04:02 > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Per user quotas > > On 10/16/2012 11:39 AM, David Travers wrote: >> Hi All, >> >> I keep going round in circles with this. >> I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! >> >> I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. >> >> I believe I need to add to a passwd file, but I have added to that and nothing has changed. >> >> > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M > > Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields > > Note that the extra fields are prefixed with userdb_ when placed in a passwd file. > > Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). > > > Jack > > > > _________________________________________ > This email has been scanned for malicious content. > _________________________________________ > From dg at dguhl.org Wed Oct 17 17:51:44 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 17 Oct 2012 16:51:44 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350429674.3360.27.camel@fermat.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> Message-ID: <20121017145144.GA777@PC211.ikt.de> On Wed, Oct 17, 2012 at 01:21:14AM +0200, Christoph Anton Mitterer wrote: > Hi. [..] > First thing I tried was to simply copy mail within Evolution (i.e. > dragging&dropping it from the local folders to the IMAP folders from > dovecot). This seems to be the smartest idea. > - that preserves the status from Evolution, but doesn't restore that > from Thunderbird Why not use TB to copy the emails from your 'TB mboxes' to Dovecot? This way I moved around 25 GiB of emails from >> 50 mbox files, created with TB 3.6 way down to some 0.x beta, to Dovecot -- without any issues I could recall. [..] > - neither does it remove the Thunderbird or the X-Evolution-Source > headers If they bug you remove them with sed or awk or perl or python or ... Dennis [..] From calestyo at scientia.net Wed Oct 17 20:57:38 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 19:57:38 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121017145144.GA777@PC211.ikt.de> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1350496658.27003.6.camel@heisenberg.scientia.net> On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: > > First thing I tried was to simply copy mail within Evolution (i.e. > > dragging&dropping it from the local folders to the IMAP folders from > > dovecot). > This seems to be the smartest idea. Well as I've mentioned... on looses the info in the From_ lines (that is the RCPT TO address and the date of arrival) because Evolution does not correctly migrated them (actually I'm not sure whether IMAP would allow that). > > - that preserves the status from Evolution, but doesn't restore that > > from Thunderbird > Why not use TB to copy the emails from your 'TB mboxes' to Dovecot? > This way I moved around 25 GiB of emails from >> 50 mbox files, > created with TB 3.6 way down to some 0.x beta, to Dovecot -- without > any issues I could recall. Sorry... too late for that... cause back in the "old" days when I went away from TB I didn't notice that the used other mail headers for their statuses... so now everthing is already mixed together. > If they bug you remove them with sed or awk or perl or python or ... Yeah... but sed alone is not enough... cause such lines may also appear in the body... and I mustn't remove them... So in principle I'm looking for a smart parser of mbox which already gives me headers and body and I can modify either. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From rob0 at gmx.co.uk Wed Oct 17 21:12:36 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 17 Oct 2012 13:12:36 -0500 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350496658.27003.6.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> Message-ID: <20121017181236.GN3672@harrier.slackbuilds.org> On Wed, Oct 17, 2012 at 07:57:38PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: > > > First thing I tried was to simply copy mail within Evolution > > > (i.e. dragging&dropping it from the local folders to the IMAP > > > folders from dovecot). > > This seems to be the smartest idea. > Well as I've mentioned... on looses the info in the From_ lines > (that is the RCPT TO address and the date of arrival) because > Evolution does not correctly migrated them (actually I'm not sure > whether IMAP would allow that). Perhaps you mean the "^From " mbox delimiter line. You do not need mbox delimiters in maildir files. Did you mention whether or not you're using maildir? -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From calestyo at scientia.net Wed Oct 17 21:21:47 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 20:21:47 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121017181236.GN3672@harrier.slackbuilds.org> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> Message-ID: <1350498107.27003.10.camel@heisenberg.scientia.net> On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: > > Well as I've mentioned... on looses the info in the From_ lines > > (that is the RCPT TO address and the date of arrival) because > > Evolution does not correctly migrated them (actually I'm not sure > > whether IMAP would allow that). > Perhaps you mean the "^From " mbox delimiter line. Yes I meant them (the _ should have denoted the space) > You do not need > mbox delimiters in maildir files. I know.. > Did you mention whether or not > you're using maildir? The reason is mainly that I have gazillions of mail in a ~ 60 GB archive... even with an fs optimised for small files I'd loose far too much space per mail than I want to afford. Also, AFAIK full text search becomes much solver in maildir (as you need to open/close endless files). On the longterm view I want to have a look into things like dbmail/archiveopteryx... for the giant local archive... and keep dovecot "only" as the internet mail server. Ideally dovecot would have such an SQL backend...or incorporate that part from Archiveopteryx. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From rob0 at gmx.co.uk Wed Oct 17 21:36:47 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 17 Oct 2012 13:36:47 -0500 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350498107.27003.10.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> <1350498107.27003.10.camel@heisenberg.scientia.net> Message-ID: <20121017183647.GO3672@harrier.slackbuilds.org> On Wed, Oct 17, 2012 at 08:21:47PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: > > Did you mention whether or not you're using maildir? > The reason is mainly that I have gazillions of mail in a ~ 60 GB > archive... even with an fs optimised for small files I'd loose far > too much space per mail than I want to afford. Fine, maildir is not the perfect solution for everyone. But I'm confused about why Evolution/Thunderbird local folders to IMAP folders does not work. That should be the best approach. If it does not work, you're going to have some perl/python/ruby scripting to do. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From list at airstreamcomm.net Wed Oct 17 22:27:04 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 17 Oct 2012 14:27:04 -0500 Subject: [Dovecot] Dsync clustering Message-ID: <507F0688.6000707@airstreamcomm.net> I have not seen mention of using dsync for clustering Dovecot in some time on the mailing list, but I believe Timo was going to write a wiki page when v2.2 became more mature. Does this documentation exist yet, or are there any resources on what dsync replication is capable of at this point (looking on the wiki and google didn't reveal much)? Thank in advance. From roundcube222 at alaadin.org Wed Oct 17 21:33:42 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 17 Oct 2012 21:33:42 +0300 Subject: [Dovecot] Disconnected for inactivity time. Message-ID: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> Hello, After 10 mins of unactivity of pop3 , dovecot disconnect the user (-ERR Disconnected for inactivity.) What is the option in the config file which control the unactivity logout time ? becuase i want to decrease the inactivity time To 5 mins instead of 10 mins Please advise Robert JR From slusarz at curecanti.org Wed Oct 17 22:47:43 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Wed, 17 Oct 2012 13:47:43 -0600 Subject: [Dovecot] Disconnected for inactivity time. In-Reply-To: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> References: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> Message-ID: <20121017134743.Horde.af5lO4F5lbhQfwtfwboGUbA@bigworm.curecanti.org> Quoting Robert JR : > After 10 mins of unactivity of pop3 , dovecot disconnect the user > (-ERR Disconnected for inactivity.) > > What is the option in the config file which control the unactivity > logout time ? becuase i want to decrease the inactivity time > > To 5 mins instead of 10 mins You can't (at least without hacking the code). The POP3 specification **requires** a minimum of 10 minutes before an autologout occurs (RFC 1939 [3]): http://tools.ietf.org/html/rfc1939#section-3 michael From stephan at rename-it.nl Wed Oct 17 23:12:00 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 17 Oct 2012 22:12:00 +0200 Subject: [Dovecot] Dovecot sieve and duplicate email subjects In-Reply-To: References: Message-ID: <507F1110.3020701@rename-it.nl> On 8/27/2012 12:40 PM, Benjamin Thomas wrote: > I was wondering if it's possible to gracefully handle duplicate subjects > lines (within a given time frame) with sieve filters ? > > Ideally, I would like the first email of the day with subject ""AutoAlert > Type1..." to get moved to the subfolder given above. Subsequent "duplicate" > emails would then go into a "duplicate" subfolder. > > Is this possible with sieve filters ? I could not find anything striking me > as obvious while reading the page http://wiki.dovecot.org/LDA/Sieve. Although the above is still not possible with the standard Sieve features, I updated the Dovecot-specific vnd.dovecot.duplicate extension with support for the scenario you describe. Basically, I made it much more flexible for checking all kinds of string value duplicates and not only strictly based on the Message-ID header. Also, the user can now directly control the time frame within which duplicates are detected (within configurable limits). http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt It will be included in the next Pigeonhole v0.3 release. Regards, Stephan. From limon at koli.be Thu Oct 18 01:40:45 2012 From: limon at koli.be (Levent Dane) Date: Wed, 17 Oct 2012 17:40:45 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c Message-ID: <456733b1b04e92265fbd9ba8e005132c@koli.be> First, I don't know why but dovecot gots this error. Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) Then, when I tried to run SELECT "INBOX/Code" (UNSEEN) virtual plugin got a segfault. Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] I'm running gentoo with kernel 2.6.32.12. My dovecot version is 2.1.10. I applied http://hg.dovecot.org/dovecot-2.1/raw-diff/0306792cc843/src/lib-storage/mail-search.c But still, i'm getting this problem. Thanks, Levent Dane From mailadmin at cubixys.com Thu Oct 18 02:16:55 2012 From: mailadmin at cubixys.com (Fasil) Date: Thu, 18 Oct 2012 02:16:55 +0300 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> Message-ID: <507F3C67.5020900@cubixys.com> Thanks Frank. Followed the URL and could not find any luck. Is there a way to change the value of 'max open files' of dovecot. I have tried to set the value in /etc/default/dovecot by setting ulimit. But the value is not getting applied. Could anyone help on this regard. Fasil. On 08/16/2012 09:17 AM, Frank Bonnet wrote: > hello > > here some useful informations > > http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ > > Envoy? de mon iPhone. > > > Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : > >> Dear all, >> >> Thank you all for such a wonderful support. Hats off to all :) >> >> Few times I came across imap login issues where new users will not be allowed to login. >> /var/log/mail.err shows the error below >> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in creating login processes, slowing down for now >> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open files >> >> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >> >> # ulimit -Hn >> 1024 >> >> # ulimit -Sn >> 1024 >> >> # cat /proc/sys/fs/file-max >> 1238548 >> >> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >> Max open files 1024 1024 files >> >> Please advice how to get rid off this. >> >> Fasil. From jtam.home at gmail.com Thu Oct 18 02:46:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 17 Oct 2012 16:46:30 -0700 (PDT) Subject: [Dovecot] Disconnected for inactivity time. In-Reply-To: References: Message-ID: Robert JR writes: > After 10 mins of unactivity of pop3 , dovecot disconnect the user (-ERR > Disconnected for inactivity.) > > What is the option in the config file which control the unactivity > logout time ? becuase i want to decrease the inactivity time > > To 5 mins instead of 10 mins Looks like it's set in the source code pop3-client.c:#define CLIENT_IDLE_TIMEOUT_MSECS (10*60*1000) but the output of # doveconf -a ... service pop3 { ... idle_kill = 0 ... } maybe points to the config that overrides this. Easy enough to test. It's non-RFC compliant as one poster points out, so unless you got a good reason to do this like lots of zombie pop3 processes, leave it alone. Joseph Tam From web at guzman.com.ar Thu Oct 18 02:47:09 2012 From: web at guzman.com.ar (Ricardo) Date: Wed, 17 Oct 2012 20:47:09 -0300 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze Message-ID: Hello list apologize in advance for my bad English, this is the first time I write to a list if I mistake Excuse me I want to implement mail server with MySQL database, Postfix and Postfixadmin, dovecot-core, dovecot-mysql dovecot-imapd dovecot-pop3d for multiple domains. I have problems installing the daemon dovecot-core, dovecot-mysql dovecot-imapd dovecot-pop3d Debian squeeze, I'm using the repositories: deb http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze main deb-src http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze main deb http://security.debian.org/ squeeze/updates main deb-src http://security.debian.org/ squeeze/updates main # squeeze-updates, previously known as 'volatile' deb http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze-updates main deb-src http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze-updates main deb http://packages.dotdeb.org squeeze all deb-src http://packages.dotdeb.org squeeze all deb http://backports.debian.org/debian-backports squeeze-backports main to install dovecot-core, dovecot-mysql, install it without problems is the version (2.1.7-2 ~ bpo60 +1) of both packages, now wanting to install dovecot-imapd dovecot-pop3d (version 1.2.15-7) breaks the dovecot-core, dovecot-mysql, apparently must be the same version all packages. Debian Wheezy, installs without problems but installs the version (dovecot-core_2.1.7-2 ~ ppa12.04 +1 _i386.deb) all packets are the same version. What is the correct version for Debian squeeze? Look for San Google but eh had success. Can anybody help? Ricardo From rfs9999 at earthlink.net Wed Oct 17 18:53:24 2012 From: rfs9999 at earthlink.net (Rick Sanders) Date: Wed, 17 Oct 2012 15:53:24 +0000 (UTC) Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: Your best bet for a clean migration is to use an IMAP migration tool (assuming both of your servers support IMAP). It avoids all of the issues surrounding the underlying databases used to store the mailboxes and messages since everything is done through IMAP commands. There are lots of different IMAP tools out there, some free some not. Using an IMAP migration tool is usually straight-forward and simple. Here is a list of some of them. Full disclosure, imap_tools is mine. imapsync: http://imapsync.lamiral.info imap_tools: http://www.athensfbc.com/imap_tools offlineimap: https://github.com/nicolas33/offlineimap mbsync: http://isync.sourceforge.net/ mailsync: http://mailsync.sourceforge.net/ mailutil: http://www.washington.edu/imap/ part of the UW IMAP tookit. imaprepl: http://www.bl0rg.net/software/ http://freecode.com/projects/imap-repl/ imapcopy: http://home.arcor.de/armin.diehl/imapcopy/imapcopy.html migrationtool: http://sourceforge.net/projects/migrationtool/ imapmigrate: http://sourceforge.net/projects/cyrus-utils/ larch: https://github.com/rgrove/larch (derived from wonko_imapsync) wonko_imapsync: http://wonko.com/article/554 pop2imap: http://www.linux-france.org/prj/pop2imap/ exchange-away: http://exchange-away.sourceforge.net/ To copy all of a user's mailboxes from one IMAP server to another using my imapcopy tool is as simple as executing the following command: imapcopy.pl -S source/username/password -D destination/user/password Regards, Rick From jbates at brightok.net Thu Oct 18 05:13:25 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 21:13:25 -0500 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <507F3C67.5020900@cubixys.com> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> <507F3C67.5020900@cubixys.com> Message-ID: <507F65C5.3090803@brightok.net> I'm using RHEL6 instead of ubuntu, but check the startup scripts. In RHEL's case, the following file is sourced, so I updated it instead of the startup scripts. cat /etc/sysconfig/dovecot # Here you can specify your dovecot command line options. # #OPTIONS="" ulimit -n 4096 ulimit -u 5120 In addition, I had to also up the max allowed processes in the dovecot config. 2.x and 1.x are different on this. http://wiki1.dovecot.org/LoginProcess <-1.x method Jack On 10/17/2012 6:16 PM, Fasil wrote: > Thanks Frank. > Followed the URL and could not find any luck. > > Is there a way to change the value of 'max open files' of dovecot. > I have tried to set the value in /etc/default/dovecot by setting > ulimit. But the value is not getting applied. > Could anyone help on this regard. > > Fasil. > > On 08/16/2012 09:17 AM, Frank Bonnet wrote: >> hello >> >> here some useful informations >> >> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ >> >> Envoy? de mon iPhone. >> >> >> Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : >> >>> Dear all, >>> >>> Thank you all for such a wonderful support. Hats off to all :) >>> >>> Few times I came across imap login issues where new users will not >>> be allowed to login. >>> /var/log/mail.err shows the error below >>> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in creating >>> login processes, slowing down for now >>> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> >>> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >>> >>> # ulimit -Hn >>> 1024 >>> >>> # ulimit -Sn >>> 1024 >>> >>> # cat /proc/sys/fs/file-max >>> 1238548 >>> >>> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >>> Max open files 1024 1024 files >>> >>> Please advice how to get rid off this. >>> >>> Fasil. > > From jbates at brightok.net Thu Oct 18 05:30:58 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 21:30:58 -0500 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze In-Reply-To: References: Message-ID: <507F69E2.7050801@brightok.net> On 10/17/2012 6:47 PM, Ricardo wrote: > > to install dovecot-core, dovecot-mysql, install it without problems is > the version (2.1.7-2 ~ bpo60 +1) of both packages, > now wanting to install dovecot-imapd dovecot-pop3d (version 1.2.15-7) > breaks the dovecot-core, dovecot-mysql, > apparently must be the same version all packages. > > I think something is wrong with your debian setup. 1.2 is normal version. 2.1 is the backports version. You should be getting this: Package: dovecot-imapd Priority: optional Section: mail Installed-Size: 559 Maintainer: Dovecot Maintainers Architecture: i386 Source: dovecot Version: 1:2.1.7-2~bpo60+1 Provides: imap-server Depends: dovecot-core (= 1:2.1.7-2~bpo60+1), libc6 (>= 2.4), ucf (>= 2.0020) That's from the Packages list http://backports.debian.org/debian-backports/dists/squeeze-backports/main/binary-i386/ Jack From tss at iki.fi Thu Oct 18 06:06:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:06:27 +0300 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <456733b1b04e92265fbd9ba8e005132c@koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> Message-ID: <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> On 18.10.2012, at 1.40, Levent Dane wrote: > First, I don't know why but dovecot gots this error. > Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) > Then, when I tried to run > SELECT "INBOX/Code" (UNSEEN) > virtual plugin got a segfault. > Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html From tss at iki.fi Thu Oct 18 06:07:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:07:39 +0300 Subject: [Dovecot] Dsync clustering In-Reply-To: <507F0688.6000707@airstreamcomm.net> References: <507F0688.6000707@airstreamcomm.net> Message-ID: On 17.10.2012, at 22.27, list at airstreamcomm.net wrote: > I have not seen mention of using dsync for clustering Dovecot in some time on the mailing list, but I believe Timo was going to write a wiki page when v2.2 became more mature. Does this documentation exist yet, or are there any resources on what dsync replication is capable of at this point (looking on the wiki and google didn't reveal much)? Thank in advance. You can probably find some mails from this mailing list. Try searching for "dsync replication". From tss at iki.fi Thu Oct 18 06:10:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:10:03 +0300 Subject: [Dovecot] Per user quotas In-Reply-To: <507EB646.5090702@brightok.net> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> <507EB646.5090702@brightok.net> Message-ID: Correct, except I wouldn't go modifying /etc/passwd directly in any case. Other software besides Dovecot might not like it. Better to create a whole new /etc/dovecot/passwd or something. On 17.10.2012, at 16.44, Jack Bates wrote: > My recommendation is that you use Passwd-file instead of Passwd and specify /etc/passwd. I mention this, as Passwd can use NSS and may not give you the results you want. Passwd-file will guarantee you use the /etc/passwd file. Also, I'm not as familiar with v1.x, but I know in v2.1.10, a userdb lookup doesn't use the userdb_ prefix. So you can try it with and without that prefix. userdb_ prefix on v2 is for cases where you do a prefetch on the passdb. > > I hope this helps. I've been using Passwd-file only for proxy settings and ldap for my backends to handle quota. > > Jack > > On 10/17/2012 2:42 AM, David Travers wrote: >> Hi Jake, >> >> Yep, similar to what I had been trying, but it doesn't seem to be working. >> >> In my /etc/passwd file I had the line:- >> dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash >> >> I have changed it to show:- >> dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M >> >> Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? >> >> Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange >> >> Also yes, I noticed the numbering and have corrected. >> >> Thanks for your quick reply. >> >> Dave >> >> -----Original Message----- >> From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates >> Sent: 17 October 2012 04:02 >> To: dovecot at dovecot.org >> Subject: Re: [Dovecot] Per user quotas >> >> On 10/16/2012 11:39 AM, David Travers wrote: >>> Hi All, >>> >>> I keep going round in circles with this. >>> I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! >>> >>> I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. >>> >>> I believe I need to add to a passwd file, but I have added to that and nothing has changed. >>> >>> >> user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M >> >> Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields >> >> Note that the extra fields are prefixed with userdb_ when placed in a passwd file. >> >> Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). >> >> >> Jack >> >> >> >> _________________________________________ >> This email has been scanned for malicious content. >> _________________________________________ >> > From tss at iki.fi Thu Oct 18 06:14:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:14:19 +0300 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <00e101cdac30$5ab63270$10229750$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> <00e101cdac30$5ab63270$10229750$@fredk.com> Message-ID: On 17.10.2012, at 9.26, Fred Kilbourn wrote: >> doveadm fts rescan makes sure that 1) all of the old messages are >> indexed and 2) there are no extra (already deleted) messages indexed. So >> it's basically repairing fts index. You probably shouldn't run it >> automatically, or at least not very often. > > Okay, you've clarified that for me. > > I understand that rescan isn't a nightly task, but could be run every now > and then periodically. How often might be appropriate if I wanted to do > this as a maintenance task? Once a month? I don't know, depends on if you have problems related to it. I think the most common answer would be "never". > Lastly, I'm trying to use the index command instead of the search command, > but I can't figure out how to make it index every mailbox for every user. > Is there a wildcard that can be used for the mailbox? Or do I need to > iterate all the mailboxes with one command and run index however many times > for each inbox? doveadm index '*' works in new versions. I don't remember from which version. From tss at iki.fi Thu Oct 18 06:18:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:18:49 +0300 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121016091153.15601eysq5n040qh@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> <20121016091153.15601eysq5n040qh@webmail.unipa.it> Message-ID: <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> On 16.10.2012, at 10.11, Benedetto Vassallo wrote: >> What are the permissions of the MailDir directory for user1/user2? >> >> ls -ld /home/user1/MailDir >> ls -ld /home/user2/MailDir >> >> > > Thank you for your reply. > They are different groups: > > drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ > drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ > drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ Not very secure permissions.. Maybe would be easiest to just have one vmail user for everyone? > I tryed to issue: > chgrp -R mail /home/user1/MailDir > chgrp -R mail /home/user2/MailDir > chgrp -R mail /home/user3/MailDir Dovecot doesn't do hard linking when it looks like the permissions aren't compatible. The current code checks that if the owner UIDs are different, then the group needs to be writable. From tss at iki.fi Thu Oct 18 06:23:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:23:58 +0300 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 In-Reply-To: <507DC6D4.7090902@brightok.net> References: <507DC6D4.7090902@brightok.net> Message-ID: <23D747EC-045A-44B0-8A85-29B76B66B969@iki.fi> On 16.10.2012, at 23.43, Jack Bates wrote: > Please check the code. I didn't add it, but a real_lip might be useful for some people as well. Also, I notice that pop3-proxy is doing a different xsession than the imap proxy. Is there an xsession standard that is different between the two, or just an oversight in the code? Both send the remote address/port, but only imap proxy sends the local address/port. > > This patch declares long variable %{real_rip} so that a backend server can declare a different login_log_format_elements > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c > > This is primarily useful for backend servers to log both the rip, lip, and in case of xsession, the real rip. I haven't done extensive testing yet, but as long as nothing does anything weird elsewhere in the code, it should be good. > > http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch Added: http://hg.dovecot.org/dovecot-2.1/rev/92364817f4ba From tss at iki.fi Thu Oct 18 06:31:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:31:19 +0300 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read In-Reply-To: <507EA81C.5060806@froglogic.com> References: <507EA81C.5060806@froglogic.com> Message-ID: On 17.10.2012, at 15.44, Frerich Raabe wrote: > I'm running Dovecot 1.2.17 on FreeBSD 9 to serve an archive of a few internal mailinglists. The archive is implemented using a public namespace: > > namespace private { > separator = / > prefix = > inbox = yes > } > > namespace public { > separator = / > prefix = Lists/ > location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > subscriptions = no > } > > As you can see, the CONTROL/INDEX files are stored per-user to allow private \Seen flags. The different mailinglists are all sent to the 'lists' user which has a Sieve script to file them into different folders, so I have directories on my harddisk like > > /home/vmail/lists/Maildir/.somelist > /home/vmail/lists/Maildir/.anotherlist > > Now, I'd like to mark the mail in *one* of those folders as \Seen by default. If the INDEX files weren't per-user, it would simply be a matter of using 'addflag "\Seen";' in the Sieve script of the lists user. Alas, this has no effect. > > Hence my question - how can I have the mail of just one mailinglist get marked as "read" for all users? You can't with the above settings. It would require writing the seen flag to all users' index files. Not easy to do and definitely not efficient to do. Maybe if there was some kind of a mixed hybrid of accepting seen flag changes from the shared index, but no such code exists (also private/shared index separation doesn't exist before v2.2). > So far, the only option I see is to add a second public namespace, with a different prefix - and this namespace doesn't use private CONTROL/INDEX files. However, I'd like to keep using the "Lists" prefix if possible to avoid too many changes to the clients. Use: prefix=Lists/anotherlist/ location = maildir:/home/vmail/lists/sharedseen/Maildir Then deliver the mails to /home/vmail/lists/sharedseen/Maildir root directly. Of course this means that you need to create a namespace for each such list. Alternative would be to use prefix=Lists/sharedseen/ and create lists under it. From jbates at brightok.net Thu Oct 18 06:48:42 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 22:48:42 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507C6DD3.2000309@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> Message-ID: <507F7C1A.9030301@brightok.net> Timo, How do you feel about parent pointers in child structures? I'm curious as the proxy structure is passed the input channel, but it doesn't know much else about the input client. Rather than pass additional information in the structure, I think it'd be better to just place a pointer back to the input client so we can access all it's details. I ask, as that might solve the problem of lack of information in logging from some of the various functions in the proxy code. I know I was limited in the quick patch I did below for my own use. I'm afraid to change it too much. You have already started xclient work in v2.2 which would necessitate a lot of changes to the lmtp/proxy code. I've actually debated backporting it to 2.1 for my own use. :) Jack On 10/15/2012 3:10 PM, Jack Bates wrote: > On 10/15/2012 2:07 PM, Jack Bates wrote: >> On 10/12/2012 2:40 AM, Timo Sirainen wrote: >>> would probably complicate the code. >>> I don't think this would be difficult to implement. Probably just a >>> few lines of code. Yeah, could be useful. >>> >>> >> > > If there's no argument over the last email, confirm and check this > patch. It's not the overall logging I would like, but the lmtp code > isn't as mature as pop3/imap and the proxy is a quick and dirty on the > lmtp code. Both need a good revamp, preferably with x-session support > and perhaps logging rip/lip similar to how we do pop3/imap logins. > > I think we should also work on adjusting all logging for services > using x-session to also log the proxy ip. rip,lip,pip. As I get time > I'll look at it. > > This patch is just to keep us from having no useful logging in lmtp > proxy. Based on lmtp pid, one can at least follow the connect, the > proxy replies, and the disconnect of a session. > > --- dovecot-2.1.10/src/lmtp/lmtp-proxy.c 2012-10-12 > 19:46:49.688952484 +0000 > +++ dovecot-2.1.10/src/lmtp/lmtp-proxy.c-new 2012-10-12 > 19:48:51.751932325 +0000 > @@ -160,6 +160,8 @@ static bool lmtp_proxy_send_data_replies > break; > o_stream_send_str(proxy->client_output, > t_strconcat(rcpt[i]->reply, "\r\n", > NULL)); > + i_info("proxy(%s): proxy host=%s: > status=%s",rcpt[i]->address, > + rcpt[i]->conn->set.host,rcpt[i]->reply); > } > o_stream_uncork(proxy->client_output); > proxy->next_data_reply_idx = i; > > From tss at iki.fi Thu Oct 18 07:07:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 07:07:03 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507F7C1A.9030301@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> Message-ID: On 18.10.2012, at 6.48, Jack Bates wrote: > How do you feel about parent pointers in child structures? I'm curious as the proxy structure is passed the input channel, but it doesn't know much else about the input client. Rather than pass additional information in the structure, I think it'd be better to just place a pointer back to the input client so we can access all it's details. Generally speaking it's cleaner to keep things as separate as possible. Maybe instead of proxy getting lmtp_client pointer both of them could contain a shared struct lmtp_client_info or something like that. But in any case I'll probably more or less rewrite the whole LMTP code at some point, because I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) From limon at koli.be Thu Oct 18 09:03:55 2012 From: limon at koli.be (Levent Dane) Date: Thu, 18 Oct 2012 01:03:55 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> Message-ID: <20121018060354.GA2528@leningrad.koli.be> On 10/18, Timo Sirainen wrote: >On 18.10.2012, at 1.40, Levent Dane wrote: >> First, I don't know why but dovecot gots this error. >> Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) >> Then, when I tried to run >> SELECT "INBOX/Code" (UNSEEN) >> virtual plugin got a segfault. >> Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] > >I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html in Code/dovecot-virtual: Archive inthread refs keyword code not deleted dovecot -n http://pastebin.com/6CQd7tJK My mail client is Mutt-hg with sidebar patch I tried to take coredump but i didn't compile with debug flags. http://pastebin.com/CMbiYJeK If you can't reproduce this error. Tomorrow, I'll compile with debug flags. Thanks, -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 From janfrode at tanso.net Thu Oct 18 11:05:44 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 18 Oct 2012 10:05:44 +0200 Subject: [Dovecot] trash plugin not doing it's job Message-ID: I enabled the trash plugin yesterday, adding "trash" to mail_plugins, and configuring the plugin setting "trash = /etc/dovecot/dovecot-trash.conf.ext". But I still see users with lots of files in INBOX.Trash getting bounced because of quota exceeded: postfix/lmtp[26273]:: C89F490061: to=, relay=loadbalancers.example.net[192.168.42.15]:24, delay=1.2, delays=0.61/0.02/0/0.54, dsn=5.2.2, status=bounced (host loadbalancers.example.net[192.168.42.15] said: 552 5.2.2 Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) dovecot:: lmtp(19730, XXXXXXX at example.no): Error: BErxFCyrf1ASTQAAWNPRnw: sieve: msgid=: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) $ sudo doveadm quota get -u XXXXXXXX at example.no Quota name Type Value Limit % UserQuota STORAGE 1048559 1048576 99 UserQuota MESSAGE 4487 - 0 Postfix if delivering via LMTP trough dovecot director. Anybody see anything obvious in my config: ------------------------------------------------------------ # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) auth_cache_size = 100 M auth_verbose = yes auth_verbose_passwords = sha1 disable_plaintext_auth = no login_trusted_networks = 192.168.0.0/16 109.247.114.192/27 mail_gid = 3000 mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u mail_max_userip_connections = 20 c = quota zlib trash mail_uid = 3000 maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = dict:UserQuota::file:%h/dovecot-quota sieve = /sieve/%1Lu/%1.1Lu/%Lu/.dovecot.sieve sieve_before = /etc/dovecot/sieve/dovecot.sieve sieve_dir = /sieve/%1Lu/%1.1Lu/%Lu sieve_max_script_size = 1M trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at example.net protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 4521 unix_listener auth-userdb { group = mode = 0600 user = atmail } } service imap-login { inet_listener imap { address = * port = 143 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login /usr/local/sbin/imap-postlogin.sh } service imap { executable = imap imap-postlogin process_limit = 2048 } service lmtp { client_limit = 1 inet_listener lmtp { address = * port = 24 } process_limit = 25 process_min_avail = 10 } service managesieve-login { inet_listener sieve { address = * port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service pop3-postlogin { executable = script-login /usr/local/sbin/pop3-postlogin.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 2048 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota zlib trash sieve } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota zlib trash imap_quota } protocol pop3 { mail_plugins = quota zlib trash pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v } protocol sieve { managesieve_logout_format = bytes=%i/%o } ------------------------------------------------------------ and my trash config: $ cat /etc/dovecot/dovecot-trash.conf.ext # Spam mailbox is emptied before Trash 1 INBOX.Spam # Trash mailbox is emptied before Sent 2 INBOX.Trash Global sieve script: $ cat /etc/dovecot/sieve/dovecot.sieve require ["comparator-i;ascii-numeric","relational","fileinto","mailbox"]; if allof ( not header :matches "x-spam-score" "-*", header :value "ge" :comparator "i;ascii-numeric" "x-spam-score" "10" ) { discard; stop; } elsif allof ( not header :matches "x-spam-score" "-*", header :value "ge" :comparator "i;ascii-numeric" "x-spam-score" "6" ) { fileinto :create "INBOX.Spam"; } -jf From stocton12 at yahoo.com Thu Oct 18 11:33:25 2012 From: stocton12 at yahoo.com (b m) Date: Thu, 18 Oct 2012 01:33:25 -0700 (PDT) Subject: [Dovecot] CAS Authentication In-Reply-To: <507E5D3A.5030900@um.es> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> <507E5D3A.5030900@um.es> Message-ID: <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> Thanks for the configuration files. I have a question. In pam_cas.conf I don't know what to put in "proxy ". In some examples I have seen something like http:///proxy.php Do I need a php file in my webmail to handle the cas tickets and if so where can I find it? ________________________________ From: Angel L. Mateo To: dovecot at dovecot.org Sent: Wednesday, October 17, 2012 10:24 AM Subject: Re: [Dovecot] CAS Authentication El 16/10/12 20:12, b m escribi?: > Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login? with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. > ??? This is my config. In /etc/pam.d/dovecot I have: auth? ? sufficient??? pam_cas_ssh.so -simap://localhost -f/etc/pam_cas.conf account sufficient??? pam_permit.so session sufficient??? pam_permit.so ??? and /etc/pam_cas.conf host port 443 uriValidate /cas/proxyValidate ssl on debug off proxy??? trusted_ca ??? in dovecot, I have these users dbs: userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } passdb { ? args = session=yes cache_key=%n dovecot ? driver = pam } ??? With this, it works fine. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From rs at sys4.de Thu Oct 18 11:42:56 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 18 Oct 2012 10:42:56 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350498107.27003.10.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> <1350498107.27003.10.camel@heisenberg.scientia.net> Message-ID: <507FC110.1040809@sys4.de> Am 17.10.2012 20:21, schrieb Christoph Anton Mitterer: > On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: >>> Well as I've mentioned... on looses the info in the From_ lines >>> (that is the RCPT TO address and the date of arrival) because >>> Evolution does not correctly migrated them (actually I'm not sure >>> whether IMAP would allow that). >> Perhaps you mean the "^From " mbox delimiter line. > Yes I meant them (the _ should have denoted the space) > > >> You do not need >> mbox delimiters in maildir files. > I know.. > > >> Did you mention whether or not >> you're using maildir? > The reason is mainly that I have gazillions of mail in a ~ 60 GB > archive... even with an fs optimised for small files I'd loose far too > much space per mail than I want to afford. > > Also, AFAIK full text search becomes much solver in maildir (as you need > to open/close endless files). On the longterm view I want to have a look > into things like dbmail/archiveopteryx... for the giant local archive... > and keep dovecot "only" as the internet mail server. > > Ideally dovecot would have such an SQL backend...or incorporate that > part from Archiveopteryx. > > > Cheers, > Chris. > this may help too http://www.stchman.com/export_evolution.html http://www.ubuntugeek.com/how-to-export-your-mails-from-evolution-to-thunderbird.html http://ubuntuforums.org/showthread.php?t=1760469 http://ubuntuforums.org/showthread.php?t=1870445 http://jaisejames.wordpress.com/2012/03/15/to-activate-maildir-in-thunderbird/ http://realtechtalk.com/ThunderbirdMBOX_to_IMAPMaildir_migration_done_easy_with_mb2md-1134-articles -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From fabio.ferrari at unimore.it Thu Oct 18 11:51:37 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Thu, 18 Oct 2012 10:51:37 +0200 Subject: [Dovecot] Problem with process_limit In-Reply-To: References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: <370893b18f6c82ba13f4cb31d19ea259.squirrel@webmail2.unimore.it> Yes, thanks, it seems that this configuration changed something, but I think there is something else. Now this particular warning in the dovecot.log disappeared, but it shows these lines instead: Oct 17 10:55:57 imap-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 17 10:55:57 imap-login: Fatal: Couldn't connect to anvil Oct 17 10:56:12 pop3-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 17 10:56:12 pop3-login: Fatal: Couldn't connect to anvil the result is quite the same, I have to reload the dovecot because it does'n accept connections. I tried to add these lines in /etc/dovecot/conf.d/10-master.conf: service anvil { client_limit = 5000 } but without good results. Any ideas? thanks in advance Fabio Ferrari > On 1.10.2012, at 12.15, FABIO FERRARI wrote: > >> Occasionally, it happens that the dovecot.log shows this line: >> master: Warning: service(imap): process_limit reached, client >> connections >> are being dropped > .. >> Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the >> line >> process_limit = 1500 > > But did you set it inside service imap {}? All of the services have > process_limit parameter. > > From CMarcus at Media-Brokers.com Thu Oct 18 14:22:35 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 18 Oct 2012 07:22:35 -0400 Subject: [Dovecot] lmtp proxy logging In-Reply-To: References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> Message-ID: <507FE67B.4030705@Media-Brokers.com> On 2012-10-18 12:07 AM, Timo Sirainen wrote: > I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) Hey Timo, I hope this means what it sounds like it means... Can you confirm that this 'submission server' would support the ability to automatically add a copy of all emails sent using it to the designated 'Sent' folder, so that email clients could simply disable the 'Save a copy to Sent folder' feature (that causes the client to upload the message to the server twice, once to send the message, and again to save the Sent copy)? This is one feature of gmail that I simply love... Thanks as always, -- Best regards, Charles From amateo at um.es Thu Oct 18 14:23:47 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 18 Oct 2012 13:23:47 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> <507E5D3A.5030900@um.es> <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <507FE6C3.80702@um.es> El 18/10/12 10:33, b m escribi?: > Thanks for the configuration files. I have a question. In pam_cas.conf I don't know what to put in "proxy ". In some examples I have seen something like http:///proxy.php > Do I need a php file in my webmail to handle the cas tickets and if so where can I find it? > You need this script at your webmail server. This script depend on the webmail you are using. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From tss at iki.fi Thu Oct 18 14:30:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 14:30:00 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507FE67B.4030705@Media-Brokers.com> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> <507FE67B.4030705@Media-Brokers.com> Message-ID: <141971D9-6DC7-4BEF-B4AE-B1EBD5314499@iki.fi> On 18.10.2012, at 14.22, Charles Marcus wrote: > On 2012-10-18 12:07 AM, Timo Sirainen wrote: >> I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) > > Hey Timo, > > I hope this means what it sounds like it means... > > Can you confirm that this 'submission server' would support the ability to automatically add a copy of all emails sent using it to the designated 'Sent' folder, so that email clients could simply disable the 'Save a copy to Sent folder' feature (that causes the client to upload the message to the server twice, once to send the message, and again to save the Sent copy)? That's not the intended reason for creating it, but easy enough to add as an option, assuming \Sent SPECIAL-USE mailbox is defined. Anyway, I don't know when I'll actually start implementing it. Mainly just a "would be nice to have some day" thing to support LEMONADE SMTP extensions. From dg at dguhl.org Thu Oct 18 15:34:41 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 18 Oct 2012 14:34:41 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350496658.27003.6.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> Message-ID: <20121018123440.GA29330@laptop-dg.leere.eu> On Wed, Oct 17, 2012 at 07:57:38PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: [move through Evolution to IMAP] > Well as I've mentioned... on looses the info in the From_ lines (that is > the RCPT TO address and the date of arrival) because Evolution does not The date and time of arrival can be concluded from the last Received: header. The RCPT TO need to be converted to a X-Original-To: header. [..] > > If they bug you remove them with sed or awk or perl or python or ... > Yeah... but sed alone is not enough... cause such lines may also appear > in the body... and I mustn't remove them... > So in principle I'm looking for a smart parser of mbox which already > gives me headers and body and I can modify either. I think, like Rob suggested, you are in need of some serious scripting. Dennis From dg at dguhl.org Thu Oct 18 17:24:02 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 18 Oct 2012 16:24:02 +0200 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze In-Reply-To: References: Message-ID: <20121018142400.GA1261@PC211.ikt.de> On Wed, Oct 17, 2012 at 08:47:09PM -0300, Ricardo wrote: > Hello list [..] > I have problems installing the daemon dovecot-core, dovecot-mysql > dovecot-imapd dovecot-pop3d [..] > to install dovecot-core, dovecot-mysql, install it without problems > is the version (2.1.7-2 ~ bpo60 +1) of both packages, now wanting to > install dovecot-imapd dovecot-pop3d (version 1.2.15-7) breaks the > dovecot-core, dovecot-mysql, apparently must be the same version all > packages. Yea, of course you must use the same version for all packages. Upstream there is only one package for dovecot. It's part of Debian's philosophy to split monolithic packages into a bunch separate packages. > Debian Wheezy, installs without problems but installs the version > (dovecot-core_2.1.7-2 ~ ppa12.04 +1 _i386.deb) all packets are the > same version. This no Debian version schema but from Ubuntu 12.04. I don't know if they work correct on Debian. > What is the correct version for Debian squeeze? Stock Squeeze ships Dovecot in Debian version 1.2.15-7. Squeeze Backports offers version 2.1.7-2~bpo60+1. To install Dovecot 2.1 for Debian Squeeze: % sudo apt-get update % apt-get -s -t squeeze-backports install dovecot-imapd dovecot-pop3d dovecot-mysql The '-s' switch simulates the installation and works without root privileges. If you get no error and apt shows to install version 2.1.7-2~bpo60+1 repeat the command with a preceding 'sudo ' and no '-s'. Dennis From alessio at skye.it Thu Oct 18 17:29:50 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 18 Oct 2012 16:29:50 +0200 Subject: [Dovecot] Add S= to maildirfile Message-ID: <5080125E.5020904@skye.it> Hi, in some old Maildir/ I have file without the S= in file name. Is possibile to add the size to the file name with some tools like doveadm? Are there other methods to update these file? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From rs at sys4.de Thu Oct 18 19:22:39 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 18 Oct 2012 18:22:39 +0200 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <5080125E.5020904@skye.it> References: <5080125E.5020904@skye.it> Message-ID: <50802CCF.3000200@sys4.de> Am 18.10.2012 16:29, schrieb Alessio Cecchi: > Hi, > > in some old Maildir/ I have file without the S= in file name. > > Is possibile to add the size to the file name with some tools like doveadm? > > Are there other methods to update these file? > > Thanks > perhaps this helps for ideas http://wiki2.dovecot.org/HowTo/RefilterMail perhaps you can use dsync also , but i am really not sure if this works http://wiki2.dovecot.org/Tools/Dsync however its easy to test -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From jbates at brightok.net Thu Oct 18 19:49:05 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 18 Oct 2012 11:49:05 -0500 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50802CCF.3000200@sys4.de> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> Message-ID: <50803301.4060508@brightok.net> On 10/18/2012 11:22 AM, Robert Schetterer wrote: > Am 18.10.2012 16:29, schrieb Alessio Cecchi: >> Hi, >> >> in some old Maildir/ I have file without the S= in file name. >> >> Is possibile to add the size to the file name with some tools like doveadm? >> >> Are there other methods to update these file? >> >> Thanks >> > perhaps this helps for ideas > > http://wiki2.dovecot.org/HowTo/RefilterMail > > perhaps you can use dsync also , but i am really not sure > if this works > > http://wiki2.dovecot.org/Tools/Dsync > > however its easy to test Dsync would be the best option, I believe. It should work moving from maildir to maildir, but if necessary, you could also convert it to another format and then put it back to maildir. Jack From nanovox at gmail.com Thu Oct 18 23:32:15 2012 From: nanovox at gmail.com (Steven Kiehl) Date: Thu, 18 Oct 2012 16:32:15 -0400 Subject: [Dovecot] Emails from invalid local accounts Message-ID: Hi, I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues with receiving spam where the "from" header contains an address like accounting at mydomain.com. Is there some way I can filter out these emails coming from outside our network with an account associated with our network which doesn't exist? Do I just need to configure some custom process to evaluate these addresses, or is there some way either in dovecot or spamassassin to do this? Thanks, Steve K From noeldude at gmail.com Fri Oct 19 00:00:21 2012 From: noeldude at gmail.com (Noel) Date: Thu, 18 Oct 2012 16:00:21 -0500 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: Message-ID: <50806DE5.6050904@gmail.com> On 10/18/2012 3:32 PM, Steven Kiehl wrote: > Hi, > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > with receiving spam where the "from" header contains an address like > accounting at mydomain.com. Is there some way I can filter out these emails > coming from outside our network with an account associated with our network > which doesn't exist? Do I just need to configure some custom process to > evaluate these addresses, or is there some way either in dovecot or > spamassassin to do this? > > Thanks, > > Steve K > This should be dealt with in postfix or SpamAssassin, not dovecot, and there are likely other, better ways to detect this particular spam rather than mucking with the From: header. Sometimes mail arrives with a header something like From: accounting and postfix appends @$myorigin to the unqualified address while passing the mail through your content_filter. The fix for that is to set in your postfix main.cf remote_header_rewrite_domain = domain.invalid so that unqualified addresses will be rewritten with a known domain. Don't be tempted to reject such mail outright since you'll reject a significant amount of non-spam mail. Another thing to consider setting in postfix main.cf is: smtpd_reject_unlisted_sender = yes which will reject invalid envelope senders in your domain. (Note the difference between envelope sender and the From: header.) -- Noel Jones From ben at morrow.me.uk Fri Oct 19 00:00:27 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 18 Oct 2012 22:00:27 +0100 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: Message-ID: <20121018210026.GB5388@anubis.morrow.me.uk> At 4PM -0400 on 18/10/12 you (Steven Kiehl) wrote: > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > with receiving spam where the "from" header contains an address like > accounting at mydomain.com. Is there some way I can filter out these emails > coming from outside our network with an account associated with our network > which doesn't exist? Do I just need to configure some custom process to > evaluate these addresses, or is there some way either in dovecot or > spamassassin to do this? You want to do this in Postfix, with either the smtpd_reject_unlisted_sender parameter or the reject_unlisted_sender policy in smtpd_sender_restrictions. You will need to make sure Postfix has access to the list of valid mailboxes at your domain, which it should have already for recipient checking. Ben From stephan at rename-it.nl Fri Oct 19 02:01:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 19 Oct 2012 01:01:43 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <507BBE00.9010007@dada.eu> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> <507BBE00.9010007@dada.eu> Message-ID: <50808A57.8040201@rename-it.nl> On 10/15/2012 9:40 AM, Sandro Tosi wrote: > Hi Stephan, > thanks a lot for your reply. > > On 10/11/2012 10:35 PM, Stephan Bosch wrote: >> On 10/10/2012 11:23 AM, Sandro Tosi wrote: >>> Hello, >>> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >>> as this will be provided with an upcoming Debian package) with MySQL >>> dict lookups with the mail setup we're designing. >>> >>> Our (main) goals are: >>> >>> 1. store the filters on the database >> That is possible with some limitations. > > Are the ones below the only limitatios (ie one script per user) or are > there any other worth knowing? You cannot currently use ManageSieve when the active script is located in a dict database. And 'one script per user' is not an fully accurate description. It is technically possible to access multiple different scripts from the dict database. It is however not possible to use dict support combination with multiscript support ( http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Executing_Multiple_Scripts_Sequentially) to execute multiple scripts in a sequence. Multiscript currently only works for Sieve scripts that are located in the filesystem. > In our situation, what would you suggest? We're now thinking of > keeping the scripts list on a separate table, and merge the "user > selected ones" in a single script to write in the filters table. Is > that what would you suggest? Is there a better solution? You can use the include extension (https://tools.ietf.org/html/draft-ietf-sieve-include-05) to access scripts in a dict database from a main active script to combine them. I believe you could even dynamically construct that main script in SQL using some string manipulation in the query, but that is a bit ugly. Could you send me an overview of your configuration, including your database layout? Provided that I have some time in the next week, I could investigate building a simple working configuration for the sake of example. Regards, Stephan. From calestyo at scientia.net Fri Oct 19 02:32:59 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Fri, 19 Oct 2012 01:32:59 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1350603179.3391.21.camel@fermat.scientia.net> Hi Rick and Robert. Thanks for the tools... I'll have a look over them. :) On Wed, 2012-10-17 at 15:53 +0000, Rick Sanders wrote: > Your best bet for a clean migration is to use an IMAP migration tool (assuming > both of your servers support IMAP). It avoids all of the issues surrounding the > underlying databases used to store the mailboxes and messages since everything > is done through IMAP commands. Well the problem is that a) the mboxes are already mixed up (with respect to different formats), which was basically my fault. b) Evolution is severely broken, amongst others for this https://bugzilla.gnome.org/show_bug.cgi?id=686258 reason. So I cannot really trust that automatic migration will work. > imapsync: http://imapsync.lamiral.info > imap_tools: http://www.athensfbc.com/imap_tools > offlineimap: https://github.com/nicolas33/offlineimap > mbsync: http://isync.sourceforge.net/ > mailsync: http://mailsync.sourceforge.net/ > mailutil: http://www.washington.edu/imap/ part of the UW IMAP tookit. > imaprepl: http://www.bl0rg.net/software/ http://freecode.com/projects/imap-repl/ > imapcopy: http://home.arcor.de/armin.diehl/imapcopy/imapcopy.html > migrationtool: http://sourceforge.net/projects/migrationtool/ > imapmigrate: http://sourceforge.net/projects/cyrus-utils/ > larch: https://github.com/rgrove/larch (derived from wonko_imapsync) > wonko_imapsync: http://wonko.com/article/554 > pop2imap: http://www.linux-france.org/prj/pop2imap/ > exchange-away: http://exchange-away.sourceforge.net/ For most of them, I unfortunately didn't found information on whether they support the different subformats of mbox... what about your MboxtoIMAP.pl ? Right now I tent to create my own converter based on mb2md... just that I don't write out maildir but again mbox. Timo, when you're reading this: I'm not sure though, on which headers I must/should stripe for dovecot? From http://wiki.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata I'd guess that I have to drop all X-IMAPbase, X-IMAP and X-UID. (Will dovcote recreate them, when it indexes the mbox file the first time?) And I have to manually create/calculate, Status, X-Status, X-Keyword (based on what either Evolution or Thunderbird set) and also Content-Length... the "From_" lines in the mails need then to be _not_ quoted. Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Fri Oct 19 02:38:38 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Fri, 19 Oct 2012 01:38:38 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121018123440.GA29330@laptop-dg.leere.eu> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121018123440.GA29330@laptop-dg.leere.eu> Message-ID: <1350603518.3391.23.camel@fermat.scientia.net> On Thu, 2012-10-18 at 14:34 +0200, Dennis Guhl wrote: > [move through Evolution to IMAP] Seriously... I can just suggest anyone to never trust this piece of crap ;) Don't know which daemons led me to using it... > I think, like Rob suggested, you are in need of some serious > scripting. Yeah... guess that's what it will end up with. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From nanovox at gmail.com Fri Oct 19 04:59:40 2012 From: nanovox at gmail.com (Steven Kiehl) Date: Thu, 18 Oct 2012 21:59:40 -0400 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: <50806DE5.6050904@gmail.com> References: <50806DE5.6050904@gmail.com> Message-ID: This is great information on some options I should look into further, however adding the "smtpd_reject_unlisted_sender" option doesn't seem to eliminate the problem. What these spammers are doing is forging the "from" header to be a full address like "accounting at mydomain.com" and they are sending to a real address like "webmaster at mydomain.com". So even if the envelope sender is valid or coming from an outside domain, the visible originating from address is invalid and is in my own domain. And I'm absolutely positive any mail received from these forged from addresses are spam that shouldn't even be delivered. This is also complicated further by the use of virtual domains and virtual alias mapping (all sql based) in the Postfix configuration. Some of my problem may be that Postfix might not be able to get a comprehensive list of valid mailboxes and aliases to deliver to the virtual transport. I've tried to define the virtual mailbox maps, but every time I do that the aliases stop working. On Thu, Oct 18, 2012 at 5:00 PM, Noel wrote: > On 10/18/2012 3:32 PM, Steven Kiehl wrote: > > Hi, > > > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > > with receiving spam where the "from" header contains an address like > > accounting at mydomain.com. Is there some way I can filter out these > emails > > coming from outside our network with an account associated with our > network > > which doesn't exist? Do I just need to configure some custom process to > > evaluate these addresses, or is there some way either in dovecot or > > spamassassin to do this? > > > > Thanks, > > > > Steve K > > > > This should be dealt with in postfix or SpamAssassin, not dovecot, > and there are likely other, better ways to detect this particular > spam rather than mucking with the From: header. > > Sometimes mail arrives with a header something like > From: accounting > and postfix appends @$myorigin to the unqualified address while > passing the mail through your content_filter. > > The fix for that is to set in your postfix main.cf > remote_header_rewrite_domain = domain.invalid > so that unqualified addresses will be rewritten with a known > domain. Don't be tempted to reject such mail outright since you'll > reject a significant amount of non-spam mail. > > Another thing to consider setting in postfix main.cf is: > smtpd_reject_unlisted_sender = yes > which will reject invalid envelope senders in your domain. (Note > the difference between envelope sender and the From: header.) > > > > -- Noel Jones > From noeldude at gmail.com Fri Oct 19 06:50:30 2012 From: noeldude at gmail.com (Noel) Date: Thu, 18 Oct 2012 22:50:30 -0500 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: <50806DE5.6050904@gmail.com> Message-ID: <5080CE06.1080706@gmail.com> On 10/18/2012 8:59 PM, Steven Kiehl wrote: > This is great information on some options I should look into > further, however adding the "smtpd_reject_unlisted_sender" > option doesn't seem to eliminate the problem. [This is OT for the dovecot list, and my last post in this thread. Please send all followups to the appropriate postfix, amavisd-new, or spamassassin list in consideration of other list members. Thank you.] smtpd_reject_unlisted_sender works with the envelope address; this option has no effect on headers. > What these spammers are doing is forging the "from" header to be > a full address like "accounting at mydomain.com > " Possible, but I doubt it. The only way you'll ever see the more likely original "From: accounting" header is by running postfix in debug mode (which is not recommended) or by using a tcp sniffer in front of postfix. That's why I recommend setting "remote_header_rewrite_domain = domain.invalid". Also, this setting requires a non-ancient postfix, but I don't remember which version; if it shows up in "postconf -n" output, you're OK. > and they are sending to a real address like > "webmaster at mydomain.com ". So even > if the envelope sender is valid or coming from an outside domain, > the visible originating from address is invalid and is in my own > domain. And I'm absolutely positive any mail received from these > forged from addresses are spam that shouldn't even be delivered. If there are a few frequently-abused addresses, you can add them to a header_checks rule. But don't get too tied up in wack-a-mole header_checks; that's a great time waster for limited benefit. > This is also complicated further by the use of virtual domains and > virtual alias mapping (all sql based) in the Postfix > configuration. Some of my problem may be that Postfix might not > be able to get a comprehensive list of valid mailboxes and aliases > to deliver to the virtual transport. I've tried to define the > virtual mailbox maps, but every time I do that the aliases stop > working. If your postfix is not able to properly validate recipients, you should ask about that on the postfix list. That is a serious problem. http://www.postfix.org/DEBUG_README.html#mail The point you're missing is that there is no way to validate the From: header. Look at other features of the unwanted mail for ways to reject it. -- Noel Jones From tomislav.mihalicek at gmail.com Fri Oct 19 10:40:50 2012 From: tomislav.mihalicek at gmail.com (tmihalicek) Date: Fri, 19 Oct 2012 00:40:50 -0700 (PDT) Subject: [Dovecot] Dovecot quota postgres dictionary problems Message-ID: <1350632450161-38234.post@n4.nabble.com> I have a strange errors in .err log file, but the postgres seem to be filling with quota changes, i will also put configs in Oct 19 09:23:52 mailstore-node-01 dovecot: imap(test at example.net): Error: read(/var/run/dovecot/dict) failed: Timeout after 30 seconds Oct 19 09:24:22 mailstore-node-01 dovecot: imap(test at example.net): Error: read(/var/run/dovecot/dict) failed: Timeout after 30 seconds Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Panic: file dict-client.c: line 270 (client_dict_finish_transaction): assertion failed: (dict->async_commits > 0) Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x484ea) [0x7fbed405d4ea] -> /usr/lib/dovecot/libdovecot.so.0(+0x48536) [0x7fbed405d536] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fbed4030eaf] -> /usr/lib/dovecot/libdovecot.so.0(+0x22337) [0x7fbed4037337] -> /usr/lib/dovecot/libdovecot.so.0(+0x2236b) [0x7fbed403736b] -> /usr/lib/dovecot/libdovecot.so.0(+0x22e78) [0x7fbed4037e78] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0x8a3f) [0x7fbed2c76a3f] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7fbed2c73262] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(quota_transaction_commit+0x1e7) [0x7fbed2c738d7] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xb39f) [0x7fbed2c7939f] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xb4f4) [0x7fbed2c794f4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x2a) [0x7fbed432396a] -> dovecot/imap [test at example.net 10.84.34.2 expunge](imap_sync_deinit+0x4d) [0x418edd] -> dovecot/imap [test at example.net 10.84.34.2 expunge]() [0x41918c] -> dovecot/imap [test at example.net 10.84.34.2 expunge](cmd_sync_delayed+0x1f5) [0x4195b5] -> dovecot/imap [test at example.net 10.84.34.2 expunge](client_handle_input+0x1fd) [0x41127d] -> dovecot/imap [test at example.net 10.84.34.2 expunge](client_input+0x5f) [0x411adf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7fbed40696c6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7fbed406a6ff] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7fbed4069668] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fbed4055043] -> dovecot/imap [test at example.net 10.84.34.2 expunge](main+0x2a4) [0x419d24] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fbed3cd1c8d] -> dovecot/imap [test at example.net 10.84.34.2 expunge]() [0x409059] doveconf.txt dovecot-dict-sql.conf.ext -- View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-quota-postgres-dictionary-problems-tp38234.html Sent from the Dovecot mailing list archive at Nabble.com. From amateo at um.es Fri Oct 19 15:38:36 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 19 Oct 2012 14:38:36 +0200 Subject: [Dovecot] Auth caching and password changes Message-ID: <508149CC.9070004@um.es> Hello, In my system I have configured auth caching. The problem I have is that whenever a user changes his password, he/she can't login to dovecot after a while and the scenarios described at http://wiki2.dovecot.org/Authentication/Caching are not applied. I have tried also with "doveadm auth cache flush ", but it didn't work. He also could to login again if he waits for a time or if I run "doveadm auth cache flush" in the server, flushing all auth information from cache. I have attached the log I had when I changed my password (and suffered the problem). I have attached my doveconf -n too. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 -------------- next part -------------- A non-text attachment was scrubbed... Name: cambioclave.log Type: text/x-log Size: 1349 bytes Desc: not available URL: -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota zlib mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 3075 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From alessio at skye.it Fri Oct 19 19:12:26 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 19 Oct 2012 18:12:26 +0200 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50803301.4060508@brightok.net> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> <50803301.4060508@brightok.net> Message-ID: <50817BEA.6090201@skye.it> Il 18/10/2012 18:49, Jack Bates ha scritto: > On 10/18/2012 11:22 AM, Robert Schetterer wrote: >> Am 18.10.2012 16:29, schrieb Alessio Cecchi: >>> Hi, >>> >>> in some old Maildir/ I have file without the S= in file name. >>> >>> Is possibile to add the size to the file name with some tools like >>> doveadm? >>> >>> Are there other methods to update these file? >>> >>> Thanks >>> >> perhaps this helps for ideas >> >> http://wiki2.dovecot.org/HowTo/RefilterMail >> >> perhaps you can use dsync also , but i am really not sure >> if this works >> >> http://wiki2.dovecot.org/Tools/Dsync >> >> however its easy to test > > Dsync would be the best option, I believe. It should work moving from > maildir to maildir, but if necessary, you could also convert it to > another format and then put it back to maildir. > > Jack > Dsync could be a good idea but I need to "sync" maildir to the same maildir and I don't think can be done this. My situation is a Maildir/cur/ with old email files without S=size and, in the same Maildir/cur/, some files with S=size. Can doveadm do this? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From wamp at promax.media.pl Fri Oct 19 15:32:20 2012 From: wamp at promax.media.pl (wamp) Date: Fri, 19 Oct 2012 05:32:20 -0700 (PDT) Subject: [Dovecot] Dovecot 2 quota limit and actual size (mysql) Message-ID: <1350649940026-38235.post@n4.nabble.com> Hello, Can You explain to me how dovecot-lda knows actual size of virtual user directory? I want to keep max size of user directory in mysql - should I also use some kind of script to upgrade actual size information in mysql ? I read docs from wiki but still dont know it. thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-2-quota-limit-and-actual-size-mysql-tp38235.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Fri Oct 19 19:17:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 19 Oct 2012 19:17:04 +0300 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50817BEA.6090201@skye.it> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> <50803301.4060508@brightok.net> <50817BEA.6090201@skye.it> Message-ID: <9BE333EF-2120-4581-9A51-79C08EAF9085@iki.fi> On 19.10.2012, at 19.12, Alessio Cecchi wrote: >>>> in some old Maildir/ I have file without the S= in file name. >>>> >>>> Is possibile to add the size to the file name with some tools like doveadm? Not directly. >>>> Are there other methods to update these file? A script that renames the files and updates dovecot-uidlist. No such script exists as far as I know. You could also switch from Maildir++ quota to dict-file quota and this wouldn't be a problem. > Dsync could be a good idea but I need to "sync" maildir to the same maildir and I don't think can be done this. You could sync to another maildir, rm -rf the original, sync back to original. > My situation is a Maildir/cur/ with old email files without S=size and, in the same Maildir/cur/, some files with S=size. > > Can doveadm do this? No. From tobias at maffert.net Fri Oct 19 21:47:20 2012 From: tobias at maffert.net (Tobias Maffert) Date: Fri, 19 Oct 2012 20:47:20 +0200 Subject: [Dovecot] Question about salted hashes Message-ID: <5081A038.8070908@maffert.net> Hello. I'm switching from b1gmail to my own setup which consists of Postfix+Dovecot+MySQL (and maybe VBoxAdm). There are two problems: - b1gmail is using unsalted MD5 hashes. Is there any good way to make my new setup backward compatible? So I don't have to force all of my 50k users to change their password. - How do I change my setup to salted SHA256 (or an even better algorithm). And how do I make the hashes compatible between Postfix, Dovecot, MySQL (and maybe VBoxAdm)? My system: Debian Squeeze Postfix version: 2.7.1 Dovecot version: 1.2.15 ----------------------------------------------- dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.6 protocols: imap pop3 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_access_groups: vmail mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): UID%u-%v lda: postmaster_address: auth_socket_path: /var/run/dovecot/auth-master mail_plugins: quota sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain login user: vmail passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir ----------------------------------------------- driver = mysql connect = host=127.0.0.1 dbname=smail user=smail password=mypw default_pass_scheme = CRYPT password_query = SELECT username AS user, password_enc AS password, CONCAT(homedir, maildir) AS userdb_home, uid AS userdb_uid, gid AS userdb_gid, CONCAT('maildir:', homedir, maildir) AS userdb_mail, CONCAT('maildir:storage=', (quota*1024)) as userdb_quota FROM mail_users WHERE (username = '%u' OR email = '%u') AND ((imap = 1 AND '%Ls' = 'imap') OR (pop3 = 1 AND '%Ls' = 'pop3') OR '%Ls' = 'smtp') user_query = SELECT CONCAT(homedir, maildir) AS home, CONCAT('maildir:', homedir, maildir) AS mail, uid, gid, CONCAT('*:storage=', (quota*1024)) as quota_rule FROM mail_users WHERE (username = '%u' OR email = '%u') ----------------------------------------------- Regards, Tobi M. From h.reindl at thelounge.net Fri Oct 19 21:50:28 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 19 Oct 2012 20:50:28 +0200 Subject: [Dovecot] Question about salted hashes In-Reply-To: <5081A038.8070908@maffert.net> References: <5081A038.8070908@maffert.net> Message-ID: <5081A0F4.9090704@thelounge.net> Am 19.10.2012 20:47, schrieb Tobias Maffert: > Hello. > > I'm switching from b1gmail to my own setup which consists of > Postfix+Dovecot+MySQL (and maybe VBoxAdm). There are two problems: - > b1gmail is using unsalted MD5 hashes. Is there any good way to make my > new setup backward compatible? So I don't have to force all of my 50k > users to change their password. - How do I change my setup to salted > SHA256 (or an even better algorithm). And how do I make the hashes > compatible between Postfix, Dovecot, MySQL (and maybe VBoxAdm)? > > My system: > > Debian Squeeze > Postfix version: 2.7.1 > Dovecot version: 1.2.15 besides your question you REALLY do not want to START with 1.2.15 while dovecot-2.1.10 is the recent version postfix is somehow OK but recent is postfix-2.9.4 these outdated versions usually results in many questions on mailing-lists because several howtos do not work or bugs fixed since years are still there -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From dave.mehler at gmail.com Fri Oct 19 23:11:07 2012 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 19 Oct 2012 16:11:07 -0400 Subject: [Dovecot] still having difficulties with per-user quotas Message-ID: Hello, I am trying to get per-user quotas working. My thanks to all who have helped so far. To recap I am running Dovecot 2.1 and Mysql where I've got my virtual users. All virtual users are under the system user vmail with a UID and GID of 5000. Looking over the wiki docs I've added a quota table and got the dict service working, I am not having problems with permissions or the login username and password, all that is working fine. Here's my current doveconf -n output it is producing the following debug error related to the userdb sql query: # 2.1.10: /etc/dovecot/dovecot.conf # XXX dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 hostname = XXX last_valid_gid = 5000 last_valid_uid = 5000 mail_location = maildir:/home/vmail/%d/%n:LAYOUT=fs mail_plugins = " quota" namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = xxx xxxx ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_cert = , method=PLAIN, rip=::1, lip=::1, mpid=29282, secured, session= Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User is missing UID (see mail_uid setting) Oct 19 15:23:52 imap(xxx): Error: Internal error occurred. Refer to server log for more information. I am wanting a majority of my users to have the global 1GB quota, but the users in the quota table to have given quotas. Here's what the virtual_users and quota tables look like: mysql> describe virtual_users; +-----------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | domain_id | int(11) | NO | MUL | NULL | | | user | varchar(40) | NO | | NULL | | | password | varchar(128) | NO | | NULL | | +-----------+--------------+------+-----+---------+----------------+ 4 rows in set (0.00 sec) mysql> describe quota; +----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ | username | varchar(100) | NO | PRI | NULL | | | bytes | bigint(20) | NO | | 0 | | | messages | int(11) | NO | | 0 | | +----------+--------------+------+-----+---------+-------+ 3 rows in set (0.00 sec) I'd appreciate any help. Thanks. Dave. From emailbuilder88 at yahoo.com Fri Oct 19 23:43:29 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Fri, 19 Oct 2012 13:43:29 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? Message-ID: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Hello, I'm having some problems getting LDA to work without userdb lookups and have a few related questions. This system has all users in MySQL, each user with unique UID/GID, no local users at all.? Installation is from apt-get. 1) If LDA is invoked without lookups, is it correct to assume that the "service auth" and "service auth-worker" can be completely removed from dovecot master configuration? (I have tried commenting them out and logging into IMAP, which seems to work, not sure if anyone else needs the auth service) 2) If LDA is invoked without lookups, will I be unable to use Dovecot quota plugin? Does it need to have a user lookup to get quota info? (haven't added quota support, need to take this one step at a time) 3) The interesting part -- I am invoking LDA from Maildrop. See: http://thread.gmane.org/gmane.mail.imap.dovecot/65473 So when invoked, Maildrop has already dropped to the destination UID/GID and the needed paths are available in the environment.? However, using as many permutations of calling LDA as I can think of (based on ??? http://wiki2.dovecot.org/LDA ), I always get this: (command line usage error. Command output: lda: Fatal: Couldn't lookup our username (uid=2500) ) The UID is correct for the target user. If I add "-d $LOGNAME" to my LDA callout, I get permission denied on the userdb lookup, which I guess is another issue to work out if I want to go with lookups. But right now I am trying not to. Why does LDA seem to try for a lookup even when I follow the wiki instructions how to call it without a lookup? 3.5) Related question, my users have separate homedir and maildir, both paths are looked up by Maildrop. I think I need to call LDA with "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 14:45:20 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 13:45:20 +0200 Subject: [Dovecot] Dovecot 2 quota limit and actual size (mysql) In-Reply-To: <1350649940026-38235.post@n4.nabble.com> References: <1350649940026-38235.post@n4.nabble.com> Message-ID: <20121020114520.GA26196@daniel.localdomain> Use LMTP instead of lda. The dovecot lmtp service automatically cares about updating quota values in mysql database when mail arrives through the lmtp socket. Regards Daniel wamp wrote: > Hello, > Can You explain to me how dovecot-lda knows actual size of virtual user > directory? I want to keep > max size of user directory in mysql - should I also use some kind of script > to upgrade actual size information in mysql ? > > I read docs from wiki but still dont know it. > > > thanks > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-2-quota-limit-and-actual-size-mysql-tp38235.html > Sent from the Dovecot mailing list archive at Nabble.com. > -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 15:47:49 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 14:47:49 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: Message-ID: <20121020124749.GA26942@daniel.localdomain> David Mehler wrote: > Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User is missing UID (see mail_uid setting) Set the following options in your dovecot.conf: mail_uid = vmail mail_gid = vmail Also see section "Mail users" at http://wiki2.dovecot.org/UserIds Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 16:51:44 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 15:51:44 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: References: Message-ID: <20121020135144.GA28609@daniel.localdomain> Jan-Frode Myklebust wrote: > $ cat /etc/dovecot/dovecot-trash.conf.ext > # Spam mailbox is emptied before Trash > 1 INBOX.Spam > # Trash mailbox is emptied before Sent > 2 INBOX.Trash Are you sure the Trash Folder of the affected users is located below "INBOX"? doveadm mailbox list -u user at domain | grep -iE "trash|spam" Example at http://wiki2.dovecot.org/Plugins/Trash omits "INBOX." Have you tried INBOX/Trash as mailbox name? Regards Daniel -- https://plus.google.com/103021802792276734820 From sven at svenhartge.de Sat Oct 20 19:39:22 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sat, 20 Oct 2012 18:39:22 +0200 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive Message-ID: <09718hdveev8@mids.svenhartge.de> Hi! I am about to migrate a perdition-based IMAP/POP3 proxy to Dovecot. Unfortunately some users are behind a firewall/NAT setup which throws away seemingly idle TCP connections sooner than the established default of 24 hours (more likely after 30 minutes ...) resulting in all kinds of weird client behavior. And unfortunately? this firewall/NAT setup is outside of my control and I have no means of correcting this (in my opinion) flawed configuration. Now, with perdition I was able to use the --tcp_keepalive option which totally solved the mentioned weird client behavior. My question is: does Dovecot2 use TCP-Keepalive on its sockets per default or do I need to enable it some way I have not yet discovered? The manual and wiki only talk about "keepalive" in connection with the IMAP protocol and IDLE and my C-fu is too weak to understand the source code. Gr??e, Sven. -- Sigmentation fault. Core dumped. From tss at iki.fi Sat Oct 20 20:02:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 20 Oct 2012 20:02:36 +0300 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive In-Reply-To: <09718hdveev8@mids.svenhartge.de> References: <09718hdveev8@mids.svenhartge.de> Message-ID: <9881D0A3-70EB-454B-A71D-9C7A253AE21E@iki.fi> On 20.10.2012, at 19.39, Sven Hartge wrote: > I am about to migrate a perdition-based IMAP/POP3 proxy to Dovecot. > > Unfortunately some users are behind a firewall/NAT setup which throws > away seemingly idle TCP connections sooner than the established default > of 24 hours (more likely after 30 minutes ...) resulting in all kinds of > weird client behavior. > > And unfortunately? this firewall/NAT setup is outside of my control and > I have no means of correcting this (in my opinion) flawed configuration. > > Now, with perdition I was able to use the --tcp_keepalive option which > totally solved the mentioned weird client behavior. > > My question is: does Dovecot2 use TCP-Keepalive on its sockets per > default or do I need to enable it some way I have not yet discovered? It's the default yes. Of course Linux's default keepalive interval is something like 90 minutes, so have you changed that already?.. > The manual and wiki only talk about "keepalive" in connection with the > IMAP protocol and IDLE and my C-fu is too weak to understand the source > code. imap_idle_notify_interval (default 2 min) causes Dovecot to send data to IDLEing connections, which pretty much makes the TCP keepalive irrelevant. For non-IDLE connections Dovecot has a disconnect timeout of 30 minutes. From dave.mehler at gmail.com Sat Oct 20 20:06:59 2012 From: dave.mehler at gmail.com (David Mehler) Date: Sat, 20 Oct 2012 13:06:59 -0400 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: <20121020124749.GA26942@daniel.localdomain> References: <20121020124749.GA26942@daniel.localdomain> Message-ID: Hello, Thank you for your reply. Adding mail_uid and mail_gid fixed it. I now have quotas going but I don't know if I have them right or just don't like my setup. My first issue is from what it is looking like I have to define all my users in the quota database not just the ones whose values I want to override the global quota declaration in 90-quota.conf. If I just add the user at domain to the database the bytes and messages columns have zero as default, this means those values override global quota in 90-quota.conf and they effectively have unlimited access. My second issue is I have entered a quota of 250 megabytes for a test user. This works but he seems to get more space everytime he logs in, started out at 250, on the next login it was 255, then 269 on the third, and so forth. I've checked the quota table and yes the value in the bytes column is increasing. Thanks for any help. Dave. On 10/20/12, Daniel Parthey wrote: > David Mehler wrote: >> Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User >> is missing UID (see mail_uid setting) > > Set the following options in your dovecot.conf: > > mail_uid = vmail > mail_gid = vmail > > Also see section "Mail users" at > http://wiki2.dovecot.org/UserIds > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From sven at svenhartge.de Sat Oct 20 20:15:25 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sat, 20 Oct 2012 19:15:25 +0200 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive References: <09718hdveev8@mids.svenhartge.de> <9881D0A3-70EB-454B-A71D-9C7A253AE21E@iki.fi> Message-ID: <1971afdveev8@mids.svenhartge.de> Timo Sirainen wrote: > On 20.10.2012, at 19.39, Sven Hartge wrote: >> My question is: does Dovecot2 use TCP-Keepalive on its sockets per >> default or do I need to enable it some way I have not yet discovered? > It's the default yes. Of course Linux's default keepalive interval is > something like 90 minutes, so have you changed that already?.. Yes, I did. For those systems it is set to 15 minutes right now. >> The manual and wiki only talk about "keepalive" in connection with >> the IMAP protocol and IDLE and my C-fu is too weak to understand the >> source code. > imap_idle_notify_interval (default 2 min) causes Dovecot to send data > to IDLEing connections, which pretty much makes the TCP keepalive > irrelevant. For non-IDLE connections Dovecot has a disconnect timeout > of 30 minutes. This is fine. As long as the client notices the termination of the connection, everything should be OK. Before I switched keepalive on for Perdition, the firewall/NAT would internally throw away a connection but neither the client or the server would notice this. Then if the client tried to do something with this connection, like select or save a message, the firewall/NAT would send a RST and the client would then bug the user with a meaningless message like "folder does not exist" which caused a lot of confusion for the end-user and created quite the bit of trouble tickets. This problem mostly happend with an IMAP connection to the "Sent Messages" folder which normally does not see much changes until the users writes and sends a mail. Then after the mail was sent via SMTP the client tries to save the message, gets sent an RST from the firewall/NAT and presents the user with a wrong and confusing error message. The user then thinks his mail was not sent and sends it again. This time the client opens a new connection to select the "Sent Messages" folder and everything works. But the recipient gets the mail twice. Again resulting in confusion and trouble tickets to be dealt with. By switching to TCP keepalive (and reducing the keepalive time to 15 minutes) all those problems were solved and my users (and support staff) were happy again ;) Gr??e, Sven. -- Sigmentation fault. Core dumped. From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 21:51:24 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 20:51:24 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> Message-ID: <20121020185124.GA2609@daniel.localdomain> David Mehler wrote: > My first issue is from what it is looking like I have to define all my > users in the quota database not just the ones whose values I want to > override the global quota declaration in 90-quota.conf. If I just add > the user at domain to the database the bytes and messages columns have > zero as default, this means those values override global quota in > 90-quota.conf and they effectively have unlimited access. This is expected behavior. If the userdb returns a quota rule, it overrides the global quota rule. Extend your SQL query to return a default quota_rule for rows without quota entry. > My second issue is I have entered a quota of 250 megabytes for a test > user. This works but he seems to get more space everytime he logs in, > started out at 250, on the next login it was 255, then 269 on the > third, and so forth. I've checked the quota table and yes the value in > the bytes column is increasing. Please show output of doveconf -n and any external (sql/dict) includes related to quota or quota_rules. Regards Daniel -- https://plus.google.com/103021802792276734820 From emailbuilder88 at yahoo.com Sat Oct 20 22:17:12 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Sat, 20 Oct 2012 12:17:12 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Message-ID: <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> > 1) If LDA is invoked without > lookups, is it correct to assume that the "service auth" and > "service > auth-worker" can be completely removed from dovecot master > configuration? (I have tried commenting them out and logging into IMAP, > which seems to work, not sure if anyone else needs the auth service) Any confirmation on this? > 2) > If LDA is invoked without lookups, will I be unable to use Dovecot > quota plugin? Does it need to have a user lookup to get quota info? > (haven't added quota support, need to take this one step at a time) I'm especially interested if someone can comment on this, since maybe it makes my efforts here wasted > 3) The interesting part -- I am invoking LDA from Maildrop. See: > http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So > when invoked, Maildrop has already dropped to the destination UID/GID > and the needed paths are available in the environment.? However, using > as many permutations of calling LDA as I can think of (based on ??? > http://wiki2.dovecot.org/LDA ), I always get this: > > (command line usage error. Command output: lda: Fatal: Couldn't lookup our > username (uid=2500) ) I could not find anything in the mailing list archives to help me, but I googled and found a link to a source file: http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 I read the source, it looks like it's not exactly a userdb lookup - LDA is trying to get the unix username for the given UID. In my case, UIDs are "virtual" so there isn't a unix username. The source doesn't really use the username that it looks up except in a call "open_logfile." Is it possible to avoid this problem? It looks like the answer is no, I have to use -d which also forces a userdb lookup. Maybe this limitation can be removed in the future? Now I suppose I have to go understand the problems of userdb lookup permissions, but I think there are solutions for that. Am I on the right understanding?? ? > The > UID is correct for the target user. If I add "-d $LOGNAME" to my LDA > callout, I get permission denied on the userdb lookup, which I guess is > another issue to work out if I want to go with lookups. But right now I > am trying not to. Why does LDA seem to try for a lookup even when I > follow the wiki instructions how to call it without a lookup? > > 3.5) > Related question, my users have separate homedir and maildir, both > paths are looked up by Maildrop. I think I need to call LDA with > "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? > From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 23:04:32 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 22:04:32 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> <20121020185124.GA2609@daniel.localdomain> Message-ID: <20121020200432.GA3161@daniel.localdomain> David Mehler wrote: > Thanks for your reply. So with the extending of the query to return a > default quota rule, do you have an example of that by the way, does > that mean I only have to put the overrided users in the quota table? Assuming that quota values are in the dovecot_users table... # passdb with userdb prefetch and default quota of 1024M for quota=0 rows # The userdb_ prefix is for prefetch userdb entries in password_query password_query = SELECT username AS user, \ password AS password, \ home AS userdb_home, \ uid AS userdb_uid, \ gid AS userdb_gid, \ CASE quota \ WHEN 0 \ THEN '*:bytes=1024M:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota AS CHAR), 'M:messages=', CAST(quota_message AS CHAR)) \ END AS `userdb_quota_rule` \ FROM dovecot_users \ WHERE username='%u'; # user_query with default quota of 1024M for quota=0 rows user_query = SELECT username AS user, \ home AS home, \ uid AS uid, \ gid as gid, \ CASE quota \ WHEN 0 \ THEN '*:bytes=1024M:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota AS CHAR), 'M:messages=', CAST(quota_message AS CHAR)) \ END AS `quota_rule` \ FROM dovecot_users \ WHERE username='%u'; Your user_query needs to return a row if the user exists, otherwise dovecot will assume that the user does not exist and the mail or user will be rejected. Regards Daniel -- https://plus.google.com/103021802792276734820 From emailbuilder88 at yahoo.com Sat Oct 20 23:34:12 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Sat, 20 Oct 2012 13:34:12 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> Message-ID: <1350765252.74118.YahooMailNeo@web39301.mail.mud.yahoo.com> >> 3) The interesting part -- I am invoking LDA from Maildrop. See: >> http://thread.gmane.org/gmane.mail.imap.dovecot/65473 >> So >> when invoked, Maildrop has already dropped to the destination UID/GID >> and the needed paths are available in the environment.? However, using >> as many permutations of calling LDA as I can think of (based on ??? >> http://wiki2.dovecot.org/LDA ), I always get this: >> >> (command line usage error. Command output: lda: Fatal: Couldn't lookup > our >> username (uid=2500) ) > > I could not find anything in the mailing list archives to help me, but I googled > and found a link to a source file: > > http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 > > I read the source, it looks like it's not exactly a userdb lookup - LDA is > trying to get the unix username for the given UID. In my case, UIDs are > "virtual" so there isn't a unix username. The source doesn't > really use the username that it looks up except in a call > "open_logfile." > > Is it possible to avoid this problem? It looks like the answer is no, I have to > use -d which also forces a userdb lookup. Maybe this limitation can be removed > in the future? Now I suppose I have to go understand the problems of userdb > lookup permissions, but I think there are solutions for that. FWIW, in this scenario, "service auth" in master config has to have its mode relaxed to 0606 to make userdb lookups work.? So ANYONE on the machine can see all userdb lookups.? I don't have local users here, so it's probably safe anyway(?). Can anyone explain if there are other security risks of running the auth service at 0606? From jeff at bubble.org Sun Oct 21 04:52:01 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Sat, 20 Oct 2012 21:52:01 -0400 Subject: [Dovecot] Configuring Dovecot & Snarf plugin for the first time Message-ID: <50835541.8000808@bubble.org> I've been using uw-imap for some time on my linux system and have been running into issues with it so I've decided to move to Dovecote, so far it seems to have solved the issues I've been having however I need/want to move the incoming emails out of /var/spool/mail/{user} in the same (or similar fashion) that uw-imap did, and I found the snarf plugin. However whenever I enable the snarf plugin using the example on the wiki page my email is not loaded and when I remove my configuration for snarf my email re-appears. Based upon what I can tell the snarf plugin is either not loading (but I see it listed in the logs) or simply not working (which is probably because its not configured properly). The system is Fedora 16 (x86_64), the MTA is Exim, and Dovecot is 2.0.21 (version supplied with Fedora). I know I'm making a newbie mistake. Any guidance would be appreciated. Thanks, Jeff dovecot -n provides the following: [root at xyzzy conf.d]# more /tmp/dovecot.changes # 2.0.21: /etc/dovecot/dovecot.conf # OS: Linux 3.4.11-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mbox mail_plugins = " zlib" mbox_write_locks = fcntl namespace default { inbox = yes location = prefix = separator = / } namespace snarf { hidden = yes list = no location = mbox:/run/dovecot/empty:INBOX=/var/spool/mail/%u prefix = /snarf separator = / } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename snarf = = /snarf/INBOX } service imap-login { inet_listener imap { address = localhost } } service pop3-login { inet_listener pop3 { address = localhost } } ssl_cert = , method=PLAIN, rip=98.109.156.118, lip=132.238.254.34, mpid=19627, TLS Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= Oct 20 21:29:46 xyzzy dovecot: imap-login: Login: user=, method=PLAIN, rip=98.109.156.118, lip=132.238.254.34, mpid=19629, TLS Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= From nicolas at devels.es Sun Oct 21 14:15:37 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 12:15:37 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log Message-ID: <5083D959.5090407@devels.es> Hi list! I'm using Dovecot along with Postfix (with MySQL) and I'm having some curious error messages in the mail log. Everything runs normally and suddenly I start viewing messages like these: Oct 12 14:24:23 dovecot: last message repeated 5 times Oct 12 14:25:23 dovecot: last message repeated 6 times Oct 12 14:26:23 dovecot: last message repeated 6 times Oct 12 14:27:23 dovecot: last message repeated 6 times Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory Oct 12 14:30:23 dovecot: last message repeated 5 times Oct 12 14:31:23 dovecot: last message repeated 6 times Oct 12 14:33:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory Oct 12 14:34:23 dovecot: last message repeated 5 times Oct 12 14:35:23 dovecot: last message repeated 6 times Oct 12 14:36:01 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory This file which Dovecot references to indeed exists, is accesible, etc. Once I get the first of these messages, the user stops receiving e-mails (the mail server keeps receiving them normally, though) until he restarts the mail client. Then the error msg is gone and he receives all their unreceived messages. This error appears in 6-12h. intervals once the user starts their client. Version is 2.0.19, and dovecot -n is: root at mail:~# dovecot -n # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-32-generic-pae i686 Ubuntu 12.04.1 LTS ext4 disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir mailbox_idle_check_interval = 2 mins namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = root } ssl_cert = Hi everyone, short version: Is there no built in failover mechanism for the director service to handle a backend failure? Long version: I have a frontend server running the director service and two backends. Due to maintenance I had to shut down one of the backends which caused connection errors for the users being directed to this backend. I was very surprised as I expected the director to redirect these users to the remaining backend. Am I wrong or is the director not working as expected? Regards Patrick # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain login director_mail_servers = 172.17.1.1 172.17.1.2 director_servers = 172.17.1.3 172.17.1.4 lmtp_proxy = yes log_path = /var/log/dovecot.log managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 172.17.1.3 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 172.17.1.3 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = References: <5083D959.5090407@devels.es> Message-ID: On 21.10.2012, at 14.15, Nicol?s wrote: > Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? From nicolas at devels.es Sun Oct 21 14:29:39 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 12:29:39 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: References: <5083D959.5090407@devels.es> Message-ID: <5083DCA3.8000808@devels.es> El 21/10/2012 12:26, Timo Sirainen escribi?: > On 21.10.2012, at 14.15, Nicol?s wrote: > >> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory > fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? > No, on that machine I'm just using local filesystem, no NFS. From tss at iki.fi Sun Oct 21 16:58:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 21 Oct 2012 16:58:36 +0300 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <5083DCA3.8000808@devels.es> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> Message-ID: <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> On 21.10.2012, at 14.29, Nicol?s wrote: > El 21/10/2012 12:26, Timo Sirainen escribi?: >> On 21.10.2012, at 14.15, Nicol?s wrote: >> >>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >> > > No, on that machine I'm just using local filesystem, no NFS. Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. From janfrode at tanso.net Sun Oct 21 19:45:18 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 21 Oct 2012 18:45:18 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: <20121020135144.GA28609@daniel.localdomain> References: <20121020135144.GA28609@daniel.localdomain> Message-ID: On Sat, Oct 20, 2012 at 3:51 PM, Daniel Parthey wrote: > Jan-Frode Myklebust wrote: >> $ cat /etc/dovecot/dovecot-trash.conf.ext >> # Spam mailbox is emptied before Trash >> 1 INBOX.Spam >> # Trash mailbox is emptied before Sent >> 2 INBOX.Trash > > Are you sure the Trash Folder of the affected users is located below "INBOX"? > doveadm mailbox list -u user at domain | grep -iE "trash|spam" $ sudo doveadm mailbox list -u XXXXX at example.no INBOX INBOX.Drafts INBOX.Sent INBOX.Spam INBOX.Trash > Example at http://wiki2.dovecot.org/Plugins/Trash omits "INBOX." > Have you tried INBOX/Trash as mailbox name? No, should I, when my prefix is "INBOX." and separator is "." ? namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } BTW: I think it's mostly working.. as the number or quota exceeded messages has clearly dropped since implementing it, but I do find a few users that get quota exceeded and has lots of messages in INBOX.Trash og INBOX.Spam.. -jf From nicolas at devels.es Sun Oct 21 20:58:46 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 18:58:46 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> Message-ID: <508437D6.8020305@devels.es> El 21/10/2012 14:58, Timo Sirainen escribi?: > On 21.10.2012, at 14.29, Nicol?s wrote: > >> El 21/10/2012 12:26, Timo Sirainen escribi?: >>> On 21.10.2012, at 14.15, Nicol?s wrote: >>> >>>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >>> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >>> >> No, on that machine I'm just using local filesystem, no NFS. > Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. > Well, at least it's good to know it's not a misconfiguration :-) If there's any additional info I can provide feel free to ask. Regards, Nicol?s From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 21 22:43:20 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 21 Oct 2012 21:43:20 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <5083D963.3000700@wk-serv.de> References: <5083D963.3000700@wk-serv.de> Message-ID: <20121021194320.GA7977@daniel.localdomain> Patrick Westenberg wrote: > Is there no built in failover mechanism for the director service to > handle a backend failure? No, the director's job is to keep a hash table and direct the connection for each user to its associated backend. Currently, there is no built-in backend monitoring. In order to handle maintenance of backends, you will need the poolmon daemon, which enables/disables backends in the director depending on their availability: https://github.com/brandond/poolmon Regards Daniel -- https://plus.google.com/103021802792276734820 From tss at iki.fi Sun Oct 21 22:45:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 21 Oct 2012 22:45:46 +0300 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <508437D6.8020305@devels.es> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> <508437D6.8020305@devels.es> Message-ID: On 21.10.2012, at 20.58, Nicol?s wrote: > El 21/10/2012 14:58, Timo Sirainen escribi?: >> On 21.10.2012, at 14.29, Nicol?s wrote: >> >>> El 21/10/2012 12:26, Timo Sirainen escribi?: >>>> On 21.10.2012, at 14.15, Nicol?s wrote: >>>> >>>>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >>>> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >>>> >>> No, on that machine I'm just using local filesystem, no NFS. >> Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. >> > > Well, at least it's good to know it's not a misconfiguration :-) If there's any additional info I can provide feel free to ask. Any way you can reproduce this somewhat easily? For example with http://imapwiki.org/ImapTest ? From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 21 23:49:19 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 21 Oct 2012 22:49:19 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> Message-ID: <20121021204919.GA9191@daniel.localdomain> Hi Dave, David Mehler wrote: > Thanks for all your help so far. I have per-user quotas now working, I > had to also alter my dict config file as well. I am having two > outstanding issues, actually one outstanding issue and one question. > > Here's the question, given that the userdb sql query returns a default > quota entry for rows of zero in quota and quota_messages is the > default quota section needed in 90-quota.conf? > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } You need quota_rule2 to give the user some additional space in the Trash folder if he/she wants to delete messages when over quota. The dict is also needed for quota lookup from the database. The only thing which might be omitted is the global quota_rule since it is returned by the userdb/passwd in any case, but I'm not sure what happens if you only configure a "quota_rule2" without configuring a "quota_rule". > My outstanding issue is whenever I as the root mysql user update a > user's quota the other user also gets an update, I noticed with one > the messages column on the other user went from 0 to 2, another time > the quota value went up from 0 to 3500 it seems random. You should not be accounting the actual mailbox usage in the same virtual_users table as the quota is read from. Use *different* column or table name in your dict file where dovecot may write the current storage/message count. > dovecot-dict-sql.conf.ext > > map { > pattern = priv/quota/storage > table = virtual_users > username_field = user > value_field = quota value_field should be current_quota_storage (writable column) > } > map { > pattern = priv/quota/messages > table = virtual_users > username_field = user > value_field = quota_messages value_field should be current_quota_messages (writable column) > I'd like to know why these columns are updating. Dovecot stores the current storage and mailcount in there. These columns should be different from the columns defining the maximum limit. Regards Daniel From daniel.parthey at informatik.tu-chemnitz.de Mon Oct 22 01:22:07 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 22 Oct 2012 00:22:07 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> <20121021204919.GA9191@daniel.localdomain> Message-ID: <20121021222207.GA10903@daniel.localdomain> David Mehler wrote: > Thanks, so if I understand what your saying the reason I'm getting the > column update issues is Dovecot is reading from and writing to the > quota and quota_messages columns in my virtual_users table? > > My database user I believe only has select permissions on that table. > > So, I either need another table and to adjust my dovecot-dict.sql file > for that table, that's where Dovecot will write to, or two more > columns in the virtual_users table? > > Which way do you recommend? I would recommend to create a new table for dovecot_usage where dovecot is granted write permission. Regards Daniel From dmalolepszy at optusnet.com.au Mon Oct 22 02:08:01 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Mon, 22 Oct 2012 10:08:01 +1100 Subject: [Dovecot] Dovecot LDA message save logging Message-ID: <50848051.6050308@optusnet.com.au> Hi, Is there any option in Dovecot that enables logging the full path of where a message is saved in the backend? Dominic From dave.mehler at gmail.com Mon Oct 22 02:14:56 2012 From: dave.mehler at gmail.com (David Mehler) Date: Sun, 21 Oct 2012 19:14:56 -0400 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: <20121021222207.GA10903@daniel.localdomain> References: <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> <20121021204919.GA9191@daniel.localdomain> <20121021222207.GA10903@daniel.localdomain> Message-ID: Hello, Thanks. I've created a quota table as described in dovecot-dict sql configuration file and granted the mail user select, insert, update, and delete rights to that table, while the virtual_users table select rights only. I configured for the new table. Here's my config, have I got it? Are the columns now going to stay where I put them? mysql> show create table virtual_users; +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | virtual_users | CREATE TABLE `virtual_users` ( `id` int(11) NOT NULL AUTO_INCREMENT, `domain_id` int(11) NOT NULL, `user` varchar(40) NOT NULL, `password` varchar(128) NOT NULL, `quota` bigint(20) NOT NULL DEFAULT '0', `quota_messages` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`id`), UNIQUE KEY `UNIQUE_EMAIL` (`domain_id`,`user`), CONSTRAINT `virtual_users_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `virtual_domains` (`id`) ON DELETE CASCADE ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 | +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> show create table quota; +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | quota | CREATE TABLE `quota` ( `username` varchar(100) NOT NULL, `bytes` bigint(20) NOT NULL DEFAULT '0', `messages` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`username`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) dovecot dictionary configuration map { pattern = priv/quota/storage table = quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = username value_field = messages } Thanks. Dave. On 10/21/12, Daniel Parthey wrote: > David Mehler wrote: >> Thanks, so if I understand what your saying the reason I'm getting the >> column update issues is Dovecot is reading from and writing to the >> quota and quota_messages columns in my virtual_users table? >> >> My database user I believe only has select permissions on that table. >> >> So, I either need another table and to adjust my dovecot-dict.sql file >> for that table, that's where Dovecot will write to, or two more >> columns in the virtual_users table? >> >> Which way do you recommend? > > I would recommend to create a new table for dovecot_usage > where dovecot is granted write permission. > > Regards > Daniel > From dovecot at knutejohnson.com Mon Oct 22 04:14:56 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 21 Oct 2012 18:14:56 -0700 Subject: [Dovecot] Anybody recognize these log lines? Message-ID: <50849E10.1080404@knutejohnson.com> WARN: Duplicate profile 'Dovecot POP3', using last found WARN: Duplicate profile 'Dovecot Secure POP3', using last found WARN: Duplicate profile 'Dovecot IMAP', using last found WARN: Duplicate profile 'Dovecot Secure IMAP', using last found Anybody know if these are dovecot generated? Thanks, -- Knute Johnson From mcguire at neurotica.com Mon Oct 22 04:17:07 2012 From: mcguire at neurotica.com (Dave McGuire) Date: Sun, 21 Oct 2012 21:17:07 -0400 Subject: [Dovecot] Anybody recognize these log lines? In-Reply-To: <50849E10.1080404@knutejohnson.com> References: <50849E10.1080404@knutejohnson.com> Message-ID: <50849E93.8040706@neurotica.com> On 10/21/2012 09:14 PM, Knute Johnson wrote: > WARN: Duplicate profile 'Dovecot POP3', using last found > WARN: Duplicate profile 'Dovecot Secure POP3', using last found > WARN: Duplicate profile 'Dovecot IMAP', using last found > WARN: Duplicate profile 'Dovecot Secure IMAP', using last found > > Anybody know if these are dovecot generated? Looks like output from the "ufw" firewall package. -Dave -- Dave McGuire, AK4HZ New Kensington, PA From alessio at skye.it Mon Oct 22 08:51:54 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 22 Oct 2012 07:51:54 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> Message-ID: <5084DEFA.6060404@skye.it> Il 13/10/2012 10:42, Timo Sirainen ha scritto: > On 13.10.2012, at 11.16, Alessio Cecchi wrote: > >> I'm running dovecot 2.1.10 on Debian 6. >> >> When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" >> >> [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] > The most helpful way to get this fixed is to get a gdb backtrace: http://dovecot.org/bugreport.html > Hi, after run "dovecot quota recalc -A" no error from "doveadm expunge -A mailbox Trash savedbefore 30d" is appeared and so have not been able to generate the "dump". -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From raabe at froglogic.com Mon Oct 22 10:21:51 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Mon, 22 Oct 2012 09:21:51 +0200 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read In-Reply-To: References: <507EA81C.5060806@froglogic.com> Message-ID: <5084F40F.7070601@froglogic.com> Am 10/18/2012 5:31 AM, schrieb Timo Sirainen: > Use: > > prefix=Lists/anotherlist/ > location = maildir:/home/vmail/lists/sharedseen/Maildir > > Then deliver the mails to /home/vmail/lists/sharedseen/Maildir root directly. Of course this means that you need to create a namespace for each such list. > > Alternative would be to use prefix=Lists/sharedseen/ and create lists under it. Thanks, the second version is basically what I did! I added a new namespace namespace public { separator = / prefix = Lists/Archive/ location = maildir:/home/vmail/lists/archive/Maildir subscriptions = no } ...and then had my Sieve script fileinto that. Works fine! Thanks for your help! -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From amateo at um.es Mon Oct 22 13:58:10 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 22 Oct 2012 12:58:10 +0200 Subject: [Dovecot] Auth caching and password changes In-Reply-To: <508149CC.9070004@um.es> References: <508149CC.9070004@um.es> Message-ID: <508526C2.8030403@um.es> El 19/10/12 14:38, Angel L. Mateo escribi?: > Hello, > > In my system I have configured auth caching. The problem I have is > that whenever a user changes his password, he/she can't login to dovecot > after a while and the scenarios described at > http://wiki2.dovecot.org/Authentication/Caching are not applied. > > I have tried also with "doveadm auth cache flush ", but it > didn't work. He also could to login again if he waits for a time or if I > run "doveadm auth cache flush" in the server, flushing all auth > information from cache. > > I have attached the log I had when I changed my password (and > suffered the problem). I have attached my doveconf -n too. > I think I have found part of the problem. My problem is that my authentication chain is first try by ldap (for normal clients authentication), and if it failed, then try with pam_cas (for webmail accesses with SSO). My change password application forms part of webmail, which also uses an imapproxy, so when I change the password, automatically seems to enter in the "Early change scenario" (I still haven't found the concrete reason for this). But I have tried to manually change the password in my ldap servers, and it works fine. So my point is that something related with this authentication chain provokes this scenario. My question now is there any way to configure authentication so a mechanism is only use when connections coming from a set of IPs? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From tss at iki.fi Mon Oct 22 15:39:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 22 Oct 2012 15:39:34 +0300 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <20120926180633.GE80443@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: On 26.9.2012, at 21.06, Kelsey Cummings wrote: > 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) > 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) > 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) Looks like I had broken this in v2.1.8. http://hg.dovecot.org/dovecot-2.1/rev/e4c337f38ed6 fixes this. I also added a bunch of other things to give better error messages and to try to fix any unexpected problems. From berni at birkenwald.de Mon Oct 22 17:23:18 2012 From: berni at birkenwald.de (Bernhard Schmidt) Date: Mon, 22 Oct 2012 14:23:18 +0000 (UTC) Subject: [Dovecot] auth timeout state=2, bad? Message-ID: Hello, we run Dovecot 2.1.7 as SASL backend for our Postfix SMTP-AUTH farm and see this error message occasionally Oct 22 16:15:32 lxmhs52 dovecot: auth: PLAIN(?,xx.xx.xx.xx): Request 0.21 timeouted after 150 secs, state=2 Since it is mostly the same IP repeating I'm assuming it's a client issue. Is that correct? What could cause this? Can we safely ignore it? Thanks, Bernhard From busseniu at in.tum.de Mon Oct 22 17:33:33 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 22 Oct 2012 16:33:33 +0200 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 Message-ID: <5085593D.3080403@in.tum.de> Hi, Thunderbird uses the following IMAP command to list shared folders: . list "" INBOX.shared.%.% Dovecot 2.1.10 does not list any folders in response to this command. Dovecot 2.0.21 does list them: * LIST (\HasNoChildren) "." "INBOX.shared.user1.folder" . OK List completed. Both versions list the folders if "*" is used instead of "INBOX.shared.%.%". Because of this issue, shared folders are not shown in Thunderbird. The ACL of the folder in question is ACL "keilrwtscd" for user1 (who is trying to access them using Thunderbird). config: # 2.1.10: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mail mail_plugins = acl mail_uid = vmail namespace { inbox = no list = children location = mdbox:%%h/mail prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } namespace default { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/mail/shared-mailboxes } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Mon Oct 22 18:59:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 22 Oct 2012 18:59:52 +0300 Subject: [Dovecot] auth timeout state=2, bad? In-Reply-To: References: Message-ID: <9790CECF-1FCE-4125-929B-CE0A53483495@iki.fi> On 22.10.2012, at 17.23, Bernhard Schmidt wrote: > we run Dovecot 2.1.7 as SASL backend for our Postfix SMTP-AUTH farm and > see this error message occasionally > > Oct 22 16:15:32 lxmhs52 dovecot: auth: PLAIN(?,xx.xx.xx.xx): Request > 0.21 timeouted after 150 secs, state=2 state 2 = "waiting for auth data from client", so what it means is that the client sent: AUTH PLAIN and then just didn't do anything. > Since it is mostly the same IP repeating I'm assuming it's a client > issue. Is that correct? What could cause this? Can we safely ignore it? You can safely ignore it. It should have been logged with "info" level and only with auth_verbose=yes. Although I guess the message could be a bit nicer. This is better I think: http://hg.dovecot.org/dovecot-2.1/rev/49bb6cc43d03 From kgc at corp.sonic.net Tue Oct 23 01:29:21 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 22 Oct 2012 15:29:21 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: <20121022222921.GE3370@corp.sonic.net> On Mon, Oct 22, 2012 at 03:39:34PM +0300, Timo Sirainen wrote: > On 26.9.2012, at 21.06, Kelsey Cummings wrote: > > > 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) > > 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) > > 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) > > Looks like I had broken this in v2.1.8. http://hg.dovecot.org/dovecot-2.1/rev/e4c337f38ed6 fixes this. I also added a bunch of other things to give better error messages and to try to fix any unexpected problems. Thanks Timo! -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From dovecot at knutejohnson.com Tue Oct 23 03:29:54 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Mon, 22 Oct 2012 17:29:54 -0700 Subject: [Dovecot] Anybody recognize these log lines? In-Reply-To: <50849E93.8040706@neurotica.com> References: <50849E10.1080404@knutejohnson.com> <50849E93.8040706@neurotica.com> Message-ID: <5085E502.3080802@knutejohnson.com> On 10/21/2012 6:17 PM, Dave McGuire wrote: > On 10/21/2012 09:14 PM, Knute Johnson wrote: >> WARN: Duplicate profile 'Dovecot POP3', using last found >> WARN: Duplicate profile 'Dovecot Secure POP3', using last found >> WARN: Duplicate profile 'Dovecot IMAP', using last found >> WARN: Duplicate profile 'Dovecot Secure IMAP', using last found >> >> Anybody know if these are dovecot generated? > > Looks like output from the "ufw" firewall package. > > -Dave > Thanks, I'll look at that. -- Knute Johnson From list at airstreamcomm.net Tue Oct 23 08:49:47 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 23 Oct 2012 00:49:47 -0500 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <20121021194320.GA7977@daniel.localdomain> References: <5083D963.3000700@wk-serv.de> <20121021194320.GA7977@daniel.localdomain> Message-ID: <50862FFB.2060108@airstreamcomm.net> On 10/21/12 2:43 PM, Daniel Parthey wrote: > Patrick Westenberg wrote: >> Is there no built in failover mechanism for the director service to >> handle a backend failure? > No, the director's job is to keep a hash table and direct > the connection for each user to its associated backend. > Currently, there is no built-in backend monitoring. > > In order to handle maintenance of backends, you will need the > poolmon daemon, which enables/disables backends in the director > depending on their availability: > > https://github.com/brandond/poolmon > > Regards > Daniel Considering the intention of the director was to alleviate locking issues in a shared storage environment are there any current solutions to improving the scalability/availability of Dovecot by implementing an alternative message storage systems such as nosql or maybe object storage that could abstract away the complexity of replicating data? We would love to finally have the ability to set our mail cluster on top of a storage subsystem that can span multiple geographic regions and do away with the NFS backend. From alessio at skye.it Tue Oct 23 09:00:52 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 23 Oct 2012 08:00:52 +0200 Subject: [Dovecot] Dovecot LDA message save logging In-Reply-To: <50848051.6050308@optusnet.com.au> References: <50848051.6050308@optusnet.com.au> Message-ID: <50863294.2010404@skye.it> Il 22/10/2012 01:08, Dominic Malolepszy ha scritto: > Hi, > > Is there any option in Dovecot that enables logging the full path of > where a message is saved in the backend? > > Dominic > With mail_debug=yes you can see it. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From eliezer at ngtech.co.il Tue Oct 23 18:19:04 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Tue, 23 Oct 2012 17:19:04 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 Message-ID: <5086B568.1010905@ngtech.co.il> Since I have lots of filtering rules in thunderbird I was thinking of using sieve instead. I want to filter incoming mail into subdirectories. like "from" store at folder "old". the script is: require ["fileinto", "envelope"]; if envelope :is "from" "eliezer at test.dom" { fileinto "old"; } else { # The rest goes into INBOX # default is "implicit keep", we do it explicitly here keep; } the result is that the mail is stored in two folders instead of just one, INBOX and old. the logs shows: Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing script from /home/vmail/domain/eliezer/home/.dovecot.svbin Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' from unknown reason(or I didnt understood how sieve works?) plugin section from dovecot -n plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.debug +imapflags +relational +comparator-i;ascii-numeric } Thanks, Eliezer From stsiol at yahoo.co.uk Tue Oct 23 19:03:38 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 23 Oct 2012 17:03:38 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb [Resolved] Message-ID: <1351008218.44057.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello all, I would like to thank you all for your kind replies and feedback in regards to migrating from a smaller hdd to a bigger one (namely from 72gb to 146gb). I finally found a painless way of doing this. Since I believe that this is still an off-topic post, if anyone is interested in the solution i've adopted for this, let me know by replying to me privately. If, however, you don't mind me posting here, let me know. Many many thanks go to?Alexander Hoogerhuis, the "mad Norwegian" :-) who helped me on this too much to describe here. Alex, you are a true sport. Thank you again people. All the Best, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From dg at dguhl.org Tue Oct 23 19:21:27 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 23 Oct 2012 18:21:27 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <20121023162125.GA7983@PC211.ikt.de> On Tue, Oct 23, 2012 at 05:19:04PM +0200, Eliezer Croitoru wrote: [..] > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; stop; # seems to be needed with explicit keep > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } Comparing various sieve scripts I concluded (but I didn't actually test it) that you need a stop; in your if or elsif clauses in case you have an else clause with an explicit keep; -- IMHO a behaviour violating RFC 5228. RFC 5228, Section 3.1 ... If the test of the "if" is false, it evaluates the test of the first "elsif" (if any). If the test of "elsif" is true, it runs the elsif's block. An elsif may be followed by an elsif, in which case, the interpreter repeats this process until it runs out of elsifs. When the interpreter runs out of elsifs, there may be an "else" case. If there is, and none of the if or elsif tests were true, the interpreter runs the else's block. ... Stefan, can you enlighten us? Dennis [..] From john.roman at dreamhost.com Tue Oct 23 20:04:12 2012 From: john.roman at dreamhost.com (john roman) Date: Tue, 23 Oct 2012 10:04:12 -0700 Subject: [Dovecot] index files created improperly in dovecot 1.2.16 Message-ID: Greetings, It seems to be a problem that ive seen occasionally on the web with few results as to a solution, but im experiencing it as well. Namely, dovecot creates index logs for users with a 600 permission, when it should create with a 700 permission. My indexes are stored in /var/indexes with the directory at 777 permission, the users are identified in MySQL. The error is as follows: Oct 23 09:03:13 mailer01 dovecot: POP3 (johnr at testing.com): stat(/var/indexes/j/johnr at testing.com/.INBOX) failed: Permission denied (euid=10509305(x10509305) egid=81607(pg199275) missing +x perm: /var/indexes/j) the permissions of /var/indexes/j are 700, owned by the euid and egid effectively. the only file that does not have this permission is in /var/indexes/j/johnr at testing.com/.INBOX/dovecot.index.log, at 600. From trashcan at odo.in-berlin.de Tue Oct 23 22:06:51 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 23 Oct 2012 21:06:51 +0200 Subject: [Dovecot] [2.2-UNSTABLE] compilation error: 'POSIX_FADV_WILLNEED' undeclared Message-ID: <198596C8-3989-4041-B96A-5D5AFEE8E3D0@odo.in-berlin.de> Hi -- I am trying to compile 2.2 (acd76b5272e9) at FreeBSD 9.0: | libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I/usr/local/include -std=gnu99 -O2 -Wall -W -Wmissing-prototypes \ -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 \ -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 \ -I/usr/local/include -MT fs-posix.lo -MD -MP -MF .deps/fs-posix.Tpo \ -c fs-posix.c -fPIC -DPIC -o .libs/fs-posix.o | fs-posix.c: In function 'fs_posix_prefetch': | fs-posix.c:298: warning: implicit declaration of function 'posix_fadvise' | fs-posix.c:298: error: 'POSIX_FADV_WILLNEED' undeclared (first use in this function) | fs-posix.c:298: error: (Each undeclared identifier is reported only once | fs-posix.c:298: error: for each function it appears in.) | gmake[3]: *** [fs-posix.lo] Error 1 | gmake[3]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src/lib-fs' | gmake[2]: *** [all-recursive] Error 1 | gmake[2]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src' | gmake[1]: *** [all-recursive] Error 1 | gmake[1]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2' | gmake: *** [all] Error 2 From configure logfile: | checking for posix_fadvise... no After a modification (stolen from src/lib-storage/index/index-mail.c) ... | --- dovecot-2.2-modified/src/lib-fs/fs-posix.c 2012-10-23 20:27:31.348919455 +0200 | +++ dovecot-2.2/src/lib-fs/fs-posix.c 2012-10-23 20:26:39.435300269 +0200 | @@ -295,10 +295,12 @@ | return TRUE; | } | | +#if defined(HAVE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED) | if (posix_fadvise(file->fd, 0, length, POSIX_FADV_WILLNEED) < 0) { | i_error("posix_fadvise(%s) failed: %m", _file->path); | return TRUE; | } | +#endif | return FALSE; | } ... the compilations runs to completion, and dovecot-2.2 UNSTABLE is running. But: I do not have the knowledge to judge if that "fix" will be the right one. Just to let you know and with regards, Michael From stephan at rename-it.nl Tue Oct 23 22:40:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 23 Oct 2012 21:40:43 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <5086F2BB.7010704@rename-it.nl> On 10/23/2012 5:19 PM, Eliezer Croitoru wrote: > Since I have lots of filtering rules in thunderbird I was thinking of > using sieve instead. > I want to filter incoming mail into subdirectories. > like "from" store at folder "old". > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } > > the result is that the mail is stored in two folders instead of just > one, INBOX and old. > the logs shows: > Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing > script from /home/vmail/domain/eliezer/home/.dovecot.svbin > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' > > from unknown reason(or I didnt understood how sieve works?) This behavior would definitely be a bug, but I cannot reproduce it even with Dovecot 2.0.17 and Pigeonhole v0.2.5: stephan at host:~/src/devel/dovecot-2.0-pigeonhole$ src/sieve-tools/sieve-test -t - -Tlevel=matching -e ~/frop.sieve ~/frop.eml ## Started executing script 'frop' 2: envelope test 2: starting `:is' match with `i;ascii-casemap' comparator: 2: getting `from' part from message envelope 2: extracting `all' part from address `eliezer at test.dom' 2: matching value `eliezer at test.dom' 2: with key `eliezer at test.dom' => 1 2: finishing match with result: matched 2: jump if result is false 2: not jumping 3: fileinto action 3: store message in mailbox `old' 3: jumping to line 7 ## Finished executing script 'frop' info: msgid=unspecified: stored mail into mailbox 'old'. sieve-test(stephan): Info: final result: success Could you test this at your end? Be careful, the above command adds a message to the user's mailbox, so read the sieve-test man page first before you try anything. Also, I executed this from my development tree, because I haven't got an operational Dovecot v2.0 installation. I don't remember any bug that was solved since that version that could explain what you're seeing. What is your Pigeonhole version? I've tried with v0.2.5 at this end. Also, could you provide your full configuration as output from `dovecot -n` ? Regards, Stephan. From benedetto.vassallo at unipa.it Tue Oct 23 23:18:06 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Tue, 23 Oct 2012 22:18:06 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> <20121016091153.15601eysq5n040qh@webmail.unipa.it> <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> Message-ID: <20121023221806.43795tha204qxgfy@webmail.unipa.it> Def. Quota Timo Sirainen : > On 16.10.2012, at 10.11, Benedetto Vassallo wrote: > >>> What are the permissions of the MailDir directory for user1/user2? >>> >>> ls -ld /home/user1/MailDir >>> ls -ld /home/user2/MailDir >>> >>> >> >> Thank you for your reply. >> They are different groups: >> >> drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ >> drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ >> drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ > > Not very secure permissions.. Maybe would be easiest to just have > one vmail user for everyone? > >> I tryed to issue: >> chgrp -R mail /home/user1/MailDir >> chgrp -R mail /home/user2/MailDir >> chgrp -R mail /home/user3/MailDir > > Dovecot doesn't do hard linking when it looks like the permissions > aren't compatible. The current code checks that if the owner UIDs > are different, then the group needs to be writable. On my production server with dovecot 2.0.13 I have same permissions and it works. I changed my permissions in any mode, changed the owner, the group but it still don't work. Any suggestion? Thank you -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From marc at perkel.com Tue Oct 23 23:51:39 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 23 Oct 2012 13:51:39 -0700 Subject: [Dovecot] Can Dovecot authenticate against an external email server? Message-ID: <5087035B.7060208@perkel.com> Just wondering if anyone has done this. I have a spam filtering service where I am now storing spam for users I filter for. It's a filter and forward service so I don't control the recipient's email server. What I would like to do somehow is have the user enter their email address and password and then look up their POP/IMAP server from a database and try to authenticate from it. If sucessful then the user will be able to access their stored spam using Dovecot and Squirrelmail. Any suggestions? Thanks in advance From troy at troyvit.com Tue Oct 23 23:52:45 2012 From: troy at troyvit.com (Troy Vitullo) Date: Tue, 23 Oct 2012 14:52:45 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver Message-ID: <20121023145245.124dd362@hrafn> Hi, My server uses a system comprised of postfix, dovecot and dspam to filter and deliver mail. Postfix used the following flags in calling spamc and dovecot: flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} after an upgrade from Debian lenny to squeeze we were able to get everything working except spam filtering. Spamassassin is able to judge whether the mail coming in is spam but everything stops there. In mail.err I see: pamc[3608]: exec failed: Permission denied spamc shows the same thing in syslog: exec failed: Permission denied postfix delays the email: postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system resource problem) Here are the permissions for deliver: -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver Here are the relevant groups: s1:~# grep dovecot /etc/group secmail:x:119:postfix,spamd,dovecot dovecot:x:111: here's the dovecot user: s1:~# grep dovecot /etc/passwd dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false here's dovecot -n: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 base_dir: /var/run/dovecot/ protocols: imap imaps pop3s pop3 ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert ssl_key_file: /etc/ssl/private/s1.troyvit.com.key ssl_cipher_list: ALL:!LOW disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_location: maildir:%h/Maildir/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s namespace: type: private separator: / inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmaster at sphere.local auth_socket_path: /var/run/dovecot/auth-master mail_plugin_dir: /usr/lib/dovecot/modules/lda/ mail_plugins: sieve auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: pam args: dovecot passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 438 user: dovecot plugin: sieve_global_path: /etc/dovecot/default.sieve sieve: /srv/%d/mail/%n/%n.sieve Many thanks in advance for any advice you can give. Troy From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 24 01:12:33 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 24 Oct 2012 00:12:33 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <50862FFB.2060108@airstreamcomm.net> References: <5083D963.3000700@wk-serv.de> <20121021194320.GA7977@daniel.localdomain> <50862FFB.2060108@airstreamcomm.net> Message-ID: <20121023221233.GA22084@daniel.localdomain> list at airstreamcomm.net wrote: > Considering the intention of the director was to alleviate locking > issues in a shared storage environment are there any current > solutions to improving the scalability/availability of Dovecot by > implementing an alternative message storage systems such as nosql or > maybe object storage that could abstract away the complexity of > replicating data? We would love to finally have the ability to set > our mail cluster on top of a storage subsystem that can span > multiple geographic regions and do away with the NFS backend. Key/value object store is planned for Dovecot v2.2 and has been discussed in this thread: http://dovecot.org/list/dovecot/2012-September/068257.html Regards Daniel -- https://plus.google.com/103021802792276734820 From Bill at KnoxvilleChristian.org Wed Oct 24 04:06:17 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:06:17 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121023145245.124dd362@hrafn> References: <20121023145245.124dd362@hrafn> Message-ID: <50873F09.4070604@KnoxvilleChristian.org> On 10/23/2012 4:52 PM, Troy Vitullo wrote: > Hi, > > My server uses a system comprised of postfix, dovecot and dspam to filter and deliver mail. > > Postfix used the following flags in calling spamc and dovecot: > > flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} > > after an upgrade from Debian lenny to squeeze we were able to get everything working except spam filtering. Spamassassin is able to judge whether the mail coming in is spam but everything stops there. > > In mail.err I see: > > pamc[3608]: exec failed: Permission denied > > spamc shows the same thing in syslog: > > exec failed: Permission denied > > postfix delays the email: > > postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system resource problem) > > Here are the permissions for deliver: > > -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver > > Here are the relevant groups: > > s1:~# grep dovecot /etc/group > secmail:x:119:postfix,spamd,dovecot > dovecot:x:111: > > here's the dovecot user: > s1:~# grep dovecot /etc/passwd > dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false > > here's dovecot -n: > > # 1.2.15: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 > base_dir: /var/run/dovecot/ > protocols: imap imaps pop3s pop3 > ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert > ssl_key_file: /etc/ssl/private/s1.troyvit.com.key > ssl_cipher_list: ALL:!LOW > disable_plaintext_auth: no > verbose_ssl: yes > login_dir: /var/run/dovecot/login > login_executable(default): /usr/lib/dovecot/imap-login > login_executable(imap): /usr/lib/dovecot/imap-login > login_executable(pop3): /usr/lib/dovecot/pop3-login > mail_location: maildir:%h/Maildir/ > mbox_write_locks: fcntl dotlock > mail_executable(default): /usr/lib/dovecot/imap > mail_executable(imap): /usr/lib/dovecot/imap > mail_executable(pop3): /usr/lib/dovecot/pop3 > mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > pop3_enable_last(default): no > pop3_enable_last(imap): no > pop3_enable_last(pop3): yes > pop3_client_workarounds(default): > pop3_client_workarounds(imap): > pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh > pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s > namespace: > type: private > separator: / > inbox: yes > list: yes > subscriptions: yes > lda: > postmaster_address: postmaster at sphere.local > auth_socket_path: /var/run/dovecot/auth-master > mail_plugin_dir: /usr/lib/dovecot/modules/lda/ > mail_plugins: sieve > auth default: > mechanisms: plain login > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: pam > args: dovecot > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: passwd > userdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: postfix > master: > path: /var/run/dovecot/auth-master > mode: 438 > user: dovecot > plugin: > sieve_global_path: /etc/dovecot/default.sieve > sieve: /srv/%d/mail/%n/%n.sieve > > Many thanks in advance for any advice you can give. > > Troy What is your mailbox_command in main.cf? I just use: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" I don't need anything in master.cf. But you should be using -u ${user} for spamc. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 04:15:34 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:15:34 -0400 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <50874136.7000903@KnoxvilleChristian.org> On 10/23/2012 11:19 AM, Eliezer Croitoru wrote: > Since I have lots of filtering rules in thunderbird I was thinking of > using sieve instead. > I want to filter incoming mail into subdirectories. > like "from" store at folder "old". > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } > > the result is that the mail is stored in two folders instead of just > one, INBOX and old. > the logs shows: > Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing > script from /home/vmail/domain/eliezer/home/.dovecot.svbin > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' > > from unknown reason(or I didnt understood how sieve works?) > > plugin section from dovecot -n > > plugin { > ... > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > sieve_extensions = +vnd.dovecot.debug +imapflags +relational > +comparator-i;ascii-numeric > } > > Thanks, > Eliezer Why to people bother with all these complex if...elsif...else structures. I just use stop a lot. This is included from my Main.seive: # # 2012-07-05 # require "include"; require "fileinto"; require "copy"; #require "body"; #require "imap4flags"; # put this in main #if header :contains "list-id" "dovecot.dovecot.org" { include "Dovecot"; } if address :is :localpart "to" "dovecot" { fileinto :copy "SystemFolders.Ham"; fileinto "Lists.Dovecot"; stop; } fileinto "Lists"; stop; As you can see, I also train Spamassassin with the mail from the list. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 04:32:59 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:32:59 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50873F09.4070604@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> Message-ID: <5087454B.5030704@KnoxvilleChristian.org> On 10/23/2012 9:06 PM, Bill Shirley wrote: > > On 10/23/2012 4:52 PM, Troy Vitullo wrote: >> Hi, >> >> My server uses a system comprised of postfix, dovecot and dspam to >> filter and deliver mail. >> >> Postfix used the following flags in calling spamc and dovecot: >> >> flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} >> -e /usr/lib/dovecot/deliver -d ${recipient} >> >> after an upgrade from Debian lenny to squeeze we were able to get >> everything working except spam filtering. Spamassassin is able to >> judge whether the mail coming in is spam but everything stops there. >> >> In mail.err I see: >> >> pamc[3608]: exec failed: Permission denied >> >> spamc shows the same thing in syslog: >> >> exec failed: Permission denied >> >> postfix delays the email: >> >> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system >> resource problem) >> >> Here are the permissions for deliver: >> >> -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver >> >> Here are the relevant groups: >> >> s1:~# grep dovecot /etc/group >> secmail:x:119:postfix,spamd,dovecot >> dovecot:x:111: >> >> here's the dovecot user: >> s1:~# grep dovecot /etc/passwd >> dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false >> >> here's dovecot -n: >> >> # 1.2.15: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 >> base_dir: /var/run/dovecot/ >> protocols: imap imaps pop3s pop3 >> ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert >> ssl_key_file: /etc/ssl/private/s1.troyvit.com.key >> ssl_cipher_list: ALL:!LOW >> disable_plaintext_auth: no >> verbose_ssl: yes >> login_dir: /var/run/dovecot/login >> login_executable(default): /usr/lib/dovecot/imap-login >> login_executable(imap): /usr/lib/dovecot/imap-login >> login_executable(pop3): /usr/lib/dovecot/pop3-login >> mail_location: maildir:%h/Maildir/ >> mbox_write_locks: fcntl dotlock >> mail_executable(default): /usr/lib/dovecot/imap >> mail_executable(imap): /usr/lib/dovecot/imap >> mail_executable(pop3): /usr/lib/dovecot/pop3 >> mail_plugin_dir(default): /usr/lib/dovecot/modules/imap >> mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap >> mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 >> pop3_enable_last(default): no >> pop3_enable_last(imap): no >> pop3_enable_last(pop3): yes >> pop3_client_workarounds(default): >> pop3_client_workarounds(imap): >> pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh >> pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s >> pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s >> pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s >> namespace: >> type: private >> separator: / >> inbox: yes >> list: yes >> subscriptions: yes >> lda: >> postmaster_address: postmaster at sphere.local >> auth_socket_path: /var/run/dovecot/auth-master >> mail_plugin_dir: /usr/lib/dovecot/modules/lda/ >> mail_plugins: sieve >> auth default: >> mechanisms: plain login >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: pam >> args: dovecot >> passdb: >> driver: sql >> args: /etc/dovecot/dovecot-sql.conf >> userdb: >> driver: passwd >> userdb: >> driver: sql >> args: /etc/dovecot/dovecot-sql.conf >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: postfix >> master: >> path: /var/run/dovecot/auth-master >> mode: 438 >> user: dovecot >> plugin: >> sieve_global_path: /etc/dovecot/default.sieve >> sieve: /srv/%d/mail/%n/%n.sieve >> >> Many thanks in advance for any advice you can give. >> >> Troy > > What is your mailbox_command in main.cf? I just use: > mailbox_command = /usr/bin/spamc -u "$USER" -e > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" > > I don't need anything in master.cf. But you should be using -u > ${user} for spamc. > > Bill > Forgot to ask, are you using Spamassassin's per-user configs? If you're not, that probably is your problem. It's probably trying to update bayes tokens and it doesn't have permission. I use per-user configs which are nice. One man's spam is another man's ham. Plus each user can have his/her own whitelist. I use these spamd args: -d -c -m10 --user-config You usually can find the args in /etc/sysconfig. Bill From rs at sys4.de Wed Oct 24 09:33:26 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 08:33:26 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5087454B.5030704@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> Message-ID: <50878BB6.2090309@sys4.de> Am 24.10.2012 03:32, schrieb Bill Shirley: > What is your mailbox_command in main.cf? I just use: > mailbox_command = /usr/bin/spamc -u "$USER" -e > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" > > I don't need anything in master.cf. But you should be using -u ${user} > for spamc. long time ago i tested this with dovecot lda postfix master.cf with a total virtual setup dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} but i strongly do not recommand this !!! use spamass-milter, amavis etc with dovecot lmtp as described on many sites Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From carsten.delellis at delellis.net Wed Oct 24 12:48:34 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Wed, 24 Oct 2012 11:48:34 +0200 Subject: [Dovecot] dovecot auth against AD on samba4 Message-ID: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> Hi group I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and I am thinking about installing it because of it's native outlook support capabilities. The ZEG appliance wouldn't be an option for me because I use a virtual server from a provider where I can't install my own vm or even an iso. When I go thru the documentation there is a part with installing OpenChange based on samba4. As far as I understood the OpenChange authentication is against the samba4 AD. Actually there is no support in syncing the AD against an OpenLdap Server and I would have to change the OpenLdap port because the AD is listening on port 389. To change the port wouldn't be a big deal, but what i was thinking about to run the dovecot auth also against the samba 4 AD. I searched around on the internet but didn't find a doc yet how to do that. Does anyone here could provide me with a link or a how-to ? Thanks very much in advance. Regards, Carsten Laun-De Lellis Hauptstrasse 13 D-67705 Trippstadt Phone: +49 6306 992140 Fax: +49 6306 992142 Mobile: +49 151 27530865 email: carsten.delellis at delellis.net From listen at mjh.name Wed Oct 24 14:28:11 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Wed, 24 Oct 2012 13:28:11 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox Message-ID: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Hello all, I have a problem with an incosistent mdbox: Oct 24 10:43:23 two dovecot: imap-login: Login: user=<...>, method=PLAIN, rip=..., lip=..., mpid=4977, TLS Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-24 10:43:23] bytes=115/53726 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: Inconsistency in map index (467,31960 != 467,36768) Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: rebuilding indexes Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Panic: file mdbox-storage-rebuild.c: line 773 (rebuild_update_refcounts): assertion failed: (map_uid < msgs[i]->map_uid) Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x380ca) [0x7f99cf35b0ca] -> /usr/lib/dovecot/libdovecot.so.0(+0x3810e) [0x7f99cf35b10e] -> /usr/li b/dovecot/libdovecot.so.0(i_fatal+0) [0x7f99cf334a67] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x10a5) [0x7f99cf5f42d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_s torage_rebuild+0x24) [0x7f99cf5f4414] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x53565) [0x7f99cf5f4565] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7f99cf5c8caa] -> /usr/lib/dovec ot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7f99cf5c8cf3] -> dovecot/imap(client_destroy+0x109) [0x7f99cfaa69e9] -> dovecot/imap(client_input+0xaa) [0x7f99cfaa6dba] -> /usr/lib/dovecot/libdovecot.so.0(i o_loop_call_io+0x48) [0x7f99cf366c98] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f99cf367d27] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f99cf366c28] -> /usr/lib/dovecot /libdovecot.so.0(master_service_run+0x13) [0x7f99cf354e33] -> dovecot/imap(main+0x304) [0x7f99cfa9e554] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f99cef8576d] -> dovecot/imap(+0x95e5) [0 x7f99cfa9e5e5] Oct 24 10:45:19 two dovecot: master: Error: service(imap): child 4977 killed with signal 6 (core dumps disabled) I use: Dovecot 2.0.19-0ubuntu1 Ubuntu 12.04, x86-64, Kernel 3.2.0-32-generic local XFS filesystem for the mdbox The problem appeared out of nowhere. Many messages been continously delivered to this mailbox on this installation since May 2012, and the mdbox was only accessed with deliver and imap/pop3 from dovecot. About four hours after the problem initially appeared, I did a hard reset of the system because it was unresponsive. The error message is exactly the same before and after the hard reset. The problem is triggered by both IMAP access and dovecot deliver access. The whole mdbox is 6.6 GiB large and I guess that it contains about 300k-600k messages. It's an archive of public mailing lists, so I could give access to the files. Can anybody say something about this? May the mdbox be repaired? Regards, Milan Holz?pfel -- Milan Holz?pfel -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doveconf.txt URL: From rs at sys4.de Wed Oct 24 14:43:19 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 13:43:19 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <5087D457.6040205@sys4.de> Am 24.10.2012 13:28, schrieb Milan Holz?pfel: > The whole mdbox is 6.6 GiB large and I guess that it contains about > 300k-600k messages. It's an archive of public mailing lists, so I could > give access to the files. > > Can anybody say something about this? May the mdbox be repaired? perhaps this helps http://wiki2.dovecot.org/Tools/Doveadm/ForceResync however upgrading to dovecot latest might be a good idea Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stan at hardwarefreak.com Wed Oct 24 17:01:24 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 09:01:24 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <5087F4B4.2060107@hardwarefreak.com> On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: > I have a problem with an incosistent mdbox: ... > four hours after the problem initially appeared, I did a hard reset of > the system because it was unresponsive. ... > Can anybody say something about this? May the mdbox be repaired? If the box is truly unresponsive, i.e. hard locked, then the corrupted indexes are only a symptom of the underlying problem, which is unrelated to Dovecot, UNLESS, the lack of responsiveness was due to massive disk access, which will occur when rebuilding indexes on a 6.6GB mailbox. You need to know the difference so we have accurate information to troubleshoot with. If the there's a kernel or hardware problem, you should see related errors in dmesg. Please share those. Neither Timo nor anyone here can fix your index problem if the cause lie elsewhere. You must fix the root problem first. -- Stan From CMarcus at Media-Brokers.com Wed Oct 24 17:45:01 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 24 Oct 2012 10:45:01 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087F4B4.2060107@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> Message-ID: <5087FEED.7060007@Media-Brokers.com> On 2012-10-24 10:01 AM, Stan Hoeppner wrote: > If the box is truly unresponsive, i.e. hard locked, then the corrupted > indexes are only a symptom of the underlying problem, which is unrelated > to Dovecot, UNLESS, the lack of responsiveness was due to massive disk > access, which will occur when rebuilding indexes on a 6.6GB mailbox. > You need to know the difference so we have accurate information to > troubleshoot with. Hmmm... I wonder would it be possible for dovecot to automatically lower the 'niceness' for index rebuilds (on systems that support such) to avoid causing such distress? -- Best regards, Charles From weber at papaya-cms.com Wed Oct 24 17:46:39 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Wed, 24 Oct 2012 16:46:39 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. Message-ID: <5087FF4F.8050103@papaya-cms.com> Hi, I've got some trouble here.. i created some sieve rules, but the debug log says that there is a invalid mailbox name error: msgid=<*>: failed to store into mailbox '/home/shared/.automail.Bugtracker/': Invalid mailbox name. ~/.dovecot.sieve if address :is "to" "mantis-admin@<*>" { fileinto "/home/shared/.automail.Bugtracker/"; } here's my doveconf -n output # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-32-generic x86_64 Ubuntu 12.04.1 LTS base_dir = /var/run/dovecot/ disable_plaintext_auth = no hostname = * mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = yes location = maildir:/home/shared:CONTROL=~/.Maildir/control/Shared:INDEX=~/.Maildir/index/Shared prefix = shared/ separator = / subscriptions = yes type = public } namespace { inbox = yes location = prefix = separator = / type = private } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@* protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } ssl_ca = was automatically rejected:%n%r } protocol imap { mail_plugins = " zlib, acl, imap_zlib" } any idea? Mit freundlichen Gruessen / best regards papaya Software GmbH i.A. Alexander Weber -- papaya Software GmbH | Im MediaPark 5 | 50670 Koeln | Germany Tel./Ph.: +49-221-5743-8070 | Fax: +49-221-5743-8099 mailto:weber at papaya-cms.com | http://www.papaya-cms.com/ -- Geschaeftsfuehrer: Andreas Jacobi, Andr? Schnitzler, Daniel Sch?fer Sitz& Registergericht: Koeln | HRB 60030 | USt.-Id.-Nr.: DE 255642963 -- From sandro.tosi at dada.eu Wed Oct 24 17:48:44 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Wed, 24 Oct 2012 16:48:44 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50808A57.8040201@rename-it.nl> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> <507BBE00.9010007@dada.eu> <50808A57.8040201@rename-it.nl> Message-ID: <5087FFCC.5080504@dada.eu> Hello Stephan, sorry for this late reply. On 10/19/2012 01:01 AM, Stephan Bosch wrote: > On 10/15/2012 9:40 AM, Sandro Tosi wrote: >> Hi Stephan, >> thanks a lot for your reply. >> >> On 10/11/2012 10:35 PM, Stephan Bosch wrote: >>> On 10/10/2012 11:23 AM, Sandro Tosi wrote: >>>> Hello, >>>> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >>>> as this will be provided with an upcoming Debian package) with MySQL >>>> dict lookups with the mail setup we're designing. >>>> >>>> Our (main) goals are: >>>> >>>> 1. store the filters on the database >>> That is possible with some limitations. >> >> Are the ones below the only limitatios (ie one script per user) or are >> there any other worth knowing? > > You cannot currently use ManageSieve when the active script is located > in a dict database. > > And 'one script per user' is not an fully accurate description. It is > technically possible to access multiple different scripts from the dict > database. It is however not possible to use dict support combination > with multiscript support ( > http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Executing_Multiple_Scripts_Sequentially) > to execute multiple scripts in a sequence. Multiscript currently only > works for Sieve scripts that are located in the filesystem. > >> In our situation, what would you suggest? We're now thinking of >> keeping the scripts list on a separate table, and merge the "user >> selected ones" in a single script to write in the filters table. Is >> that what would you suggest? Is there a better solution? > > You can use the include extension > (https://tools.ietf.org/html/draft-ietf-sieve-include-05) to access > scripts in a dict database from a main active script to combine them. I > believe you could even dynamically construct that main script in SQL > using some string manipulation in the query, but that is a bit ugly. > > Could you send me an overview of your configuration, including your > database layout? Provided that I have some time in the next week, I > could investigate building a simple working configuration for the sake > of example. I will follow this up privately (you know, we can't disclose too much) but JFTR we decided to follow a half-and-half solution: - we keep on the backend database all the scripts the customer could activate in separate rows - from them, we merge into a single sieve script file all the filter the customer has decided to activate. This way we still record the script separately in the db, so once we'll be able to feed pigeonhole with multiple lines, it's already there, and then merging into a single file is the most straightforward and simple solution to make what we need to work. Thanks for the support, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From Bill at KnoxvilleChristian.org Wed Oct 24 18:47:07 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 11:47:07 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50878BB6.2090309@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> Message-ID: <50880D7B.4090407@KnoxvilleChristian.org> On 10/24/2012 2:33 AM, Robert Schetterer wrote: > Am 24.10.2012 03:32, schrieb Bill Shirley: >> What is your mailbox_command in main.cf? I just use: >> mailbox_command = /usr/bin/spamc -u "$USER" -e >> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >> >> I don't need anything in master.cf. But you should be using -u ${user} >> for spamc. > long time ago i tested this with dovecot lda postfix master.cf > with a total virtual setup > > dovecot unix - n n - - pipe > flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e > /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} > > but i strongly do not recommand this !!! > > use spamass-milter, amavis etc with dovecot lmtp > as described on many sites > > > Best Regards > MfG Robert Schetterer > Can you get per-user Spamassassin configs this way? Why user=vmail:vmail? Is this for virtual domains? I didn't think we were talking about them. Instead of strongly recommending against this, why not elaborate on the problems with using spamc in the mailbox_command? Bill From bob at computerisms.ca Wed Oct 24 19:04:39 2012 From: bob at computerisms.ca (Bob Miller) Date: Wed, 24 Oct 2012 09:04:39 -0700 Subject: [Dovecot] dovecot auth against AD on samba4 In-Reply-To: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> References: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> Message-ID: <1351094679.2143.474.camel@worklian> I don't have it in production yet because there are other things I am still trying to add to samba4, but my test server has dovecot authenticating against samba4. Without openchange or any other non-native mechanism. Dovecot supports authenticating against ldap, the settings are in your auth-ldap.conf file. Samba4/Active Directory is just another ldap implementation. between the config files and the wiki, I believe all the documentation you need is there... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Wed, 2012-10-24 at 11:48 +0200, Carsten Laun-De Lellis wrote: > Hi group > > I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, > dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and > I am thinking about installing it because of it's native outlook support > capabilities. > > The ZEG appliance wouldn't be an option for me because I use a virtual > server from a provider where I can't install my own vm or even an iso. > > When I go thru the documentation there is a part with installing > OpenChange based on samba4. As far as I understood the OpenChange > authentication is against the samba4 AD. Actually there is no support in > syncing the AD against an OpenLdap Server and I would have to change the > OpenLdap port because the AD is listening on port 389. To change the port > wouldn't be a big deal, but what i was thinking about to run the dovecot > auth also against the samba 4 AD. > > I searched around on the internet but didn't find a doc yet how to do that. > > Does anyone here could provide me with a link or a how-to ? > > Thanks very much in advance. > > Regards, > > Carsten Laun-De Lellis > > Hauptstrasse 13 > D-67705 Trippstadt > > Phone: +49 6306 992140 > Fax: +49 6306 992142 > Mobile: +49 151 27530865 > email: carsten.delellis at delellis.net > > > From rs at sys4.de Wed Oct 24 19:09:12 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 18:09:12 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50880D7B.4090407@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> Message-ID: <508812A8.8000603@sys4.de> Am 24.10.2012 17:47, schrieb Bill Shirley: > > On 10/24/2012 2:33 AM, Robert Schetterer wrote: >> Am 24.10.2012 03:32, schrieb Bill Shirley: >>> What is your mailbox_command in main.cf? I just use: >>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >>> >>> I don't need anything in master.cf. But you should be using -u ${user} >>> for spamc. >> long time ago i tested this with dovecot lda postfix master.cf >> with a total virtual setup >> >> dovecot unix - n n - - pipe >> flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e >> /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} >> >> but i strongly do not recommand this !!! >> >> use spamass-milter, amavis etc with dovecot lmtp >> as described on many sites >> >> >> Best Regards >> MfG Robert Schetterer >> > > Can you get per-user Spamassassin configs this way? > > Why user=vmail:vmail? Is this for virtual domains? I didn't think we > were talking about them. > > Instead of strongly recommending against this, why not elaborate on the > problems with using spamc in the mailbox_command? > > Bill > Hi Bill, you missed my "i tested this with dovecot lda" in hope you may adapt the syntax to your needs by your own here are the recommanded setups http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix http://wiki.dovecot.org/LDA/Postfix --snip mailbox_command = /usr/bin/spamc -e /usr/lib/dovecot/deliver --snipend by the way using dovecot lmtp and i.e amavis or spamass-milter/clamav-milter might be better choice in many ways Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From troy at troyvit.com Wed Oct 24 19:10:38 2012 From: troy at troyvit.com (Troy Vitullo) Date: Wed, 24 Oct 2012 10:10:38 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5087454B.5030704@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> Message-ID: <20121024101038.5f3316f2@hrafn> On Tue, 23 Oct 2012 21:32:59 -0400 Bill Shirley wrote: > On 10/23/2012 9:06 PM, Bill Shirley wrote: > > > > > > What is your mailbox_command in main.cf? I just use: > > mailbox_command = /usr/bin/spamc -u "$USER" -e > > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m > > "$EXTENSION" > > > > I don't need anything in master.cf. But you should be using -u > > ${user} for spamc. > > > > Bill > > > Forgot to ask, are you using Spamassassin's per-user configs? If > you're not, that probably is your problem. It's probably trying to > update bayes tokens and it doesn't have permission. > > I use per-user configs which are nice. One man's spam is another > man's ham. Plus each user can have his/her own whitelist. > > I use these spamd args: -d -c -m10 --user-config > You usually can find the args in /etc/sysconfig. > > Bill Thanks for getting back to me Bill. Actually I'm using per-user prefs and permissions look great all the way down. When I send a test mail with everything turned on the bayes tokens are updated. Things appear to die later in the process. Regarding the mailbox command I was using: mailbox_command = /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" I tried removing the flags from master.cf and changing my command to: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" and then: mailbox_command = /usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} -m "$EXTENSION" and everything in between. No mail made it through, so I kept this in master.cf: dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/lib/dovecot/deliver -d ${recipient} and of course it over-rode my mailbox_command. Mail came thrrough but it contained no spamassassin header. I'm starting to thing that spamc doesn't have the permissions to write its headers to the message. How can I test that theory? spamd runs witht these flags: /usr/sbin/spamd --create-prefs -x --max-children 3 --username spamd --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid It's pretty much the same as yours, I just use the long versions of the args. the spamd user exists: spamd:x:1010:1011::/var/lib/spamassassin:/bin/false I was missing /etc/dovecot/default.sieve, which had to be a big problem, but I recovered it. Here's are its contents: require "fileinto"; if exists "X-Spam-Flag" { if header :contains "X-Spam-Flag" "NO" { } else { discard; stop; } } Anything else I could be missing? I even insanely running spamd as the root user: /usr/sbin/spamd --create-prefs -x --max-children 3 --username root --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid Thanks, Troy From rs at sys4.de Wed Oct 24 19:16:43 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 18:16:43 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087FEED.7060007@Media-Brokers.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> Message-ID: <5088146B.606@sys4.de> Am 24.10.2012 16:45, schrieb Charles Marcus: > On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Hmmm... I wonder would it be possible for dovecot to automatically lower > the 'niceness' for index rebuilds (on systems that support such) to > avoid causing such distress? > i think you missed Stans point ,looking for some hardware problems first Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From Bill at KnoxvilleChristian.org Wed Oct 24 19:28:48 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 12:28:48 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508812A8.8000603@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> Message-ID: <50881740.90207@KnoxvilleChristian.org> On 10/24/2012 12:09 PM, Robert Schetterer wrote: > Am 24.10.2012 17:47, schrieb Bill Shirley: >> On 10/24/2012 2:33 AM, Robert Schetterer wrote: >>> Am 24.10.2012 03:32, schrieb Bill Shirley: >>>> What is your mailbox_command in main.cf? I just use: >>>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >>>> >>>> I don't need anything in master.cf. But you should be using -u ${user} >>>> for spamc. >>> long time ago i tested this with dovecot lda postfix master.cf >>> with a total virtual setup >>> >>> dovecot unix - n n - - pipe >>> flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e >>> /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} >>> >>> but i strongly do not recommand this !!! >>> >>> use spamass-milter, amavis etc with dovecot lmtp >>> as described on many sites >>> >>> >>> Best Regards >>> MfG Robert Schetterer >>> >> Can you get per-user Spamassassin configs this way? >> >> Why user=vmail:vmail? Is this for virtual domains? I didn't think we >> were talking about them. >> >> Instead of strongly recommending against this, why not elaborate on the >> problems with using spamc in the mailbox_command? >> >> Bill >> > Hi Bill, you missed > > my > > "i tested this with dovecot lda" > in hope you may adapt the syntax to your needs by your own > > here are the recommanded setups > > http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix > http://wiki.dovecot.org/LDA/Postfix > > --snip > mailbox_command = /usr/bin/spamc -e /usr/lib/dovecot/deliver > --snipend > > by the way using dovecot lmtp and i.e amavis or spamass-milter/clamav-milter > > might be better choice in many ways > > > Best Regards > MfG Robert Schetterer > I'm saying I have a WORKING setup (local and virtual) where spamc runs and then uses dovecot deliver. spamd uses spamassassin per-user configs. master.cf has (caution, line wraps around in email): vdovecot unix - n n - 5 pipe flags=DRuh user=vmail:vmail argv=/usr/bin/spamc -p 784 -u ${recipient} -e /usr/lib64/dovecot/deliver -d ${user}@${domain} -a {recipient} -f ${sender} -n -m ${extension} main.cf has: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" virtual_transport = vdovecot vdovecot_destination_recipient_limit = 1 I don't understand why you strongly recommend against using the mailbox_command. Is there a security risk here? I've read all the howtos. There are many ways to setup a mail server. That's the beauty of postfix, spamassassin, dovecot, etc; you can make it do what you want. Yes, some setups are bad. I am not the original poster. Hope this clears things up, Bill From rob0 at gmx.co.uk Wed Oct 24 19:32:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 11:32:55 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121023145245.124dd362@hrafn> References: <20121023145245.124dd362@hrafn> Message-ID: <20121024163255.GI3672@harrier.slackbuilds.org> There seems to be much confusion in this thread. I might be able to help clear up some of it, but probably not all, because I agree with Robert about using amavisd-new for filtering and LMTP for delivery. On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > My server uses a system comprised of postfix, dovecot and dspam to > filter and deliver mail. > > Postfix used the following flags in calling spamc and dovecot: > > flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} > -e /usr/lib/dovecot/deliver -d ${recipient} This looks like you might be using pipe(8). If so, refer to the manual, and note that you are invoking this command as user "dovecot" and group "secmail". That is wrong use of the "dovecot" user. You probably should have made and used a dedicated "vmail" user. And according to your own post, q.v., the group "secmail" is definitely wrong. > after an upgrade from Debian lenny to squeeze we were able to get > everything working except spam filtering. Spamassassin is able to > judge whether the mail coming in is spam but everything stops > there. Automated or semi-automated upgrades are often a source of pain. > In mail.err I see: > > pamc[3608]: exec failed: Permission denied I guess that is spamc, and yes, of course. > spamc shows the same thing in syslog: > > exec failed: Permission denied > > postfix delays the email: > > postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > (system resource problem) > > Here are the permissions for deliver: > > -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver The pipe command is not executed as root. Nor is it invoked with the GID "dovecot". You specified group "secmail". Therefore the "other" permissions are what apply. "---" is no read, no write, no execute. > Here are the relevant groups: > > s1:~# grep dovecot /etc/group > secmail:x:119:postfix,spamd,dovecot This is not relevant. The process has EGID secmail, and the fact that dovecot is a member of secmail does not matter. Bottom line here: it seems that you misunderstood what the group permissions meant. > dovecot:x:111: > > here's the dovecot user: > s1:~# grep dovecot /etc/passwd > dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false > > here's dovecot -n: > > # 1.2.15: /etc/dovecot/dovecot.conf You upgraded -- to 1.2.15? Why? snip > Many thanks in advance for any advice you can give. Again, you should check on the wiki about the appropriate use of the "dovecot" user, and also read the wiki about virtual mailboxes. Fix that. Even if you make it work with permissions, you are breaking Dovecot's security model of privilege separation. The "dovecot" user is for Dovecot's internal use only, not for delivering mail and ownership of mailboxes. The poster who was talking about postconf(5) mailbox_command was bringing in a red herring. That is for local(8) delivery, and you evidently are using pipe(8). -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rob0 at gmx.co.uk Wed Oct 24 19:44:48 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 11:44:48 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50881740.90207@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> Message-ID: <20121024164448.GJ3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 12:28:48PM -0400, Bill Shirley wrote: > I don't understand why you strongly recommend against using the > mailbox_command. Is there a security risk here? One issue is that mailbox_command is only used for local(8) delivery. You brought that up for the OP, who is reporting a problem in trying to use pipe(8). mailbox_command is not relevant for pipe. That added more confusion to the issue at hand. I can't speak for Robert, but as I said in the other post I agree with him, so I will say why. You will get better overall performance with amavisd-new and LMTP, rather than invoking a command via pipe for every delivery. No, mailbox_command in itself is not a security risk, except insofar as you could DoS yourself with more deliveries at once than the system is able to handle. Some risk of DoS is present for any kind of content filtering, though. But amavisd-new after-queue reduces that risk. > I've read all the howtos. Eww. I have not. I have made extensive referral to the documentation, however, and that is what I recommend. Many thousands of people who are generating web content do not know much about email. You don't want to turn to them for advice about this! (FWIW, many of the howtos I have looked at are very bad.) > There are many ways to setup a mail server. That's the beauty of > postfix, spamassassin, dovecot, etc; you can make it do what you > want. Yes, some setups are bad. Yes and yes. > I am not the original poster. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From Bill at KnoxvilleChristian.org Wed Oct 24 20:13:42 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:13:42 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024101038.5f3316f2@hrafn> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <20121024101038.5f3316f2@hrafn> Message-ID: <508821C6.4010608@KnoxvilleChristian.org> On 10/24/2012 12:10 PM, Troy Vitullo wrote: > On Tue, 23 Oct 2012 21:32:59 -0400 > Bill Shirley wrote: > >> On 10/23/2012 9:06 PM, Bill Shirley wrote: >>> >>> What is your mailbox_command in main.cf? I just use: >>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m >>> "$EXTENSION" >>> >>> I don't need anything in master.cf. But you should be using -u >>> ${user} for spamc. >>> >>> Bill >>> >> Forgot to ask, are you using Spamassassin's per-user configs? If >> you're not, that probably is your problem. It's probably trying to >> update bayes tokens and it doesn't have permission. >> >> I use per-user configs which are nice. One man's spam is another >> man's ham. Plus each user can have his/her own whitelist. >> >> I use these spamd args: -d -c -m10 --user-config >> You usually can find the args in /etc/sysconfig. >> >> Bill > Thanks for getting back to me Bill. Actually I'm using per-user prefs and permissions look great all the way down. When I send a test mail with everything turned on the bayes tokens are updated. Things appear to die later in the process. > > Regarding the mailbox command I was using: > mailbox_command = /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" > > I tried removing the flags from master.cf and changing my command to: > mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" What was your setting for mailbox_transport (in main.cf) when you did this? mailbox_transport could be overriding mailbox_command. > > and then: > mailbox_command = /usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} -m "$EXTENSION" > > and everything in between. > > No mail made it through, so I kept this in master.cf: > > dovecot unix - n n - - pipe > flags=DRhu user=dovecot:dovecot argv=/usr/lib/dovecot/deliver -d ${recipient} Where are you calling spamc with this? > > and of course it over-rode my mailbox_command. Mail came thrrough but it contained no spamassassin header. > > I'm starting to thing that spamc doesn't have the permissions to write its headers to the message. How can I test that theory? > > spamd runs witht these flags: > /usr/sbin/spamd --create-prefs -x --max-children 3 --username spamd --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid > > It's pretty much the same as yours, I just use the long versions of the args. > > the spamd user exists: > spamd:x:1010:1011::/var/lib/spamassassin:/bin/false Your permissions on /var/lib/spamassassin are probably right, but check them and the subdirectories. > > I was missing /etc/dovecot/default.sieve, which had to be a big problem, but I recovered it. Here's are its contents: > > require "fileinto"; > if exists "X-Spam-Flag" { > if header :contains "X-Spam-Flag" "NO" { > } else { > discard; > stop; > } > } > > Anything else I could be missing? I even insanely running spamd as the root user: > > /usr/sbin/spamd --create-prefs -x --max-children 3 --username root --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid > > Thanks, > > Troy I have two instances of spamd running. One for local users and the other for virtual users (note the port here and in master.cf): [root at elmo includes]# ps aux | grep spamd root 2684 0.1 1.0 173760 88484 ? SN 03:30 0:34 spamd child root 23987 0.0 0.7 147524 61900 ? SNs Oct23 0:05 /usr/bin/spamd -d -c -m10 --user-config root 24004 0.0 0.7 147504 61844 ? SNs Oct23 0:05 /usr/bin/spamd -d -c -m5 -x --virtual-config-dir=/home/vmail/domains/%d/%l/.spamassassin -u vmail --port=784 -H vmail 24014 0.0 0.9 161204 75880 ? SN Oct23 0:05 spamd child vmail 24015 0.0 0.7 147504 59700 ? SN Oct23 0:00 spamd child root 25772 0.0 0.8 155020 69188 ? SN 12:07 0:00 spamd child root 28981 0.0 0.0 16688 940 pts/4 S+ 12:36 0:00 grep --color spamd My vmail user: [root at elmo includes]# grep vmail /etc/{group,passwd} /etc/group:vmail:x:399: /etc/passwd:vmail:x:399:399:Virtual Mail:/home/vmail:/bin/bash My virtual user .spamassassin permissions: [root at elmo includes]# ldp /home/vmail/domains/example.com/bill/.spamassassin drwxr-xr-x 20 root root 4096 May 8 2011 /home drwxr-xr-x 10 vmail vmail 4096 Oct 22 10:59 /home/vmail drwxr-x--- 9 vmail vmail 4096 Oct 21 21:24 /home/vmail/domains drwxr-x--- 6 vmail vmail 4096 Jul 4 2007 /home/vmail/domains/example.com drwxr-x--- 4 vmail vmail 4096 Jul 4 2007 /home/vmail/domains/example.com/bill drwxr-s--- 3 vmail vmail 4096 Jan 30 2012 /home/vmail/domains/example.com/bill/.spamassassin My local user: [root at elmo includes]# ldp /home/bill/.spamassassin drwxr-xr-x 20 root root 4096 May 8 2011 /home drwxr-xr-x 32 bill bill 4096 Oct 22 17:42 /home/bill drwxr-s--- 2 bill bill 4096 Oct 24 12:42 /home/bill/.spamassassin My main.cf: mailbox_transport = mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" virtual_transport = vdovecot vdovecot_destination_recipient_limit = 1 My master.cf: vdovecot unix - n n - 5 pipe flags=DRuh user=vmail:vmail argv=/usr/bin/spamc -p 784 -u ${recipient} -e /usr/lib64/dovecot/deliver -d ${user}@${domain} -a {recipient} -f ${sender} -n -m ${extension} You could try my config substituting your user and directory for mine: I'm using user=vmail:vmail and --virtual-config-dir=/home/vmail/domains/%d/%l/.spamassassin You're using user=dovecot:secmail and --virtual-config-dir=/var/lib/spamassassin/users/%d/%l Currently, your user=dovecot:secmail should probably be user=spamd:spamd in master.cf unless group secmail has write permissions on /var/lib/spamassassin and subdirectories. Hope this helps, Bill From Bill at KnoxvilleChristian.org Wed Oct 24 20:21:58 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:21:58 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024164448.GJ3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> Message-ID: <508823B6.3040208@KnoxvilleChristian.org> On 10/24/2012 12:44 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 12:28:48PM -0400, Bill Shirley wrote: >> I don't understand why you strongly recommend against using the >> mailbox_command. Is there a security risk here? > One issue is that mailbox_command is only used for local(8) delivery. > You brought that up for the OP, who is reporting a problem in trying > to use pipe(8). mailbox_command is not relevant for pipe. That added > more confusion to the issue at hand. It was my understanding that he is implementing local users. > > I can't speak for Robert, but as I said in the other post I agree > with him, so I will say why. You will get better overall performance > with amavisd-new and LMTP, rather than invoking a command via pipe > for every delivery. Admittedly, I have not used amavisd-new or LMTP; they may be better. But will they allow spamassassin per-user prefs? Performance is a plus; another daemon is not. That saying, I'll run another daemon if I get something out of it. Any benchmarks on this? > > No, mailbox_command in itself is not a security risk, except insofar > as you could DoS yourself with more deliveries at once than the > system is able to handle. Some risk of DoS is present for any kind of > content filtering, though. But amavisd-new after-queue reduces that > risk. > >> I've read all the howtos. > Eww. I have not. I have made extensive referral to the documentation, > however, and that is what I recommend. Many thousands of people who > are generating web content do not know much about email. You don't > want to turn to them for advice about this! Probably mis-spoke; I said howtos instead of documentation. Yes, there are many bad howtos out there. > > (FWIW, many of the howtos I have looked at are very bad.) > >> There are many ways to setup a mail server. That's the beauty of >> postfix, spamassassin, dovecot, etc; you can make it do what you >> want. Yes, some setups are bad. > Yes and yes. > >> I am not the original poster. Respectfully, Bill From Bill at KnoxvilleChristian.org Wed Oct 24 20:28:41 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:28:41 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024163255.GI3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> Message-ID: <50882549.3020505@KnoxvilleChristian.org> On 10/24/2012 12:32 PM, /dev/rob0 wrote: > There seems to be much confusion in this thread. I might be able to > help clear up some of it, but probably not all, because I agree with > Robert about using amavisd-new for filtering and LMTP for delivery. > > On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: >> My server uses a system comprised of postfix, dovecot and dspam to >> filter and deliver mail. >> >> Postfix used the following flags in calling spamc and dovecot: >> >> flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} >> -e /usr/lib/dovecot/deliver -d ${recipient} > This looks like you might be using pipe(8). If so, refer to the > manual, and note that you are invoking this command as user "dovecot" > and group "secmail". > > That is wrong use of the "dovecot" user. You probably should have > made and used a dedicated "vmail" user. And according to your own > post, q.v., the group "secmail" is definitely wrong. > >> after an upgrade from Debian lenny to squeeze we were able to get >> everything working except spam filtering. Spamassassin is able to >> judge whether the mail coming in is spam but everything stops >> there. > Automated or semi-automated upgrades are often a source of pain. > >> In mail.err I see: >> >> pamc[3608]: exec failed: Permission denied > I guess that is spamc, and yes, of course. > >> spamc shows the same thing in syslog: >> >> exec failed: Permission denied >> >> postfix delays the email: >> >> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >> (system resource problem) >> >> Here are the permissions for deliver: >> >> -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver > The pipe command is not executed as root. Nor is it invoked with the > GID "dovecot". You specified group "secmail". Therefore the "other" > permissions are what apply. "---" is no read, no write, no execute. > >> Here are the relevant groups: >> >> s1:~# grep dovecot /etc/group >> secmail:x:119:postfix,spamd,dovecot > This is not relevant. The process has EGID secmail, and the fact that > dovecot is a member of secmail does not matter. Bottom line here: it > seems that you misunderstood what the group permissions meant. > >> dovecot:x:111: >> >> here's the dovecot user: >> s1:~# grep dovecot /etc/passwd >> dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false >> >> here's dovecot -n: >> >> # 1.2.15: /etc/dovecot/dovecot.conf > You upgraded -- to 1.2.15? Why? > > snip >> Many thanks in advance for any advice you can give. > Again, you should check on the wiki about the appropriate use of the > "dovecot" user, and also read the wiki about virtual mailboxes. Fix > that. Even if you make it work with permissions, you are breaking > Dovecot's security model of privilege separation. The "dovecot" user > is for Dovecot's internal use only, not for delivering mail and > ownership of mailboxes. > > The poster who was talking about postconf(5) mailbox_command was > bringing in a red herring. That is for local(8) delivery, and you > evidently are using pipe(8). Just a note: the original post did NOT have the word 'virtual' in it. If it did, I missed it and apologize for introducing confusion. Bill From rs at sys4.de Wed Oct 24 20:37:35 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 19:37:35 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50881740.90207@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> Message-ID: <5088275F.1030507@sys4.de> Am 24.10.2012 18:28, schrieb Bill Shirley: > I don't understand why you strongly recommend against using the > mailbox_command. Is there a security risk here? no ,until you dont have made any setup failures... your right there are tons of working possible setups your free to configure as you like, but lmtp with dovecot is state of the art in my eyes, these days in my tests lda combined with spamc had not enough performance for my needs and used to much resources compared to lmtp sometimes it crashed, but as i said ,long time ago however i found total virtual setups much more easy then with local by permissions stuff etc, and milters are much more easy to use and setup, also i.e amavis gives great other choices beside spamassassin stuff but do as you like ,no need to flame Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From rob0 at gmx.co.uk Wed Oct 24 20:39:18 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 12:39:18 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50882549.3020505@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> Message-ID: <20121024173918.GK3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: > On 10/24/2012 12:32 PM, /dev/rob0 wrote: > >There seems to be much confusion in this thread. I might be able > >able to help clear up some of it, but probably not all, because I > >agree with Robert about using amavisd-new for filtering and LMTP > >for delivery. > > > >On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: snip > >>postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > >>delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > >>(system resource problem) > >The poster who was talking about postconf(5) mailbox_command > >was bringing in a red herring. That is for local(8) delivery, > >and you evidently are using pipe(8). > Just a note: the original post did NOT have the word 'virtual' in > it. If it did, I missed it and apologize for introducing confusion. It did not, but it did indeed include the pipe log output shown above, and therefore ^mailbox_.* postconf settings do not apply. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From jeff at bubble.org Wed Oct 24 20:40:25 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Wed, 24 Oct 2012 13:40:25 -0400 Subject: [Dovecot] Snarf plugin Message-ID: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> I've now upgraded dovecot from 2.0.21 to 2.1.10 and the good news is I no longer see dovecot crashing when loading the snarf plugin however snarf still does not do anything except make the inbox disappear. I've come to the conclusion that either snarf does not actually work, possible, but I doubt it, or more likely I have a configuration issue preventing it from working. The system is simple, all email is stored in /var/spool/mail/{username} and I want all the mail moved to ~/mbox when the user logs in via imap, similar to uw-imap. Any guidance would really be appreciated. Thanks, Jeff dovecot -n # 2.1.10: //etc/dovecot/dovecot.conf # OS: Linux 3.6.2-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mbox mail_plugins = snarf zlib mbox_write_locks = fcntl namespace default { inbox = yes location = prefix = separator = / } namespace snarf { hidden = yes list = no location = mbox:/run/dovecot/empty:INBOX=/var/spool/mail/%u:INDEX=MEMORY prefix = /snarf separator = / } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mbox_snarf = ~/mbox snarf = /snarf/INBOX } service imap-login { inet_listener imap { address = localhost } } service pop3-login { inet_listener pop3 { address = localhost } } ssl_cert = , method=PLAIN, rip=::1, lip=::1, mpid=28089, secured, session= Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff:INDEX=MEMORY Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Disconnected: Logged out in=117 out=1504 From rob0 at gmx.co.uk Wed Oct 24 20:49:03 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 12:49:03 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508823B6.3040208@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> Message-ID: <20121024174903.GL3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 01:21:58PM -0400, Bill Shirley wrote: > On 10/24/2012 12:44 PM, /dev/rob0 wrote: > >I can't speak for Robert, but as I said in the other post I > >agree with him, so I will say why. You will get better overall > >performance with amavisd-new and LMTP, rather than invoking a > >command via pipe for every delivery. > Admittedly, I have not used amavisd-new or LMTP; they may be > better. But will they allow spamassassin per-user prefs? Amavisd-new is indeed capable of per-user preferences. > Performance is a plus; another daemon is not. That saying, I'll > run another daemon if I get something out of it. Any benchmarks > on this? A daemon is generally (I'd almost daresay "always") less overhead than the invocation of many single-delivery processes. No benchmarking is needed to support this fact. That said, for many small sites, it does not matter much. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From Bill at KnoxvilleChristian.org Wed Oct 24 20:56:18 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:56:18 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5088275F.1030507@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <5088275F.1030507@sys4.de> Message-ID: <50882BC2.2010702@KnoxvilleChristian.org> On 10/24/2012 1:37 PM, Robert Schetterer wrote: > Am 24.10.2012 18:28, schrieb Bill Shirley: >> I don't understand why you strongly recommend against using the >> mailbox_command. Is there a security risk here? > no ,until you dont have made any setup failures... > > your right there are tons of working possible setups > your free to configure as you like, but lmtp with dovecot is state of > the art in my eyes, these days > > in my tests lda combined with spamc had not enough > performance for my needs and used to much resources compared to lmtp > sometimes it crashed, but as i said ,long time ago > > however i found total virtual setups much more easy then with local > by permissions stuff etc, and milters are much more easy to use and > setup, also i.e amavis gives great other choices beside spamassassin stuff > > but do as you like ,no need to flame > > Best Regards > MfG Robert Schetterer > I don't see a flame anywhere in my posts. The list is for respectfully exchanging information. I thought that was what we were doing. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 21:04:39 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 14:04:39 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024173918.GK3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> Message-ID: <50882DB7.5030202@KnoxvilleChristian.org> On 10/24/2012 1:39 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: >> On 10/24/2012 12:32 PM, /dev/rob0 wrote: >>> There seems to be much confusion in this thread. I might be able >>> able to help clear up some of it, but probably not all, because I >>> agree with Robert about using amavisd-new for filtering and LMTP >>> for delivery. >>> >>> On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > snip >>>> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >>>> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >>>> (system resource problem) >>> The poster who was talking about postconf(5) mailbox_command >>> was bringing in a red herring. That is for local(8) delivery, >>> and you evidently are using pipe(8). >> Just a note: the original post did NOT have the word 'virtual' in >> it. If it did, I missed it and apologize for introducing confusion. > It did not, but it did indeed include the pipe log output shown > above, and therefore ^mailbox_.* postconf settings do not apply. Could be he was going about it the wrong way; mixing the two. Do you know whether he's trying to do virtual or local? My postings describe my implementation. I'm just trying to help him. But I don't think my posts are being received that way. Bill From carsten.delellis at delellis.net Wed Oct 24 21:22:14 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Wed, 24 Oct 2012 20:22:14 +0200 Subject: [Dovecot] dovecot auth against AD on samba4 In-Reply-To: <1351094679.2143.474.camel@worklian> References: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> <1351094679.2143.474.camel@worklian> Message-ID: <296201cdb214$7ef15e50$7cd41af0$@delellis.net> Dear Bob Thankx for your hint. I tried with jxplorer to connect to the AD ldap and I am pretty sure that I will get it up and running like I did with the openldap server. Carsten -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Bob Miller Sent: Mittwoch, 24. Oktober 2012 18:05 To: dovecot at dovecot.org Subject: Re: [Dovecot] dovecot auth against AD on samba4 I don't have it in production yet because there are other things I am still trying to add to samba4, but my test server has dovecot authenticating against samba4. Without openchange or any other non-native mechanism. Dovecot supports authenticating against ldap, the settings are in your auth-ldap.conf file. Samba4/Active Directory is just another ldap implementation. between the config files and the wiki, I believe all the documentation you need is there... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Wed, 2012-10-24 at 11:48 +0200, Carsten Laun-De Lellis wrote: > Hi group > > I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, > dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and > I am thinking about installing it because of it's native outlook support > capabilities. > > The ZEG appliance wouldn't be an option for me because I use a virtual > server from a provider where I can't install my own vm or even an iso. > > When I go thru the documentation there is a part with installing > OpenChange based on samba4. As far as I understood the OpenChange > authentication is against the samba4 AD. Actually there is no support in > syncing the AD against an OpenLdap Server and I would have to change the > OpenLdap port because the AD is listening on port 389. To change the port > wouldn't be a big deal, but what i was thinking about to run the dovecot > auth also against the samba 4 AD. > > I searched around on the internet but didn't find a doc yet how to do that. > > Does anyone here could provide me with a link or a how-to ? > > Thanks very much in advance. > > Regards, > > Carsten Laun-De Lellis > > Hauptstrasse 13 > D-67705 Trippstadt > > Phone: +49 6306 992140 > Fax: +49 6306 992142 > Mobile: +49 151 27530865 > email: carsten.delellis at delellis.net > > > From rs at sys4.de Wed Oct 24 21:24:31 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 20:24:31 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508823B6.3040208@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> Message-ID: <5088325F.7020102@sys4.de> Am 24.10.2012 19:21, schrieb Bill Shirley: > Admittedly, I have not used amavisd-new or LMTP; they may be better. > But will they allow spamassassin per-user prefs? Performance is a plus; > another daemon is not. That saying, I'll run another daemon if I get > something out of it. Any benchmarks on this? this went away from the orig post, it went to general design of a email system, i think rob did explain the possible problems to the orginal poster very fine some people may start with local users as traditional mailsetup depend on this next steps they are going to use lda perhaps trying combined with spamc with local users so there is nothing bad on it, its somehow old school, after all, as said ,there are many broken advices out in www by all setups, and sometimes there are mixed up by local and virtual, so people may fail with permissions of local users , daemons etc sometimes later if more domains should be hosted pure virtual setups are the better way, and making stuff more simple ( but often people fail first in seeing virtual more easy ), lmtp is the best choice for it compared starting a deliver process for each mail, its working as a service So anyone should think about what he needs before starting to setup i.e amavis is a well supported framework since long time, it has tons of features you might wanna have and as well it can be used with per-user prefs if you dont like the complex amavis style ( many functions have many config points ), you could simple use a chain of milter i.e spamass-milter ( also with per-user prefs ), clamav-milter with milter you are able to reject on smtp income stage which is very cool anyway milters also have their pros an contras, read postfix sites about them i didnt tested dspam looks like it chained between lmtp so perhaps also good choice, and could be combined with milters i had other setups with chained spampd/clamsmtp amavis on seperate filter hosts etc all worked fine but as dovecot/postfix development going forward , i redesigned all these depending to have more functions and performance so i recommand, use your working setups as i.e lifetime of your hardware etc, but if building new mailserver choose modern setup ideas and daemon combinations Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From rob0 at gmx.co.uk Wed Oct 24 21:25:52 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 13:25:52 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50882DB7.5030202@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> Message-ID: <20121024182552.GM3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 02:04:39PM -0400, Bill Shirley wrote: > On 10/24/2012 1:39 PM, /dev/rob0 wrote: > >On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: > >>On 10/24/2012 12:32 PM, /dev/rob0 wrote: > >>>On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > >snip > >>>>postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > >>>>delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > >>>>(system resource problem) > >>>The poster who was talking about postconf(5) mailbox_command > >>>was bringing in a red herring. That is for local(8) delivery, > >>>and you evidently are using pipe(8). > >>Just a note: the original post did NOT have the word 'virtual' > >>in it. If it did, I missed it and apologize for introducing > >>confusion. > >It did not, but it did indeed include the pipe log output shown > >above, and therefore ^mailbox_.* postconf settings do not apply. > > Could be he was going about it the wrong way; mixing the two. > Do you know whether he's trying to do virtual or local? There are lots of wrong ways. The most wrongful of the OP's ways I found was the misuse of the dovecot user. The second most wrong, which was the actual problem at hand, was a misunderstanding of how group permissions are applied. Mixing virtual and local in Postfix and Dovecot is no problem at all, and in fact multiple modes of delivery are possible, even within a given address class or even within a domain. All we know here is what the OP posted. You don't usually use pipe for delivery to local (Unix) users. > My postings describe my implementation. For the OP to change to local delivery would require reworking his setup extensively, on the Postfix side, and here we are on the Dovecot list, so I wouldn't go into that here. But sure, there are other (and for many purposes, better) means of doing what he might want to do. > I'm just trying to help him. But I don't think my posts are > being received that way. Regarding Robert's "flame" comment in the other subthread, I agree with you; I saw no flame. And I did not suggest that you were not trying to help. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rs at sys4.de Wed Oct 24 21:32:19 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 20:32:19 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024182552.GM3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> Message-ID: <50883433.8010609@sys4.de> Am 24.10.2012 20:25, schrieb /dev/rob0: > Regarding Robert's "flame" comment in the other subthread, I agree > with you; I saw no flame. And I did not suggest that you were not > trying to help take my sorry, as non native english, perhaps i missused "flame" here Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From CMarcus at Media-Brokers.com Wed Oct 24 21:48:57 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 24 Oct 2012 14:48:57 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5088146B.606@sys4.de> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> <5088146B.606@sys4.de> Message-ID: <50883819.7010005@Media-Brokers.com> On 2012-10-24 12:16 PM, Robert Schetterer wrote: > Am 24.10.2012 16:45, schrieb Charles Marcus: >> On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >>> If the box is truly unresponsive, i.e. hard locked, then the corrupted >>> indexes are only a symptom of the underlying problem, which is unrelated >>> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >>> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >>> You need to know the difference so we have accurate information to >>> troubleshoot with. >> Hmmm... I wonder would it be possible for dovecot to automatically lower >> the 'niceness' for index rebuilds (on systems that support such) to >> avoid causing such distress? > i think you missed Stans point ,looking for some hardware problems first No, I was simply commenting on the one point about heavy load during large index rebuilds - which is why I trimmed the quoted text... maybe I could have trimmed more? -- Best regards, Charles From Bill at KnoxvilleChristian.org Wed Oct 24 21:51:12 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 14:51:12 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024182552.GM3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> Message-ID: <508838A0.6090100@KnoxvilleChristian.org> On 10/24/2012 2:25 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 02:04:39PM -0400, Bill Shirley wrote: >> On 10/24/2012 1:39 PM, /dev/rob0 wrote: >>> On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: >>>> On 10/24/2012 12:32 PM, /dev/rob0 wrote: >>>>> On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: >>> snip >>>>>> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >>>>>> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >>>>>> (system resource problem) >>>>> The poster who was talking about postconf(5) mailbox_command >>>>> was bringing in a red herring. That is for local(8) delivery, >>>>> and you evidently are using pipe(8). >>>> Just a note: the original post did NOT have the word 'virtual' >>>> in it. If it did, I missed it and apologize for introducing >>>> confusion. >>> It did not, but it did indeed include the pipe log output shown >>> above, and therefore ^mailbox_.* postconf settings do not apply. >> Could be he was going about it the wrong way; mixing the two. >> Do you know whether he's trying to do virtual or local? > There are lots of wrong ways. The most wrongful of the OP's ways I > found was the misuse of the dovecot user. The second most wrong, > which was the actual problem at hand, was a misunderstanding of how > group permissions are applied. > > Mixing virtual and local in Postfix and Dovecot is no problem at all, > and in fact multiple modes of delivery are possible, even within a > given address class or even within a domain. > > All we know here is what the OP posted. You don't usually use pipe > for delivery to local (Unix) users. > >> My postings describe my implementation. > For the OP to change to local delivery would require reworking his > setup extensively, on the Postfix side, and here we are on the > Dovecot list, so I wouldn't go into that here. But sure, there are > other (and for many purposes, better) means of doing what he might > want to do. > >> I'm just trying to help him. But I don't think my posts are >> being received that way. > Regarding Robert's "flame" comment in the other subthread, I agree > with you; I saw no flame. And I did not suggest that you were not > trying to help. Thank you for saying this. My intent was to help. I make my living setting up/programming with open source software. I don't want to only 'take'. I want to show my gratitude for is so freely given to me by also giving. I don't program in C so I can't help with that. But I can share configurations/experiences and hopefully that is a contribution. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 23:04:20 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 16:04:20 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5088325F.7020102@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> <5088325F.7020102@sys4.de> Message-ID: <508849C4.9060800@KnoxvilleChristian.org> On 10/24/2012 2:24 PM, Robert Schetterer wrote: > Am 24.10.2012 19:21, schrieb Bill Shirley: >> Admittedly, I have not used amavisd-new or LMTP; they may be better. >> But will they allow spamassassin per-user prefs? Performance is a plus; >> another daemon is not. That saying, I'll run another daemon if I get >> something out of it. Any benchmarks on this? > this went away from the orig post, it went to general design > of a email system, i think rob did explain the possible problems > to the orginal poster very fine > > some people may start with local users as traditional > mailsetup depend on this next steps they are going to use lda > perhaps trying combined with spamc with local users > so there is nothing bad on it, its somehow old school, > after all, as said ,there are many broken advices out in www by all > setups, and sometimes there are mixed up by local and virtual, so people > may fail with permissions of local users , daemons etc > > sometimes later if more domains should be hosted > pure virtual setups are the better way, and making stuff more simple ( > but often people fail first in seeing virtual more easy ), > > lmtp is the best choice for it compared starting a deliver process for > each mail, its working as a service > > So anyone should think about what he needs before starting to setup > > i.e amavis is a well supported framework since long time, it has tons of > features > you might wanna have and as well it can be used with per-user prefs > > if you dont like the complex amavis style ( many functions have many > config points ), you could simple use a chain of milter i.e > spamass-milter ( also with per-user prefs ), clamav-milter > > with milter you are able to reject on smtp income stage > which is very cool > anyway milters also have their pros an contras, read postfix sites about > them > > i didnt tested dspam looks like it chained between lmtp > so perhaps also good choice, and could be combined with milters > > i had other setups with chained spampd/clamsmtp > amavis on seperate filter hosts etc > all worked fine > > but as dovecot/postfix development going forward , i redesigned all > these depending to have more functions and performance > > so i recommand, use your working setups as i.e lifetime of your hardware > etc, but if building new mailserver choose modern setup ideas > and daemon combinations > > > Best Regards > MfG Robert Schetterer > Thank you for a very informative post. I took a quick look at spamass-milter but I can't find any configuration information on how to use spamasssassin's per-user prefs. I thought the only way to support per-user prefs was post queue since you have to know who is getting the email to check their prefs. I am using clamav-milter. Milters are nice. I set my mail server up 15+ years ago, so it's time for me to have a re-think here. At that time there were no milters for postfix (don't remember a Dovecot either). I've try to steer away from re-injects since they affect the mail received numbers. Are we saying Dovecot's LMTP can call spamd? I'm on Dovecot 1.2 at home until I can upgrade. There is no LMTP in Dovecot 1.x, right? I have a few mail servers running Dovecot 2.0 and 2.1 and yes, I want them to perform well. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 23:06:01 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 16:06:01 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50883433.8010609@sys4.de> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> <50883433.8010609@sys4.de> Message-ID: <50884A29.1020901@KnoxvilleChristian.org> On 10/24/2012 2:32 PM, Robert Schetterer wrote: > Am 24.10.2012 20:25, schrieb /dev/rob0: >> Regarding Robert's "flame" comment in the other subthread, I agree >> with you; I saw no flame. And I did not suggest that you were not >> trying to help > take my sorry, as non native english, perhaps i missused "flame" here > > > > Best Regards > MfG Robert Schetterer > No problem. You do very well at speaking English. Bill From roundcube222 at alaadin.org Wed Oct 24 23:04:31 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 24 Oct 2012 23:04:31 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot Message-ID: <4c9608dd97036a48885e68205922d6b2@Coptics.org> Hello, I have a question regarding mailbox locking and hope any one can help me to better understanding the locking of mbox My Postfix lock option is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is fcntl only mbox_write_locks = fcntl mbox_read_locks = fcntl now, when user is download a large mail (20 MB) for example, dovecot locks /var/mail/user with fcntl until the users finish downloading the mail ok here comes my question. While the user is downloading the mail , a mail arrives so postfix make some thing weird 1- Postfix creates /var/mail/user.lock for 5 secs , then postfix defer the mail with reason that the /var/mail/user is locked and then delete the /var/mail/user.lock (after 5 secs) My question is ? why postfix create user.lock although it shouldn't because already dovecot fcntl it ? 1- Dovecot locks /var/mail/user using FCNTL 2- Posttix at the same time tries to FCNTL /var/mail/user .. but it fail since dovecot already fcnl it. 3- Postfix at the same time add dot lock /var/mail/user for 5 secs then remove the lock. the question is how come postfix dot lock /var/mail although it couldn't FCNTL the file in the first place ??????? what i was expecting is 1- Dovecot locks /var/mail/user using FCNTL 2- Postfix tries to FCNTL /var/mail/user 3- POSTFIX WILL NOT CREATE DOTLOCK file unless the FCNTL is released by dovecot!!!! Please advise if postfix will dot lock the file even if it couldn't FCNTL the file in the first place?? Regards From mailadmin at cubixys.com Thu Oct 25 00:07:24 2012 From: mailadmin at cubixys.com (Fasil) Date: Thu, 25 Oct 2012 00:07:24 +0300 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <507F65C5.3090803@brightok.net> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> <507F3C67.5020900@cubixys.com> <507F65C5.3090803@brightok.net> Message-ID: <5088588C.2060005@cubixys.com> Thanks for the input Jack. As I am using debian, the location to edit the ulimit is /etc/default/dovecot. There is no effect even after changing this value. I tried putting the value in the /etc/init.d/dovecot script without success. Fasil. On 10/18/2012 05:13 AM, Jack Bates wrote: > I'm using RHEL6 instead of ubuntu, but check the startup scripts. In > RHEL's case, the following file is sourced, so I updated it instead of > the startup scripts. > > cat /etc/sysconfig/dovecot > # Here you can specify your dovecot command line options. > # > #OPTIONS="" > ulimit -n 4096 > ulimit -u 5120 > > In addition, I had to also up the max allowed processes in the dovecot > config. 2.x and 1.x are different on this. > > http://wiki1.dovecot.org/LoginProcess <-1.x method > > > Jack > > > > On 10/17/2012 6:16 PM, Fasil wrote: >> Thanks Frank. >> Followed the URL and could not find any luck. >> >> Is there a way to change the value of 'max open files' of dovecot. >> I have tried to set the value in /etc/default/dovecot by setting >> ulimit. But the value is not getting applied. >> Could anyone help on this regard. >> >> Fasil. >> >> On 08/16/2012 09:17 AM, Frank Bonnet wrote: >>> hello >>> >>> here some useful informations >>> >>> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ >>> >>> Envoy? de mon iPhone. >>> >>> >>> Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : >>> >>>> Dear all, >>>> >>>> Thank you all for such a wonderful support. Hats off to all :) >>>> >>>> Few times I came across imap login issues where new users will not >>>> be allowed to login. >>>> /var/log/mail.err shows the error below >>>> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in >>>> creating login processes, slowing down for now >>>> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> >>>> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >>>> >>>> # ulimit -Hn >>>> 1024 >>>> >>>> # ulimit -Sn >>>> 1024 >>>> >>>> # cat /proc/sys/fs/file-max >>>> 1238548 >>>> >>>> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >>>> Max open files 1024 1024 files >>>> >>>> Please advice how to get rid off this. >>>> >>>> Fasil. >> >> > From troy at troyvit.com Thu Oct 25 00:34:18 2012 From: troy at troyvit.com (Troy Vitullo) Date: Wed, 24 Oct 2012 15:34:18 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024163255.GI3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> Message-ID: <20121024153418.7a183681@hrafn> On Wed, 24 Oct 2012 11:32:55 -0500 /dev/rob0 wrote: > There seems to be much confusion in this thread. I might be able to > help clear up some of it, but probably not all, because I agree with > Robert about using amavisd-new for filtering and LMTP for delivery. > Thanks for the reality check Rob. I'm circling back with the guy who originally set this up to see if we can get back on the right track. We are using pipe with virtual users, and dovecot doesn't own the mailboxes. If/when we get our collective act together and have more questions I'll respond in more detail. Thanks again, Troy From rs at sys4.de Thu Oct 25 00:40:36 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 23:40:36 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508849C4.9060800@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> <5088325F.7020102@sys4.de> <508849C4.9060800@KnoxvilleChristian.org> Message-ID: <50886054.9090503@sys4.de> Am 24.10.2012 22:04, schrieb Bill Shirley: > > On 10/24/2012 2:24 PM, Robert Schetterer wrote: >> Am 24.10.2012 19:21, schrieb Bill Shirley: >>> Admittedly, I have not used amavisd-new or LMTP; they may be better. >>> But will they allow spamassassin per-user prefs? Performance is a plus; >>> another daemon is not. That saying, I'll run another daemon if I get >>> something out of it. Any benchmarks on this? >> this went away from the orig post, it went to general design >> of a email system, i think rob did explain the possible problems >> to the orginal poster very fine >> >> some people may start with local users as traditional >> mailsetup depend on this next steps they are going to use lda >> perhaps trying combined with spamc with local users >> so there is nothing bad on it, its somehow old school, >> after all, as said ,there are many broken advices out in www by all >> setups, and sometimes there are mixed up by local and virtual, so people >> may fail with permissions of local users , daemons etc >> >> sometimes later if more domains should be hosted >> pure virtual setups are the better way, and making stuff more simple ( >> but often people fail first in seeing virtual more easy ), >> >> lmtp is the best choice for it compared starting a deliver process for >> each mail, its working as a service >> >> So anyone should think about what he needs before starting to setup >> >> i.e amavis is a well supported framework since long time, it has tons of >> features >> you might wanna have and as well it can be used with per-user prefs >> >> if you dont like the complex amavis style ( many functions have many >> config points ), you could simple use a chain of milter i.e >> spamass-milter ( also with per-user prefs ), clamav-milter >> >> with milter you are able to reject on smtp income stage >> which is very cool >> anyway milters also have their pros an contras, read postfix sites about >> them >> >> i didnt tested dspam looks like it chained between lmtp >> so perhaps also good choice, and could be combined with milters >> >> i had other setups with chained spampd/clamsmtp >> amavis on seperate filter hosts etc >> all worked fine >> >> but as dovecot/postfix development going forward , i redesigned all >> these depending to have more functions and performance >> >> so i recommand, use your working setups as i.e lifetime of your hardware >> etc, but if building new mailserver choose modern setup ideas >> and daemon combinations >> >> >> Best Regards >> MfG Robert Schetterer >> > Thank you for a very informative post. I took a quick look at > spamass-milter but I can't find any configuration information on how to > use spamasssassin's per-user prefs. I thought the only way to support > per-user prefs was post queue since you have to know who is getting the > email to check their prefs. you have to study its parameters ( they may differ by version and distro ) http://linux.die.net/man/1/spamass-milter i use it like /usr/sbin/spamass-milter -P /var/spool/postfix/spamass-milter/spamass.pid -f -p /var/spool/postfix/spamass/spamass.sock -f -e -x -I -u vmail -r 15 -i 127.0.0.1 i have my spamassassin setup with mysql for users self settings use i.e.e webmail horde with sam module, or something equal with i.e squirrelmail or roundcube but i managed it before ,also in using local files with maildrop as i said ,its not ideal cause of pre queue design, but reality shows good enough for big isp setup and it may be combined > > I am using clamav-milter. Milters are nice. for antispam using sanesecurity antispam signatures are nice thats "cheaper" then spamassassin > > I set my mail server up 15+ years ago, so it's time for me to have a > re-think here. At that time there were no milters for postfix (don't > remember a Dovecot either). I've try to steer away from re-injects > since they affect the mail received numbers. Are we saying Dovecot's > LMTP can call spamd? i dont tested ,looks like dspam can do it http://wiki2.dovecot.org/HowTo/Virtual%2BPostfix%2BDspam%2BDovecot I'm on Dovecot 1.2 at home until I can upgrade. > There is no LMTP in Dovecot 1.x, right? yes ,you should use 2.1.x > > I have a few mail servers running Dovecot 2.0 and 2.1 and yes, I want > them to perform well. so you may change setup layout > > Bill > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stocton12 at yahoo.com Thu Oct 25 01:13:14 2012 From: stocton12 at yahoo.com (b m) Date: Wed, 24 Oct 2012 15:13:14 -0700 (PDT) Subject: [Dovecot] Public folders and groups Message-ID: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? From stan at hardwarefreak.com Thu Oct 25 06:48:33 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 22:48:33 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087FEED.7060007@Media-Brokers.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> Message-ID: <5088B691.7030100@hardwarefreak.com> On 10/24/2012 9:45 AM, Charles Marcus wrote: > On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Hmmm... I wonder would it be possible for dovecot to automatically lower > the 'niceness' for index rebuilds (on systems that support such) to > avoid causing such distress? Changing the process priority would not help. Indexing a large mailbox is an IO bound, not a compute bound, operation. With Linux, changing from the CFQ to deadline scheduler may help some with low responsiveness. But the only real solution for such a case where iowait is bringing the system to its knees is to acquire storage with far greater IOPS and concurrent IO capability. I.e. a server. -- Stan From stan at hardwarefreak.com Thu Oct 25 06:57:47 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 22:57:47 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <4c9608dd97036a48885e68205922d6b2@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> Message-ID: <5088B8BB.9000109@hardwarefreak.com> On 10/24/2012 3:04 PM, Robert JR wrote: > I have a question regarding mailbox locking and hope any one can help me > to better understanding the locking of mbox > > My Postfix lock option is fcntl dotlock > mailbox_delivery_lock = fcntl, dotlock > virtual_mailbox_lock = fcntl, dotlock > > My Dovecot lock option is fcntl only > mbox_write_locks = fcntl > mbox_read_locks = fcntl Postfix is delivering the mail to dovecot. This is done via the deliver program or lmtp which are pipes, not files. Thus, why is Postfix attempting to write files in the user's mail directory? You write new mail to the mailbox file with either Dovecot or Postfix, not both. Fix that problem and the locking problem disappears. -- Stan From eliezer at ngtech.co.il Thu Oct 25 10:16:28 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 25 Oct 2012 09:16:28 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086F2BB.7010704@rename-it.nl> References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> Message-ID: <5088E74C.9030006@ngtech.co.il> On 10/23/2012 9:40 PM, Stephan Bosch wrote: > Also, could you provide your full configuration as output from `dovecot > -n` ? > > Regards, > > Stephan. > Thanks Stephan, I just upgraded from 2.0.17 to 2.1.9 and ph 0.3.1 (gentoo) and it seems to work as expected and dont leave any traces in the INBOX with the same script. I dont know the old ph version and since it was resolved i'm ok with it. My only problem is that it will put the file in the folder but will not mark the folder with the new file until I actually check the folder manually. it's not that much hustle but if there is a way to solve it I will be more then happy to hear about it. Thanks, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From roundcube222 at alaadin.org Thu Oct 25 10:23:29 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 10:23:29 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <5088B8BB.9000109@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: On 2012-10-25 06:57, Stan Hoeppner wrote: > On 10/24/2012 3:04 PM, Robert JR wrote: > >> I have a question regarding mailbox locking and hope any one can >> help >> me to better understanding the locking of mbox My Postfix lock >> option >> is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock >> virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is >> fcntl >> only mbox_write_locks = fcntl mbox_read_locks = fcntl > > > > Postfix is delivering the mail to dovecot. This is done via the > deliver > program or lmtp which are pipes, not files. Thus, why is Postfix > attempting to write files in the user's mail directory? > > You write new mail to the mailbox file with either Dovecot or > Postfix, > not both. Fix that problem and the locking problem disappears. Stan, sorry but you didnot understand my question at all, dovecot in this case is reading the mailbox file while user downloading the mail and not WRITING. only postfix write when a mail arrives and DOVECOT only read the mail. And even if both write to the file, I have already set the locking option of both to FCNTL so no problem should happen. My question is postfix locking option is = FCNTL, DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, why it dotlock the file 5 secs then remove the dotlock and say resource unaviable? why in the first place it dotlock the file, althought it couldnot FCNTL it in the first place since it is FCNTL by dovecot while reading the in the moment Any help will be greatly appreciated..... Regards Robert. From dg at dguhl.org Thu Oct 25 11:56:00 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 25 Oct 2012 10:56:00 +0200 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025085559.GA7323@laptop-dg.leere.eu> On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: [..] > should happen. My question is postfix locking option is = FCNTL, > DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, > why it dotlock the file 5 secs then remove the dotlock and say > resource unaviable? why in the first place it dotlock the file, > althought it couldnot FCNTL it in the first place since it is FCNTL > by dovecot while reading the in the moment You are on the wrong mailing list, this is no problem of Dovecot. Ask this question on postfix-users (maybe you are requested to show evidence of this behaviour). Dennis From roundcube222 at alaadin.org Thu Oct 25 12:26:10 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 12:26:10 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025085559.GA7323@laptop-dg.leere.eu> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025085559.GA7323@laptop-dg.leere.eu> Message-ID: On 2012-10-25 11:56, Dennis Guhl wrote: > On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > > [..] > >> should happen. My question is postfix locking option is = FCNTL, >> DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, >> why it dotlock the file 5 secs then remove the dotlock and say >> resource >> unaviable? why in the first place it dotlock the file, althought it >> couldnot FCNTL it in the first place since it is FCNTL by dovecot >> while >> reading the in the moment > > You are on the wrong mailing list, this is no problem of Dovecot. > > Ask this question on postfix-users (maybe you are requested to show > evidence of this behaviour). > > Dennis\ Thanks dennis for your reply. But, This is also an dovecot issue, because how dovecot use the FCNTL lock maybe different on how Postfix use it , that's why i had to ask also here at dovecot mailist this question. I think This behavior is understood by any one already have experience on how locking is made to /var/mail/files, I am just trying to understand how/why it happened. My question is postfix has locking option: FCNTL, DOTLOCK, and a file is already FCNTL by dovecot, will postfix Apply the DOTLOCK and continue try to FCNTL , althugh postfix already found that the file is already FCNTL by dovecot. So it shouldnot dotlock the mbx file untill it frist FCNTL the mailbox after dovecot finished. When i asked this question on irc postfix, they said that maybe FCNTL is used in different option with dovecot and this Might be the reason of the issue and recommended to ask here for help .. I guess Timo will be able to respond to this on the spot ... Thanks again. Robert JR. From jg at softjury.de Thu Oct 25 13:35:53 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Thu, 25 Oct 2012 12:35:53 +0200 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> Message-ID: <50891609.9070709@softjury.de> Am 25.10.2012 00:13, schrieb b m: > Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? Here a sentence to this, found in the dovecot wiki. (http://wiki2.dovecot.org/ACL) ACL groups support works by returning a comma-separated acl_groups extra field from userdb, which contains all the groups the user belongs to. It seems to be possible, I had an acl_groups field in my MySQL Database for this, I'am sure it is something like that in an AD too. From r.ordinas at math.univ-paris-diderot.fr Thu Oct 25 16:09:47 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Thu, 25 Oct 2012 15:09:47 +0200 Subject: [Dovecot] Small issue with "submission host" Message-ID: <50893A1B.2060205@math.univ-paris-diderot.fr> Hi everyone, I'm facing a small issue with the lda/lmtp "submission_host" feature in dovecot 2.0.14. When sending mail to MTA (in case of sieve filter forwarding for example), dovecot pass a RCPT TO command just after the EHLO. He's missing the MAIL FROM command. Therefore, my MTA show me a warning like this : "improper command pipelining after EHLO". How can i solve that ? Regards, Raphael From tlx at leuxner.net Thu Oct 25 17:08:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 25 Oct 2012 16:08:52 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <50893A1B.2060205@math.univ-paris-diderot.fr> References: <50893A1B.2060205@math.univ-paris-diderot.fr> Message-ID: <20121025140852.GA15639@nihlus.leuxner.net> On Thu, Oct 25, 2012 at 03:09:47PM +0200, Raphael Ordinas wrote: > When sending mail to MTA (in case of sieve filter forwarding for > example), dovecot pass a RCPT TO command just after the EHLO. He's > missing the MAIL FROM command. > Therefore, my MTA show me a warning like this : "improper command > pipelining after EHLO". Works for me with latest and greatest although I'm not using the 'submission_host' option but pure LMTP Unix socket: [...] service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } Best to show your 'doveconf -n' for more thoughts. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From rob0 at gmx.co.uk Thu Oct 25 17:28:00 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 25 Oct 2012 09:28:00 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025142800.GN3672@harrier.slackbuilds.org> On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > Stan, sorry but you didnot understand my question at all, dovecot > in this case is reading the mailbox file while user downloading the > mail and not WRITING. only postfix write when a mail arrives and > DOVECOT only read the mail. And even if both write to the file, I I can't answer (don't know), but I can tell you that this is not true. Dovecot also writes to the file: updating message read flags and such. > Any help will be greatly appreciated..... Maildir is not for everyone, but it does handle issues like this smoothly. The delivery agent is always able to deliver new mail. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From roundcube222 at alaadin.org Thu Oct 25 19:08:25 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 19:08:25 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025142800.GN3672@harrier.slackbuilds.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025142800.GN3672@harrier.slackbuilds.org> Message-ID: <23542f848cc61c879822b03810621256@Coptics.org> On 2012-10-25 17:28, /dev/rob0 wrote: > On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > >> Stan, sorry but you didnot understand my question at all, dovecot in >> this case is reading the mailbox file while user downloading the >> mail >> and not WRITING. only postfix write when a mail arrives and DOVECOT >> only read the mail. And even if both write to the file, I > > I can't answer (don't know), but I can tell you that this is not > true. Dovecot also writes to the file: updating message read flags > and such. > >> Any help will be greatly appreciated..... > > Maildir is not for everyone, but it does handle issues like this > smoothly. The delivery agent is always able to deliver new mail. We can not convert to maildir now as we have alot of users and converting each account will take a huge time .. well.. we can live with /var/mail/mailbox format , I just need to understand the locking issue and hope Timo will be able to answer our question.. Thanks again From stan at hardwarefreak.com Thu Oct 25 21:00:24 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:00:24 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <50897E38.6070304@hardwarefreak.com> On 10/25/2012 2:23 AM, Robert JR wrote: > On 2012-10-25 06:57, Stan Hoeppner wrote: > >> On 10/24/2012 3:04 PM, Robert JR wrote: >> >>> I have a question regarding mailbox locking and hope any one can help >>> me to better understanding the locking of mbox My Postfix lock option >>> is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock >>> virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is fcntl >>> only mbox_write_locks = fcntl mbox_read_locks = fcntl >> >> >> >> Postfix is delivering the mail to dovecot. This is done via the deliver >> program or lmtp which are pipes, not files. Thus, why is Postfix >> attempting to write files in the user's mail directory? >> >> You write new mail to the mailbox file with either Dovecot or Postfix, >> not both. Fix that problem and the locking problem disappears. > > Stan, sorry but you didnot understand my question at all Yes, actually I did, but I missed one part of it because I assumed you had Dovecot setup properly. It doesn't matter if the mbox locks are write or read or both. Locks are the problem, period, because you have two daemons fighting over the same files. The fix is absolutely trivial: Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates the file locking issue. Thus it also increases throughput as lock latency is eliminated. It also enables using all the Dovecot delivery plugins such as Sieve, Quota, anti-spam, etc. I had the same Postfix/Dovecot mbox locking problem many years ago when I first started using Dovecot. After the suggestions from the fine folks on this list I switched to LDA. It not only eliminated locking, it completely changed the character of my Dovecot install, both in performance and capabilities, as well as fixed some message flag problems, etc. If you're not using LDA with Postfix/Dovecot you're insane. ;) Some might say you're insane for using mbox but I feel it's fine for many installations. I use it myself. Our setups are very similar. To switch to LDA... If you're using 1.2.x http://wiki.dovecot.org/LDA/Postfix If you're using 2.x http://wiki2.dovecot.org/LDA/Postfix -- Stan From brintoul at sbcglobal.net Thu Oct 25 21:01:53 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 11:01:53 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process Message-ID: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Hello: I'm using Dovecot to provide IMAP services for Thunderbird clients. The user's mail is stored in Maildir format and the individual emails which the user "receives" are actually downloaded and put into the Maildir "repository" using a Python email client (POP3 client). I am trying to create the dovecot-uidlist file and maintain the "next UID" value within it, but I am having trouble with the UIDVALIDITY and the 128 bit mailbox global UID among other things. How can I "seed" a dovecot-uidlist file with these values..? Sorry if the information is not much to go on, please let me know what more information you might need to help point me in the right direction. Thanks, B. RIntoul From sven at svenhartge.de Thu Oct 25 21:06:17 2012 From: sven at svenhartge.de (Sven Hartge) Date: Thu, 25 Oct 2012 20:06:17 +0200 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <697ejv7veev8@mids.svenhartge.de> Bradley Rintoul wrote: > I'm using Dovecot to provide IMAP services for Thunderbird clients. > The user's mail is stored in Maildir format and the individual emails > which the user "receives" are actually downloaded and put into the > Maildir "repository" using a Python email client (POP3 client). > I am trying to create the dovecot-uidlist file and maintain the "next > UID" value within it, but I am having trouble with the UIDVALIDITY and > the 128 bit mailbox global UID among other things. How can I "seed" a > dovecot-uidlist file with these values..? > Sorry if the information is not much to go on, please let me know what > more information you might need to help point me in the right > direction. How about injecting the mails into the LDA. It will take care of proper delivery without the need for your programm to know the internal workings of dovecot. S? -- Sigmentation fault. Core dumped. From stan at hardwarefreak.com Thu Oct 25 21:23:37 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:23:37 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50897E38.6070304@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> Message-ID: <508983A9.9090605@hardwarefreak.com> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: > Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > the file locking issue. Thus it also increases throughput as lock > latency is eliminated. It also enables using all the Dovecot delivery > plugins such as Sieve, Quota, anti-spam, etc. I forgot to mention one very important feature of Dovecot LDA: New messages delivered by Postfix are indexed by LDA as they are written to the mailbox, flags updated at this time, etc. Thus when a mailbox is opened in an IMAP MUA, new messages are displayed instantly (I don't use POP but it's probably faster as well). With your current setup it can take from a few to many seconds to show new mail, depending on message count. With LDA new message count seems to have no impact on the speed of display. -- Stan From stan at hardwarefreak.com Thu Oct 25 21:37:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:37:50 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <508986FE.6060309@hardwarefreak.com> On 10/25/2012 1:01 PM, Bradley Rintoul wrote: > the individual emails which the user > "receives" are actually downloaded and put into the Maildir "repository" using a > Python email client (POP3 client). Tell us more about this. This doesn't sound like 'normal' email being fetched from an external service provider over a slow link scenario. This sounds more like an application server generating data files that are then POP'd down to the Dovecot server. Assuming that for now... If you're able to run a popd on this application server, why not run a simple smtp MTA and send these files directly to the user email addresses? Injecting the payload is a pretty simply shell command line, or from within a Python/Perl/etc script. Dovecot handles the rest as it arrives. Problem solved. The more you can do with smtp the better off you are and the easier it is. -- Stan From roundcube222 at alaadin.org Thu Oct 25 20:38:26 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 20:38:26 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508983A9.9090605@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> Message-ID: <239f53fb233beb2b45e68638cca24260@Coptics.org> On 2012-10-25 21:23, Stan Hoeppner wrote: > On 10/25/2012 1:00 PM, Stan Hoeppner wrote: > >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place >> of >> the Postfix local/virtual delivery agent. Using Dovecot LDA >> eliminates >> the file locking issue. Thus it also increases throughput as lock >> latency is eliminated. It also enables using all the Dovecot >> delivery >> plugins such as Sieve, Quota, anti-spam, etc. > > I forgot to mention one very important feature of Dovecot LDA: > > New messages delivered by Postfix are indexed by LDA as they are > written > to the mailbox, flags updated at this time, etc. Thus when a mailbox > is > opened in an IMAP MUA, new messages are displayed instantly (I don't > use > POP but it's probably faster as well). With your current setup it can > take from a few to many seconds to show new mail, depending on > message > count. With LDA new message count seems to have no impact on the > speed > of display. Thanks stan very much for your detailed answer, i will read about LDA to know how it works. But i still wonder why this mailbox locking issue and I hope for my referece some one at this form explain to me the issue reason. as since postfix,dovecot has the same lockign settings so why this issue happen ? I have spend 3 days searching for an answer and i couldnot find any.. I think only Timo can answer my question... Thanks again stan From brintoul at sbcglobal.net Thu Oct 25 22:13:34 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 12:13:34 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <697ejv7veev8@mids.svenhartge.de> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> Message-ID: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> This could be good. I'd never looked into the LDA - I will study up on it. Someone else was helping out here and I thought I'd shed some more light on what I'm doing here... Let's say someone has an account with Yahoo, for example. My Python code is fetching email from the user's Yahoo! account and placing it into the Dovecot Maildir storage for a particular user. Now when the user retrieves their mail, they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, of sorts... Thanks for the responses! (Is there an IRC channel?) ________________________________ From: Sven Hartge To: dovecot at dovecot.org Sent: Thu, October 25, 2012 11:07:26 AM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process Bradley Rintoul wrote: > I'm using Dovecot to provide IMAP services for Thunderbird clients. > The user's mail is stored in Maildir format and the individual emails > which the user "receives" are actually downloaded and put into the > Maildir "repository" using a Python email client (POP3 client). > I am trying to create the dovecot-uidlist file and maintain the "next > UID" value within it, but I am having trouble with the UIDVALIDITY and > the 128 bit mailbox global UID among other things. How can I "seed" a > dovecot-uidlist file with these values..? > Sorry if the information is not much to go on, please let me know what > more information you might need to help point me in the right > direction. How about injecting the mails into the LDA. It will take care of proper delivery without the need for your programm to know the internal workings of dovecot. S? -- Sigmentation fault. Core dumped. From slitt at troubleshooters.com Thu Oct 25 22:31:38 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 15:31:38 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <697ejv7veev8@mids.svenhartge.de> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> Message-ID: <20121025153138.39c6b363@mydesk> On Thu, 25 Oct 2012 20:06:17 +0200, Sven Hartge said: > Bradley Rintoul wrote: > > > I'm using Dovecot to provide IMAP services for Thunderbird clients. > > The user's mail is stored in Maildir format and the individual > > emails which the user "receives" are actually downloaded and put > > into the Maildir "repository" using a Python email client (POP3 > > client). > > > I am trying to create the dovecot-uidlist file and maintain the > > "next UID" value within it, but I am having trouble with the > > UIDVALIDITY and the 128 bit mailbox global UID among other things. > > How can I "seed" a dovecot-uidlist file with these values..? > > > Sorry if the information is not much to go on, please let me know > > what more information you might need to help point me in the right > > direction. > > How about injecting the mails into the LDA. It will take care of > proper delivery without the need for your programm to know the > internal workings of dovecot. What is an LDA? SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From slitt at troubleshooters.com Thu Oct 25 22:35:06 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 15:35:06 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <20121025153138.39c6b363@mydesk> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <20121025153138.39c6b363@mydesk> Message-ID: <20121025153506.1f959544@mydesk> On Thu, 25 Oct 2012 15:31:38 -0400, Steve Litt said: > On Thu, 25 Oct 2012 20:06:17 +0200, Sven Hartge said: > > How about injecting the mails into the LDA. It will take care of > > proper delivery without the need for your programm to know the > > internal workings of dovecot. > > What is an LDA? Nevermind, somebody had already answered that question. Sorry for asking too quickly. SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From loshkovskyi at yandex.ua Thu Oct 25 13:03:23 2012 From: loshkovskyi at yandex.ua (Loshkovskyi Andrii) Date: Thu, 25 Oct 2012 13:03:23 +0300 Subject: [Dovecot] Dovecot sends BYE while fetching X-GM-MSGID Message-ID: <1698191351159403@web29e.yandex.ru> Hello everyone, While using the following set of commands, I am having the error as below: FETCH 7 (X-GM-MSGID) A15 FETCH 7 (X-GM-MSGID) A15 BAD Error in IMAP command FETCH: Unknown parameter X-GM-MSGID Can I somehow disable such errors so that Dovecot won't send BYE on X-GM-MSGID but just proceed with following emails? -- My configuration is below: [root at server ~]# cat /etc/issue CentOS release 6.3 (Final) [root at server ~]# dovecot --version 2.0.9 [root at server ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.17.1.el6.centos.plus.i686 i686 CentOS release 6.3 (Final) ext4 auth_debug = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes default_client_limit = 10000 default_process_limit = 300 disable_plaintext_auth = no listen = * mail_location = maildir:/var/spool/mail/%d/%n/Maildir mbox_write_locks = fcntl passdb { args = scheme=SSHA512 username_format=%u /etc/dovecot/users driver = passwd-file } service imap-login { service_count = 0 vsz_limit = 128 M } ssl_cert = References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <508995D2.3070705@whyscream.net> On 25/10/12 21:13, Bradley Rintoul wrote: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on what > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... I'm intrigued by this. Why are you using some self-baked(?) python script to fetch the mail in stead of using ready-made components like fetchmail? Unless there's a special reason not to, try using the LDA (and fetchmail/getmail for that matter). This sounds exactly what you want: http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal -- Tom > > Thanks for the responses! > > (Is there an IRC channel?) > > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > From rs at sys4.de Thu Oct 25 22:46:20 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 25 Oct 2012 21:46:20 +0200 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <5089970C.1080900@sys4.de> Am 25.10.2012 21:13, schrieb Bradley Rintoul: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on what > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... > > Thanks for the responses! > > (Is there an IRC channel?) > perhaps you could use parts from here http://wiki.dovecot.org/HowTo/RefilterMail > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft M?nchen Amtsgericht M?nchen HRB 0000 Vorstandsmitglieder Patrick Ben Koetter Axel von der Ohe Marc Schifbauer Vorstandsvorsitzender Patrick Ben Koetter Aufsichtsratsvorsitzender J?rg Heidrich From brintoul at sbcglobal.net Thu Oct 25 22:54:43 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 12:54:43 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <508995D2.3070705@whyscream.net> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> Message-ID: <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> I am brand new to this whole "email" thing. I am looking at this article right now: http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot Thanks for the input! ________________________________ From: Tom Hendrikx To: dovecot at dovecot.org Sent: Thu, October 25, 2012 12:41:24 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process On 25/10/12 21:13, Bradley Rintoul wrote: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on >what > > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their >mail, > > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > > of sorts... I'm intrigued by this. Why are you using some self-baked(?) python script to fetch the mail in stead of using ready-made components like fetchmail? Unless there's a special reason not to, try using the LDA (and fetchmail/getmail for that matter). This sounds exactly what you want: http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal -- Tom > > Thanks for the responses! > > (Is there an IRC channel?) > > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > From rob0 at gmx.co.uk Thu Oct 25 23:10:59 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 25 Oct 2012 15:10:59 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <20121025201059.GO3672@harrier.slackbuilds.org> > From: Tom Hendrikx > I'm intrigued by this. Why are you using some self-baked(?) python > script to fetch the mail in stead of using ready-made components > like fetchmail? > > Unless there's a special reason not to, try using the LDA (and > fetchmail/getmail for that matter). > > This sounds exactly what you want: > http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal > On Thu, Oct 25, 2012 at 12:54:43PM -0700, Bradley Rintoul wrote: > I am brand new to this whole "email" thing. I am looking at this > article right now: > http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot I did not see where you described the ultimate goal. That should have been the starting point of this thread. Describe the problem, not how you think it should be solved, because you are new to this, and your ideas might benefit from some scrutiny. Use plain language. I have not reviewed your howto, but personally I would recommend neither fetchmail (I'd choose getmail) nor procmail (other choices exist, depending on what you are trying to do.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Thu Oct 25 23:18:51 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 15:18:51 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <239f53fb233beb2b45e68638cca24260@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> Message-ID: <50899EAB.2030003@hardwarefreak.com> On 10/25/2012 12:38 PM, Robert JR wrote: > On 2012-10-25 21:23, Stan Hoeppner wrote: > >> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: >> >>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >>> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >>> the file locking issue. Thus it also increases throughput as lock >>> latency is eliminated. It also enables using all the Dovecot delivery >>> plugins such as Sieve, Quota, anti-spam, etc. >> >> I forgot to mention one very important feature of Dovecot LDA: >> >> New messages delivered by Postfix are indexed by LDA as they are written >> to the mailbox, flags updated at this time, etc. Thus when a mailbox is >> opened in an IMAP MUA, new messages are displayed instantly (I don't use >> POP but it's probably faster as well). With your current setup it can >> take from a few to many seconds to show new mail, depending on message >> count. With LDA new message count seems to have no impact on the speed >> of display. > > Thanks stan very much for your detailed answer, i will read about LDA to > know how it works. But i still wonder why this mailbox locking issue and I > hope for my referece some one at this form explain to me the issue reason. > as since postfix,dovecot has the same lockign settings so why this issue > happen ? I have spend 3 days searching for an answer and i couldnot find > any.. I think only Timo can answer my question... I chose to focus on the permanent and proper solution, which is eliminating your lock contention altogether and enabling maximum performance and features. Learning about file locking problems between applications may be a noble endeavor, but at this point it's simply a waste of your time. A gazillion papers have been written about this subject over the years. If you're that interested hunt them down and read them. I'm sure Google can find 1000 of them quickly. You'll be looking for academic papers, not forum posts. > Thanks again stan You're welcome. -- Stan From slitt at troubleshooters.com Thu Oct 25 23:18:52 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 16:18:52 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <20121025161852.5d3c448f@mydesk> On Thu, 25 Oct 2012 12:13:34 -0700 (PDT), Bradley Rintoul said: > This could be good. I'd never looked into the LDA - I will study up > on it. > > Someone else was helping out here and I thought I'd shed some more > light on what I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python > code is fetching email from the user's Yahoo! account and placing it > into the Dovecot Maildir storage for a particular user. Now when > the user retrieves their mail, they are doing so using my Dovecot - > my Dovecot instance is acting as a proxy, of sorts... > > Thanks for the responses! > > (Is there an IRC channel?) Hi Bradley, I'm doing almost the exact same thing, but with fetchmail and procmail. I go out and grab my email from about five different places using fetchmail, which feeds the messages to procmail, with .procmailrc deciding where in the Dovecot maildir tree to put them. Your only need I *haven't* accomplished is having different users get their mail from my Dovecot, and to make sure each users' email goes where they can get it via IMAP connection to your Dovecot. If you can get different IMAP mailboxes for different users, you can put each user in .procmailrc so as to deliver to the correct box. Anyway, Procmail knows exactly how to submit an email to Dovecot, so you don't need to worry about actually placing the file into the tree, or anything like that. You mention you've written some Python code. If the purpose of your Python code is just to retrieve from SMTP servers, you can drop your Python code in favor of Fetchmail and Procmail. If your Python code actually does something with the emails, you can call a subset of your Python code from Procmail, to do its magic on each email. Here's how my fetchmail is running: 29588 ? Ss 0:21 fetchmail -f /home/slitt/.fetchmailrc And here's a partial view of my .fetchmailrc, showing my retrieval from Bluehost and gmail: =================================== set postmaster "slitt" set bouncemail set no spambounce set properties "" set daemon 180 #poll mail.a3b3.com protocol POP3: poll mail.a3b3.com protocol IMAP: user 'slitt at troubleshooters.com' there is 'slitt' here pass wouldnt_you_like_to_know limit 50000000 warnings 3200 expunge 60 ssl #Use ssl encryption sslcommonname "*.bluehost.com" sslcertck mda "/usr/bin/procmail -d %T" fetchall; poll imap.gmail.com protocol IMAP user 'litttest at gmail.com' there is 'slitt' here pass 'I_just_cant_say' #portnumber 993 limit 50000000 warnings 3200 expunge 60 mda "/usr/bin/procmail -d %T" fetchlimit 50 ssl; ==================================== Do you notice the "mda" line on both pulls? That means "use procmail as your mda", which just ships each email to Procmail. Procmail knows exactly how to deliver stuff to Dovecot. The following are the top several lines of my .procmailrc: =================================== DEFAULT=$HOME/mail/Maildir/.INBOX/ MAILDIR=$HOME/mail/Maildir/ LOCKFILE=$HOME/mail/.lock VERBOSE=no LOGFILE=$HOME/procmail/log #GARBAGE=.garbage/ GARBAGE=/dev/null PURGATORY=.garbage/ SUPREMUM=9876543210 #PROCMAIL SUPREMUM NUMBER, SEE http://www.perlcode.org/tutorials/procmail/proctut/proctip2.pod #### HANDLE STUFF FROM littdom at gmail.com and litttest at gmail.com #### :0: * ^Delivered-To:.*littdom at gmail.com .littdom_gmail/ :0: * ^Delivered-To:.*litttest at gmail.com .litttest_gmail/ =================================== A few explanations: First, I couldn't include my actual filters, because they are full of very unflattering comments concerning various trolls, ignos, blabbermouths, and proudly helpless fools. The $MAILDIR environment variable is the rood directory of your Maildir tree. $DEFAULT is the location of the main inbox for that -- I think it's where you put email that doesn't get routed elsewhere by Procmail. $GARBAGE is an environment var I made up as code for where filtered stuff gets sent. It's usually /dev/null because I don't want to see that junk again. However, I can temporarily change it to an actual IMAP directory for troubleshooting. $PURGATORY is junk that I actually want to OK the deletion of. I actually currently have nothing filtered to $PURGATORY, but it's there. $SUPREMUM is a very large number that is used in making OR logic, which is otherwise difficult. I couldn't make the $SUPREMUM env var work, so I had to use a literal, and here's a way I got all my magazines into one mailbox: :0: * 9876543210^0 ^From.*onsale.com * 9876543210^0 ^From.*pcmag.com * 9876543210^0 ^From.*itworld.com * 9876543210^0 ^From.*networkworld.info * 9876543210^0 ^From.*infoworld.com * 9876543210^0 ^From.*whatsnewnow.com * 9876543210^0 ^From.*eweek.com * 9876543210^0 ^From.*computerworld.com .mags/ By the way, BE SURE to note the slash after the directory name. That trailing slash tells Procmail that it's delivering to a Maildir, not to an (ugh) mbox. Anyway, I think you and I are doing very similar things, albeit for very different reasons. My motivation is that I consider all currently available email clients to be junk, and don't want them holding my email, so I hold it in a Dovecot hosted Maildir instead. I'll be interested in how you solve this. Please keep me (and probably everyone on this list) in the loop. Thank you so much! SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From mike at alaadin.org Thu Oct 25 22:21:24 2012 From: mike at alaadin.org (Mike John) Date: Thu, 25 Oct 2012 22:21:24 +0300 Subject: [Dovecot] Changing password for users Message-ID: <7a6a7cc64756b895059f7384b256019b@coptics.org> Hello, I am using dovecot (2.0.9) and using virtual users using passdb { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i make my virtual users change their passwords using web interface ? My users already uses squirrelmail to access their mail. is there a program to add to squirrelmail to add this function to the clients ? or should i user different separate website for password changing ? and what program/tool can help me with this ? Any ideas is greatly appreciated. Mike. From roundcube222 at alaadin.org Thu Oct 25 22:42:32 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 22:42:32 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50899EAB.2030003@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> <50899EAB.2030003@hardwarefreak.com> Message-ID: On 2012-10-25 23:18, Stan Hoeppner wrote: > On 10/25/2012 12:38 PM, Robert JR wrote: > >> On 2012-10-25 21:23, Stan Hoeppner wrote: >> >>> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: >>> >>>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in >>>> place of the Postfix local/virtual delivery agent. Using Dovecot >>>> LDA eliminates the file locking issue. Thus it also increases >>>> throughput as lock latency is eliminated. It also enables using >>>> all >>>> the Dovecot delivery plugins such as Sieve, Quota, anti-spam, etc. >>> I forgot to mention one very important feature of Dovecot LDA: New >>> messages delivered by Postfix are indexed by LDA as they are >>> written >>> to the mailbox, flags updated at this time, etc. Thus when a >>> mailbox >>> is opened in an IMAP MUA, new messages are displayed instantly (I >>> don't use POP but it's probably faster as well). With your current >>> setup it can take from a few to many seconds to show new mail, >>> depending on message count. With LDA new message count seems to >>> have >>> no impact on the speed of display. >> Thanks stan very much for your detailed answer, i will read about >> LDA >> to know how it works. But i still wonder why this mailbox locking >> issue >> and I hope for my referece some one at this form explain to me the >> issue reason. as since postfix,dovecot has the same lockign settings >> so >> why this issue happen ? I have spend 3 days searching for an answer >> and >> i couldnot find any.. I think only Timo can answer my question... > > I chose to focus on the permanent and proper solution, which is > eliminating your lock contention altogether and enabling maximum > performance and features. > > Learning about file locking problems between applications may be a > noble > endeavor, but at this point it's simply a waste of your time. A > gazillion papers have been written about this subject over the years. > If you're that interested hunt them down and read them. I'm sure > Google > can find 1000 of them quickly. You'll be looking for academic papers, > not forum posts. > >> Thanks again stan > > You're welcome. Thanks again Stan, you are very helpfull, i will start learning how to configure LDA, but hopefully i can also have an answer from Timo about why this issue happened.. i am sure he is aware of it and can explain it.. From stocton12 at yahoo.com Thu Oct 25 23:49:52 2012 From: stocton12 at yahoo.com (b m) Date: Thu, 25 Oct 2012 13:49:52 -0700 (PDT) Subject: [Dovecot] Public folders and groups In-Reply-To: <50891609.9070709@softjury.de> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> Message-ID: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> No AD doesn't have such a field, but I could use some unused field to get what I want. Let's say set "Attribute1" to "group1". The problem is how to get that info. I guess I have to edit dovecot-ldap.conf and put in user_attrs something like that ",=acl_groups=Attribute1". Any suggestions? ________________________________ From: Jan Phillip Greimann To: b m ; Dovecot Mailing List Sent: Thursday, October 25, 2012 1:35 PM Subject: Re: [Dovecot] Public folders and groups Am 25.10.2012 00:13, schrieb b m: > Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? Here a sentence to this, found in the dovecot wiki. (http://wiki2.dovecot.org/ACL) ACL groups support works by returning a comma-separated acl_groups extra field from userdb, which contains all the groups the user belongs to. It seems to be possible, I had an acl_groups field in my MySQL Database for this, I'am sure it is something like that in an AD too. From stan at hardwarefreak.com Fri Oct 26 00:01:58 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 16:01:58 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <5089A8C6.1090308@hardwarefreak.com> On 10/25/2012 2:13 PM, Bradley Rintoul wrote: > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... There are already a gazillion email services that do mailbox consolidation--the term for what you're attempting. Why are you reinventing the wheel? Must be a family and the dog project... A quick browse of your Wordpress blog confirms such. It also explains why you wrote code to create maildir files from scratch. Only a programmer assumes that the first path to a solution is to write new code. A system administrator on the other hand, most of us here, Google's for suitable FOSS, then looks in his distro repos for it, and builds from source if a package isn't available. We only script when out of Elmer's and duct tape. ;) -- Stan From ben at morrow.me.uk Fri Oct 26 00:15:09 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 25 Oct 2012 22:15:09 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025211509.GE5388@anubis.morrow.me.uk> At 10AM +0300 on 25/10/12 you (Robert JR) wrote: > > Stan, sorry but you didnot understand my question at all, dovecot in > this case is reading the mailbox file while user downloading the mail > and not WRITING. only postfix write when a mail arrives and DOVECOT > only read the mail. And even if both write to the file, I have already > set the locking option of both to FCNTL so no problem should happen. > My question is postfix locking option is = FCNTL, DOTLOCK , and > dovecot = FCNTL, if postfix find a file already FCNTL, why it dotlock > the file 5 secs then remove the dotlock and say resource unaviable? As Stan said earlier, this is a Postfix question. The rule for dotlocking is that you must create the .lock *before* opening the file, in case whoever has it locked will be replacing the file altogether; but with fcntl locking you must acquire the lock *after* opening the file, since that's the way the syscall works. This means that if Postfix is going to use both forms of lock, it has to acquire a dotlock before it can look for a fcntl lock. In other words: the methods in mailbox_delivery_lock are *not* tried in order, because they can't be. Dotlock is always tried first. You should have compatible locking settings for all your programs accessing your mboxes. If Postfix is using dotlock, Dovecot should be using dotlock as well. If you don't have any local programs (mail clients, for instance) which require dotlocks, you should probably change Postfix to just use fcntl locks. Ben From ben at morrow.me.uk Fri Oct 26 00:24:03 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 25 Oct 2012 22:24:03 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50897E38.6070304@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> Message-ID: <20121025212403.GF5388@anubis.morrow.me.uk> At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > > Yes, actually I did, but I missed one part of it because I assumed you > had Dovecot setup properly. > > It doesn't matter if the mbox locks are write or read or both. Locks > are the problem, period, because you have two daemons fighting over the > same files. The fix is absolutely trivial: > > Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > the file locking issue. Thus it also increases throughput as lock > latency is eliminated. Nonsense. deliver and imap are still separate processes accessing the same mbox, so they still need to use locks. The only difference is that since they are both dovecot programs, they will automatically be using the *same* locking strategies, and things will Just Work. Ben From brintoul at sbcglobal.net Fri Oct 26 01:02:05 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 15:02:05 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <5089A8C6.1090308@hardwarefreak.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <5089A8C6.1090308@hardwarefreak.com> Message-ID: <1351202525.21830.YahooMailRC@web184702.mail.ne1.yahoo.com> I didn't actually write code to create Maildir files from scratch, technically. I used 15-20 lines of Python to do that and to actually fetch the mail in addition... Mailbox consolidation isn't the only thing which this project I'm working on requires, but it's a big part of it. :) Anyway, I really appreciate all the advice and help! ________________________________ From: Stan Hoeppner To: dovecot at dovecot.org Sent: Thu, October 25, 2012 2:02:10 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process On 10/25/2012 2:13 PM, Bradley Rintoul wrote: > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their >mail, > > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > > of sorts... There are already a gazillion email services that do mailbox consolidation--the term for what you're attempting. Why are you reinventing the wheel? Must be a family and the dog project... A quick browse of your Wordpress blog confirms such. It also explains why you wrote code to create maildir files from scratch. Only a programmer assumes that the first path to a solution is to write new code. A system administrator on the other hand, most of us here, Google's for suitable FOSS, then looks in his distro repos for it, and builds from source if a package isn't available. We only script when out of Elmer's and duct tape. ;) -- Stan From brintoul at sbcglobal.net Fri Oct 26 01:04:02 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 15:04:02 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <20121025201059.GO3672@harrier.slackbuilds.org> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> <20121025201059.GO3672@harrier.slackbuilds.org> Message-ID: <1351202642.93491.YahooMailRC@web184703.mail.ne1.yahoo.com> Cool. Thanks for the input. Can you tell me briefly why you'd choose getmail? Thanks again. B. RIntoul ________________________________ From: /dev/rob0 To: dovecot at dovecot.org Sent: Thu, October 25, 2012 1:11:13 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process > From: Tom Hendrikx > I'm intrigued by this. Why are you using some self-baked(?) python > script to fetch the mail in stead of using ready-made components > like fetchmail? > > Unless there's a special reason not to, try using the LDA (and > fetchmail/getmail for that matter). > > This sounds exactly what you want: > http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal > On Thu, Oct 25, 2012 at 12:54:43PM -0700, Bradley Rintoul wrote: > I am brand new to this whole "email" thing. I am looking at this > article right now: > http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot I did not see where you described the ultimate goal. That should have been the starting point of this thread. Describe the problem, not how you think it should be solved, because you are new to this, and your ideas might benefit from some scrutiny. Use plain language. I have not reviewed your howto, but personally I would recommend neither fetchmail (I'd choose getmail) nor procmail (other choices exist, depending on what you are trying to do.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Fri Oct 26 01:33:48 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 17:33:48 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> <50899EAB.2030003@hardwarefreak.com> Message-ID: <5089BE4C.4010709@hardwarefreak.com> On 10/25/2012 2:42 PM, Robert JR wrote: > Thanks again Stan, you are very helpfull, i will start learning how to > configure LDA, but hopefully i can also have an answer from Timo about > why this issue happened.. i am sure he is aware of it and can explain it.. Probably not. You describe a dot locking "problem" with Postfix. If you want information about that you need to ask on the Postfix list, not the Dovecot list. Wietse will answer you properly. Give it a subject "dot lock problem". -- Stan From jeff at bubble.org Fri Oct 26 01:57:47 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Thu, 25 Oct 2012 18:57:47 -0400 Subject: [Dovecot] Snarf plugin In-Reply-To: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> References: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> Message-ID: <5089C3EB.1040301@bubble.org> Just thought I'd follow up on my original post, I got the snarf plugin to work properly with some help from Jonathan at PSU (need to give credit where credit is due) For anybody else looking for the configuration here is the relevant output from dovecot -n # 2.1.10: //etc/dovecot/dovecot.conf # OS: Linux 3.6.2-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) ext4 mail_location = mbox:/home/%u/mail:INBOX=/home/%u/mbox mail_plugins = snarf zlib namespace Snarf { hidden = yes list = no location = mbox:/home/%u/mbox:INBOX=/var/spool/mail/%u:INDEX=MEMORY prefix = ~~Snarfbox/ separator = / } namespace default { inbox = yes location = prefix = separator = / } plugin { snarf = ~~Snarfbox/INBOX } What I found was I was specifying the snarf mbox location as ~/mbox, once I changed to /home/%u/mbox things started working. Thanks, Jeff From jeff at bubble.org Fri Oct 26 02:07:04 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Thu, 25 Oct 2012 19:07:04 -0400 Subject: [Dovecot] Changing password for users In-Reply-To: <7a6a7cc64756b895059f7384b256019b@coptics.org> References: <7a6a7cc64756b895059f7384b256019b@coptics.org> Message-ID: <5089C618.6090605@bubble.org> On 10/25/2012 03:21 PM, Mike John wrote: > Hello, > > I am using dovecot (2.0.9) and using virtual users using > > passdb { > args = /etc/dovecot/dovecotpasswd > driver = passwd-file > } > > How can i make my virtual users change their passwords using web > interface ? > > My users already uses squirrelmail to access their mail. is there a > program to add to squirrelmail to add this function to the clients ? > or should i user different separate website for password changing ? > and what program/tool can help me with this ? > > Any ideas is greatly appreciated. > > Mike. Mike, I don't know about forcing users to change their passwords however with Squirrelmail there are several password change plugins available that use "poppasswd" to actually change the password. Of course poppasswd will probably need to be modified to go against your password data base, in my case it simply uses PAM. The version I use is poppassd version 1.8.5. Oh you probably want to restrict access to the port from the local host only since passwords are transmitted in clear text. Jeff From roundcube222 at alaadin.org Fri Oct 26 01:11:14 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Fri, 26 Oct 2012 01:11:14 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025211509.GE5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> Message-ID: <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> On 2012-10-26 00:15, Ben Morrow wrote: > At 10AM +0300 on 25/10/12 you (Robert JR) wrote: > >> Stan, sorry but you didnot understand my question at all, dovecot in >> this case is reading the mailbox file while user downloading the >> mail >> and not WRITING. only postfix write when a mail arrives and DOVECOT >> only read the mail. And even if both write to the file, I have >> already >> set the locking option of both to FCNTL so no problem should happen. >> My >> question is postfix locking option is = FCNTL, DOTLOCK , and dovecot >> = >> FCNTL, if postfix find a file already FCNTL, why it dotlock the file >> 5 >> secs then remove the dotlock and say resource unaviable? > > As Stan said earlier, this is a Postfix question. The rule for > dotlocking is that you must create the .lock *before* opening the > file, > in case whoever has it locked will be replacing the file altogether; > but > with fcntl locking you must acquire the lock *after* opening the > file, > since that's the way the syscall works. This means that if Postfix is > going to use both forms of lock, it has to acquire a dotlock before > it > can look for a fcntl lock. > > In other words: the methods in mailbox_delivery_lock are *not* tried > in > order, because they can't be. Dotlock is always tried first. > > You should have compatible locking settings for all your programs > accessing your mboxes. If Postfix is using dotlock, Dovecot should be > using dotlock as well. If you don't have any local programs (mail > clients, for instance) which require dotlocks, you should probably > change Postfix to just use fcntl locks. > > Ben Thanks Ben for your valuable support and detailed explanation .. but according to your explanation dovecot documentation contains wrong explanation to the dotlock and that's why i was confused. it is written as follow ** If multiple lock methods are used, which is usually the case since dotlocks aren't typically used for read locking, the order in which the locking is done is important. Consider if two programs were running at the same time, both use dotlock and fcntl locking but in different order: Program A: fcntl locks the mbox Program B at the same time: dotlocks the mbox Program A continues: tries to dotlock the mbox, but since it's already dotlocked by B, it starts waiting Program B continues: tries to fcntl lock the mbox, but since it's already fcntl locked by A, it starts waiting Now both of them are waiting for each others locks. Finally after a couple of minutes they time out and fail the operation. ** So this means that the documentation as mentioned above is wrong and dotlock is always first even if the order is different .. Anyway i have sent this question to postfix maillist and i will wait their reply . Thanks again Robert. From mike at alaadin.org Fri Oct 26 01:17:43 2012 From: mike at alaadin.org (Mike John) Date: Fri, 26 Oct 2012 01:17:43 +0300 Subject: [Dovecot] Changing password for users Message-ID: <02a35ba19c559b258dba0de278e31a4d@coptics.org> > Hello, > > I am using dovecot (2.0.9) and using virtual users using > > passdb { > args = /etc/dovecot/dovecotpasswd > driver = passwd-file > } > > How can i make my virtual users change their passwords using web > interface ? > > My users already uses squirrelmail to access their mail. is there a > program to add to squirrelmail to add this function to the clients ? > or should i user different separate website for password changing ? > and what program/tool can help me with this ? > > Any ideas is greatly appreciated. > > Mike. >Mike, >I don't know about forcing users to change their passwords however > with >Squirrelmail there are several password change plugins available that >use "poppasswd" to actually change the password. >Of course poppasswd will probably need to be modified to go against > your >password data base, in my case it simply uses PAM. The version I use > is >poppassd version 1.8.5. >Oh you probably want to restrict access to the port from the local > host >only since passwords are transmitted in clear text. >Jeff I know about poppassd , but it works only for /etc/passwd , /etc/shadow, but my dovecot virtual users password files are in different location and i do not know how to modify poppassd, any idea how can i do that? and is there another way other than poppassd? From stan at hardwarefreak.com Fri Oct 26 03:25:53 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 19:25:53 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025212403.GF5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> Message-ID: <5089D891.9080207@hardwarefreak.com> On 10/25/2012 4:24 PM, Ben Morrow wrote: > At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >> >> Yes, actually I did, but I missed one part of it because I assumed you >> had Dovecot setup properly. >> >> It doesn't matter if the mbox locks are write or read or both. Locks >> are the problem, period, because you have two daemons fighting over the >> same files. The fix is absolutely trivial: >> >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >> the file locking issue. Thus it also increases throughput as lock >> latency is eliminated. > > Nonsense. deliver and imap are still separate processes accessing the > same mbox, so they still need to use locks. The only difference is that > since they are both dovecot programs, they will automatically be using > the *same* locking strategies, and things will Just Work. "Nonsense" implies what I stated was factually incorrect, which is not the case. There's a difference between factual incorrectness and simply staying out of the weeds. If you want to get into the weeds, and have me call you out for "nonsense", LDA/deliver is not a separate UNIX process. The LDA code runs within the imap process for the given user. This is what allows Dovecot to perform 'simultaneous' reads/writes to an mbox file, avoiding filesystem level locking latency. Using filesystem level locking to control read/write access between processes of own's program would be insane on many levels. -- Stan From simon.buongiorno at gmail.com Fri Oct 26 05:59:13 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Thu, 25 Oct 2012 22:59:13 -0400 Subject: [Dovecot] Changing password for users In-Reply-To: <02a35ba19c559b258dba0de278e31a4d@coptics.org> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> Message-ID: On Oct 25, 2012 7:20 PM, "Mike John" wrote: >> >> Hello, >> >> I am using dovecot (2.0.9) and using virtual users using >> >> passdb { >> args = /etc/dovecot/dovecotpasswd >> driver = passwd-file >> } >> >> How can i make my virtual users change their passwords using web >> interface ? >> >> My users already uses squirrelmail to access their mail. is there a >> program to add to squirrelmail to add this function to the clients ? >> or should i user different separate website for password changing ? >> and what program/tool can help me with this ? >> >> Any ideas is greatly appreciated. >> >> Mike. >> Mike, > > >> I don't know about forcing users to change their passwords however with >> Squirrelmail there are several password change plugins available that >> use "poppasswd" to actually change the password. > > >> Of course poppasswd will probably need to be modified to go against your >> password data base, in my case it simply uses PAM. The version I use is >> poppassd version 1.8.5. > > >> Oh you probably want to restrict access to the port from the local host >> only since passwords are transmitted in clear text. > > >> Jeff > > > I know about poppassd , but it works only for /etc/passwd , /etc/shadow, but my dovecot virtual users password files > are in different location and i do not know how to modify poppassd, any idea how can i do that? and is there another way other than poppassd? Horde has a change password module too. And essentially it's trivial to write your own php page to do it. I'll do it if you want to contract it out. Simon From stan at hardwarefreak.com Fri Oct 26 06:12:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 22:12:34 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508983A9.9090605@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> Message-ID: <5089FFA2.9050105@hardwarefreak.com> On 10/25/2012 1:23 PM, Stan Hoeppner wrote: > I forgot to mention one very important feature of Dovecot LDA: > > New messages delivered by Postfix are indexed by LDA as they are written > to the mailbox, flags updated at this time, etc. Thus when a mailbox is > opened in an IMAP MUA, new messages are displayed instantly (I don't use > POP but it's probably faster as well). With your current setup it can > take from a few to many seconds to show new mail, depending on message > count. With LDA new message count seems to have no impact on the speed > of display. Robert JR, you posted relevant information to the Postfix list that you omitted here, or at least I didn't see it. This may directly affect the advice myself and others gave you. You stated there that you're using Dovecot POP only and not IMAP. Given the nature of POP, using LDA may not help much even if it eliminates the filesystem locking contention between processes. I don't know if Dovecot will append an mbox file while reading the entire file. This Timo will have an answer to. Timo may also state, as he has before on this list, that: 1. there are better, faster, POPers available 2. Dovecot is developed primarily as an IMAP server 3. the POP function has received little development for quite some time -- Stan From ben at morrow.me.uk Fri Oct 26 06:54:59 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 04:54:59 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <5089D891.9080207@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> Message-ID: <20121026035458.GG5388@anubis.morrow.me.uk> At 7PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > On 10/25/2012 4:24 PM, Ben Morrow wrote: > > At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > >> > >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > >> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > >> the file locking issue. Thus it also increases throughput as lock > >> latency is eliminated. > > > > Nonsense. deliver and imap are still separate processes accessing the > > same mbox, so they still need to use locks. The only difference is that > > since they are both dovecot programs, they will automatically be using > > the *same* locking strategies, and things will Just Work. > > "Nonsense" implies what I stated was factually incorrect, which is not > the case. There's a difference between factual incorrectness and simply > staying out of the weeds. What you stated was factually incorrect. > If you want to get into the weeds, and have me call you out for > "nonsense", LDA/deliver is not a separate UNIX process. The LDA code > runs within the imap process for the given user. Nonsense. dovecot-lda runs in its own process, and does not involve the imap process in any way. As such it has to do locking. If I have the following in my dovecot.conf: mbox_read_locks = fcntl mbox_write_locks = dotlock fcntl namespace { location = mbox:~/mbox separator = / type = private list = yes prefix = MBOX/ } and I run ktrace dovecot-lda -f mauzo at localhost -m MBOX/foo < mail then the ktrace contains 44973 dovecot-lda CALL access(0x8021f5f68,0x6) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo" 44973 dovecot-lda RET access 0 [Check the mbox exists and is accessible] 44973 dovecot-lda CALL lstat(0x8020196c0,0x7fffffffcb60) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat -1 errno 2 No such file or directory [Look for a .lock file] 44973 dovecot-lda CALL open(0x8020196c0,0xa02, 0x1b6) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET open 9 44973 dovecot-lda CALL write(0x9,0x802019830,0x19) 44973 dovecot-lda RET write 25/0x19 44973 dovecot-lda CALL clock_gettime(0xd,0x7fffffffcbf0) 44973 dovecot-lda RET clock_gettime 0 44973 dovecot-lda CALL fstat(0x9,0x7fffffffcd90) 44973 dovecot-lda RET fstat 0 44973 dovecot-lda CALL close(0x9) 44973 dovecot-lda RET close 0 44973 dovecot-lda CALL lstat(0x8020196c0,0x7fffffffce60) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat 0 [Create a .lock file, and check it was successful] 44973 dovecot-lda CALL open(0x8021f5f68,0x2,0xfac3c0) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo" 44973 dovecot-lda RET open 9 [Open the mbox file itself] 44973 dovecot-lda CALL fcntl(0x9,F_SETLKW,0xffffffffffffcfd0) 44973 dovecot-lda RET fcntl 0 [Set a fcntl lock on the mbox file] 44973 dovecot-lda CALL pread(0,0x802031000,0x1000,0) 44973 dovecot-lda RET pread 43/0x2b 44973 dovecot-lda CALL write(0x9,0x802228000,0xf5) 44973 dovecot-lda RET write 245/0xf5 44973 dovecot-lda CALL fsync(0x9) 44973 dovecot-lda RET fsync 0 [Read from stdin and write to the mbox file] 44973 dovecot-lda CALL lstat(0x802028440,0x7fffffffd010) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat 0 44973 dovecot-lda CALL unlink(0x802028440) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET unlink 0 [Remove the .lock file] 44973 dovecot-lda CALL fcntl(0x9,F_SETLK,0xffffffffffffd160) 44973 dovecot-lda RET fcntl 0 [Clear the fcntl lock] 44973 dovecot-lda CALL close(0x9) 44973 dovecot-lda RET close 0 [Close the mbox file] > This is what allows > Dovecot to perform 'simultaneous' reads/writes to an mbox file, avoiding > filesystem level locking latency. Using filesystem level locking to > control read/write access between processes of own's program would be > insane on many levels. I'm not sure what you mean by 'processes of [one's own] program' but it's extremely common for a process to have to take locks against another copy of itself. All traditional Unix LDAs and MUAs do this; for instance, procmail will take locks in part so that if another instance of procmail is delivering another mail to the same user at the same time the mbox won't end up corrupted. Ben From ben at morrow.me.uk Fri Oct 26 07:16:28 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 05:16:28 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> Message-ID: <20121026041628.GH5388@anubis.morrow.me.uk> At 1AM +0300 on 26/10/12 you (Robert JR) wrote: > On 2012-10-26 00:15, Ben Morrow wrote: > > > As Stan said earlier, this is a Postfix question. The rule for [Looking back at the thread it wasn't Stan, it was Dennis Guhl. Sorry about that.] > > dotlocking is that you must create the .lock *before* opening the > > file, in case whoever has it locked will be replacing the file > > altogether; but with fcntl locking you must acquire the lock *after* > > opening the file, since that's the way the syscall works. This means > > that if Postfix is going to use both forms of lock, it has to > > acquire a dotlock before it can look for a fcntl lock. > > > > In other words: the methods in mailbox_delivery_lock are *not* tried > > in order, because they can't be. Dotlock is always tried first. > > > > You should have compatible locking settings for all your programs > > accessing your mboxes. If Postfix is using dotlock, Dovecot should be > > using dotlock as well. If you don't have any local programs (mail > > clients, for instance) which require dotlocks, you should probably > > change Postfix to just use fcntl locks. > > Thanks Ben for your valuable support and detailed explanation .. but > according to your explanation dovecot documentation contains wrong > explanation to the dotlock and that's why i was confused. > > it is written as follow > > ** > If multiple lock methods are used, which is usually the case since > dotlocks aren't typically used for read locking, the order in which the > locking is done is important. Consider if two programs were running at > the same time, both use dotlock and fcntl locking but in different > order: > > Program A: fcntl locks the mbox > Program B at the same time: dotlocks the mbox > Program A continues: tries to dotlock the mbox, but since it's already > dotlocked by B, it starts waiting > Program B continues: tries to fcntl lock the mbox, but since it's > already fcntl locked by A, it starts waiting > Now both of them are waiting for each others locks. Finally after a > couple of minutes they time out and fail the operation. > ** > > So this means that the documentation as mentioned above is wrong and > dotlock is always first even if the order is different .. I just checked, and you are right: Dovecot *will* use the locking strategies in the order listed. This is different from Postfix, which *will not*, so the 'Postfix' section of the MboxLocking page on the wiki is incorrect. For compatibility with Postfix, dotlock should always be listed first. Note that you will still frequently see Postfix acquiring a dotlock but then failing to acquire a fcntl lock. Dovecot's mbox_read_locks is usually set to just 'fcntl', which means that when Dovecot has an mbox open read-only it won't take a dotlock. This isn't really a problem, assuming you don't use any programs locally which only take dotlocks; but if that is the case you might as well configure everything to just use fcntl locks, and forget dotlocks altogether. Stan's earlier point is fundamentally correct: if you can treat the Dovecot mailstore as a black box, with mail going in through the LDA and LMTP and mail coming out through POP and IMAP, your life will be much easier. Traditional Unix mailbox locking strategies are *completely* insane, and if all you are doing is delivering mail from Postfix and reading it from Dovecot it would be better to avoid them altogether, and switch to dbox if you can. However, if you have any other programs which touch the mail spool (local or NFS mail clients, deliveries through procmail) this may not be possible. Ben From stan at hardwarefreak.com Fri Oct 26 09:24:00 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 01:24:00 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026035458.GG5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> Message-ID: <508A2C80.6010803@hardwarefreak.com> On 10/25/2012 10:54 PM, Ben Morrow wrote: > At 7PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >> On 10/25/2012 4:24 PM, Ben Morrow wrote: >>> At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >>>> >>>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >>>> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >>>> the file locking issue. Thus it also increases throughput as lock >>>> latency is eliminated. >>> >>> Nonsense. deliver and imap are still separate processes accessing the >>> same mbox, so they still need to use locks. The only difference is that >>> since they are both dovecot programs, they will automatically be using >>> the *same* locking strategies, and things will Just Work. >> >> "Nonsense" implies what I stated was factually incorrect, which is not >> the case. There's a difference between factual incorrectness and simply >> staying out of the weeds. > > What you stated was factually incorrect. > >> If you want to get into the weeds, and have me call you out for >> "nonsense", LDA/deliver is not a separate UNIX process. The LDA code >> runs within the imap process for the given user. > > Nonsense. dovecot-lda runs in its own process, and does not involve the > imap process in any way. As such it has to do locking. You apparently know your tools better than I do. Neither ps nor top show a 'dovecot-lda' or similarly named process on my systems. When I send a test message from gmail through Postfix I only see CPU or memory activity in an imap process. When I close the MUA to end the imap processes and then send a test message I don't see any CPU or memory activity in any dovecot processes, only Postfix processes, including local, and spamd. So is devecot-lda running as a sub-process or thread of Postfix' local process? Or is it part of the 'dovecot' process, and the message goes through so quick that top doesn't show any CPU usage by the 'dovecot' process? > If I have the following in my dovecot.conf: ... ... > I'm not sure what you mean by 'processes of [one's own] program' but I.e. Dovecot has its own set of processes, Postfix has its processes, etc. With "one's one processes" I'd think it makes more sense to use IPC and other tricks to accomplish concurrent access to a file rather than filesystem locking features. > it's extremely common for a process to have to take locks against > another copy of itself. All traditional Unix LDAs and MUAs do this; for > instance, procmail will take locks in part so that if another instance > of procmail is delivering another mail to the same user at the same time > the mbox won't end up corrupted. I guess I've given MDAs w/mbox too much credit, without actually looking at the guts. Scalable databases such Oracle, db2, etc, are far more intelligent about this, and can have many thousands of processes reading and writing the same file concurrently, usually via O_DIRECT, not buffered IO, so they have complete control over IO. This is accomplished with a record lock manager and IPC, preventing more than one process from accessing one record concurrently, but allowing massive read/write concurrency to multiple records in a file. I'd think the same concurrency optimization could be done with Dovecot. However, as Timo has pointed out, so few people use mbox these days that he simply hasn't spent much, if any, time optimizing mbox. Implementing some kind of lock manager and client code just for mbox IO concurrency simply wouldn't be worth the time. Unless he's already done something similar with mdbox. If he has, maybe that could be 'ported' to mbox as well. But again, it's probably not worth the effort given the number of mbox users, and the fact that nobody is complaining about mbox performance. I'm certainly not. It works great here. -- Stan From jg at softjury.de Fri Oct 26 09:49:04 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Fri, 26 Oct 2012 08:49:04 +0200 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <508A3260.3060803@softjury.de> I didn't know ADs well, but...can't you simply add the Field? In LDAP it should be possible, if you use MS AD, i dunno. Am 25.10.2012 22:49, schrieb b m:> No AD doesn't have such a field, but I could use some unused field to > get what I want. Let's say set "Attribute1" to "group1". The problem is > how to get that info. I guess I have to edit dovecot-ldap.conf and put > in user_attrs something like that ",=acl_groups=Attribute1". Any > suggestions? From stan at hardwarefreak.com Fri Oct 26 10:05:31 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 02:05:31 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026041628.GH5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> <20121026041628.GH5388@anubis.morrow.me.uk> Message-ID: <508A363B.3040100@hardwarefreak.com> On 10/25/2012 11:16 PM, Ben Morrow wrote: > At 1AM +0300 on 26/10/12 you (Robert JR) wrote: >> On 2012-10-26 00:15, Ben Morrow wrote: >> >>> As Stan said earlier, this is a Postfix question. The rule for > > [Looking back at the thread it wasn't Stan, it was Dennis Guhl. Sorry > about that.] I prodded him a second time, might have been off-list, and he finally posted there. So call it a team effort. ;) Wietse has already replied, and in typical fashion, asked for "concrete" evidence that Postfix was performing fcntl before dotlock, because he obviously knows better than anyone that Postfix applies a dotlock first, which you already explained here. >>> dotlocking is that you must create the .lock *before* opening the >>> file, in case whoever has it locked will be replacing the file >>> altogether; but with fcntl locking you must acquire the lock *after* >>> opening the file, since that's the way the syscall works. This means >>> that if Postfix is going to use both forms of lock, it has to >>> acquire a dotlock before it can look for a fcntl lock. >>> >>> In other words: the methods in mailbox_delivery_lock are *not* tried >>> in order, because they can't be. Dotlock is always tried first. >>> >>> You should have compatible locking settings for all your programs >>> accessing your mboxes. If Postfix is using dotlock, Dovecot should be >>> using dotlock as well. If you don't have any local programs (mail >>> clients, for instance) which require dotlocks, you should probably >>> change Postfix to just use fcntl locks. > but if that is the case you might as well configure everything to just > use fcntl locks, and forget dotlocks altogether. Yep. Postfix can use either or both. And, surprise, recommends using maildir to avoid mailbox locking entirely. > Stan's earlier point is fundamentally correct: if you can treat the > Dovecot mailstore as a black box, with mail going in through the LDA and > LMTP and mail coming out through POP and IMAP, your life will be much > easier. Traditional Unix mailbox locking strategies are *completely* > insane, and if all you are doing is delivering mail from Postfix and > reading it from Dovecot it would be better to avoid them altogether, and > switch to dbox if you can. However, if you have any other programs which > touch the mail spool (local or NFS mail clients, deliveries through > procmail) this may not be possible. And since this is a POP only server, users' MUAs should be deleting after download, so there shouldn't be much mail in these mbox files at any given time, making migration to maildir or dbox relatively simple. When using Dovecot LDA you'll eliminate the filesystem level locking problems with mbox. However, you may still have read/write contention within Dovecot, such as in your 20MB download as new mail arrives example, especially if the new message has an xx MB attachment. I don't believe Dovecot is going to start appending a new message while it's still reading out the existing 20MB of emails. Depending on how long this takes Dovecot may still issue a 4xx to Postfix, which will put the new message in the deferred queue. With maildir or dbox, reading existing mail and writing new messages occurs concurrently, as each message is a different file. -- Stan From ben at morrow.me.uk Fri Oct 26 11:11:20 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 09:11:20 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508A2C80.6010803@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> <508A2C80.6010803@hardwarefreak.com> Message-ID: <20121026081120.GI5388@anubis.morrow.me.uk> At 1AM -0500 on 26/10/12 you (Stan Hoeppner) wrote: > On 10/25/2012 10:54 PM, Ben Morrow wrote: > > > > dovecot-lda runs in its own process, and does not involve the > > imap process in any way. As such it has to do locking. > > You apparently know your tools better than I do. Neither ps nor top > show a 'dovecot-lda' or similarly named process on my systems. When I > send a test message from gmail through Postfix I only see CPU or memory > activity in an imap process. When I close the MUA to end the imap > processes and then send a test message I don't see any CPU or memory > activity in any dovecot processes, only Postfix processes, including > local, and spamd. So is devecot-lda running as a sub-process or thread > of Postfix' local process? Or is it part of the 'dovecot' process, and > the message goes through so quick that top doesn't show any CPU usage by > the 'dovecot' process? Assuming you have mailbox_command = /.../dovecot-lda -a "${RECIPIENT}" or something equivalent in your Postfix configuration, dovecot-lda runs as a subprocess of local(8) under the uid of the delivered-to user. > > If I have the following in my dovecot.conf: > ... > > ... > > > I'm not sure what you mean by 'processes of [one's own] program' but > > I.e. Dovecot has its own set of processes, Postfix has its processes, > etc. With "one's one processes" I'd think it makes more sense to use > IPC and other tricks to accomplish concurrent access to a file rather > than filesystem locking features. Filesystem locking, at least if NFS is not involved, is not that expensive. Successfully acquiring a flock or fcntl lock takes only a single syscall which doesn't have to touch the disk, and any form of IPC is going to need to do that. (Even something like a shared memory region will need a mutex for synchronisation, and acquiring the mutex has to go through the kernel.) Dotlocking *is* expensive, because acquiring a dotlock is a complicated process requiring lots of syscalls, some of which have to write to disk; and any scheme involving acquiring several locks on the same file is going to be more so, especially if you can end up getting the first lock but finding you can't get the second, so then you have to undo the first and try again. More importantly, the biggest problem with mbox as a mailbox format is that any access at all has to lock the whole mailbox. If the LDA is trying to deliver a new message at the same time as an IMAP user is fetching a completely different message, or if two instances of the LDA are trying to deliver at the same time, they will be competing for the same lock even though they don't really need to be. A file-per-message format like Maildir avoids this, to the point of being mostly lockless, but that brings its own efficiency problems; the point of dbox is to find the compromise between these positions that works best. > > it's extremely common for a process to have to take locks against > > another copy of itself. All traditional Unix LDAs and MUAs do this; for > > instance, procmail will take locks in part so that if another instance > > of procmail is delivering another mail to the same user at the same time > > the mbox won't end up corrupted. > > I guess I've given MDAs w/mbox too much credit, without actually looking > at the guts. I wouldn't look too hard at the details of the various ways there are of locking and parsing mbox files, or the ways in which they can go wrong. It's enough to make anyone swear off email for life :). > Scalable databases such Oracle, db2, etc, are far more > intelligent about this, and can have many thousands of processes reading > and writing the same file concurrently, usually via O_DIRECT, not > buffered IO, so they have complete control over IO. This is > accomplished with a record lock manager and IPC, preventing more than > one process from accessing one record concurrently, but allowing massive > read/write concurrency to multiple records in a file. I'd think the > same concurrency optimization could be done with Dovecot. > > However, as Timo has pointed out, so few people use mbox these days that > he simply hasn't spent much, if any, time optimizing mbox. Implementing > some kind of lock manager and client code just for mbox IO concurrency > simply wouldn't be worth the time. Unless he's already done something > similar with mdbox. If he has, maybe that could be 'ported' to mbox as > well. But again, it's probably not worth the effort given the number of > mbox users, and the fact that nobody is complaining about mbox > performance. I'm certainly not. It works great here. The only reason for using mbox is for compatibility with other systems which use mbox, which means you have to do the locking the same way as they do (assuming you can work out what that is). If you're going to change the locking rules you might as well change the file format at the same time, both to remove the insanity and to make it actually suitable for use as an IMAP mailstore. That's what Timo did with dbox, so if you've got your systems to the point where nothing but Dovecot touches the mail files you should seriously consider switching. Ben From r.ordinas at math.univ-paris-diderot.fr Fri Oct 26 11:51:52 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Fri, 26 Oct 2012 10:51:52 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121025140852.GA15639@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> Message-ID: <508A4F28.80606@math.univ-paris-diderot.fr> Hi, Here's the doveconf -n output : # doveconf -n # 2.0.14: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.1-RELEASE-p5 amd64 auth_mechanisms = plain login auth_username_format = %Lu auth_worker_max_count = 90 default_process_limit = 1024 first_valid_gid = 1500 first_valid_uid = 1500 hostname = mailhost.mydomain.tld last_valid_gid = 1500 last_valid_uid = 1500 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * mail_gid = 1500 mail_location = maildir:~/Maildir mail_plugins = acl quota mail_log notify mail_privileged_group = mail mail_uid = 1500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/usr/local/etc/dovecot-acls:cache_secs=300 autocreate = Sent autocreate1 = Trash autocreate2 = Drafts autocreate3 = Spam autocreate4 = Faux-positif autosubscribe = Sent autosubscribe1 = Trash autosubscribe2 = Drafts autosubscribe3 = Spam autosubscribe4 = Faux-positif autosubscribe5 = INBOX mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_rule = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 quota_warning2 = storage=80%% quota-warning 80 sieve = ~/.dovecot.sieve sieve_before = /usr/local/lib/dovecot/sieve/backup-all.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at mydomain.tld protocols = imap lmtp sieve quota_full_tempfail = yes service anvil { client_limit = 3500 } service auth-worker { user = $default_internal_user } service auth { client_limit = 5500 unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = doveauth } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 172.0.0.1 port = 2525 } } service managesieve-login { inet_listener sieve_deprecated { port = 2000 } process_limit = 1024 } service managesieve { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } shutdown_clients = no ssl = required ssl_ca = On Thu, Oct 25, 2012 at 03:09:47PM +0200, Raphael Ordinas wrote: >> When sending mail to MTA (in case of sieve filter forwarding for >> example), dovecot pass a RCPT TO command just after the EHLO. He's >> missing the MAIL FROM command. >> Therefore, my MTA show me a warning like this : "improper command >> pipelining after EHLO". > Works for me with latest and greatest although I'm not using the > 'submission_host' option but pure LMTP Unix socket: > > [...] > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > > Best to show your 'doveconf -n' for more thoughts. > > Regards > Thomas From ben at morrow.me.uk Fri Oct 26 11:54:56 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 09:54:56 +0100 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <20121026085456.GJ5388@anubis.morrow.me.uk> At 1PM -0700 on 25/10/12 b m wrote: > From: Jan Phillip Greimann >> Am 25.10.2012 00:13, schrieb b m: >> >> > Currently I have dovecot working with Active Directory >> > authentication and public folders with acl. In acl I have the users >> > I want to access the public folders. It'll be easier for me to use >> > one group instead of 50 users but I can't get it to work. From where >> > does dovecot get the "group" attribute for a user? Can it read the >> > groups that a user belongs from AD? >> >> ACL groups support works by returning a comma-separated acl_groups >> extra field from userdb, which contains all the groups the user >> belongs to. >> >> It seems to be possible, I had an acl_groups field in my MySQL >> Database for this, I'am sure it is something like that in an AD too. > > No AD doesn't have such a field, but I could use some unused field to > get what I want. Let's say set "Attribute1" to "group1". The problem > is how to get that info. I guess I have to edit dovecot-ldap.conf and > put in user_attrs something like that ",=acl_groups=Attribute1". Any > suggestions? That's the wrong way around. Assuming you created an 'imapGroups' attribute containing a comma-separated list of IMAP groups, you would want to add 'imapGroups=acl_groups' to user_attrs. Alternatively, if you don't want to duplicate the information in the LDAP directory, you can use post-login scripting to set up the groups list (see http://wiki2.dovecot.org/PostLoginScripting). If you have your system set up with nss_ldap or winbind so that AD users show up as system users with their proper groups, the example on the wiki using the 'groups' command will work. Otherwise, you can pull the information directly from LDAP, something like #!/bin/sh do_ldap () { /usr/local/bin/ldapsearch -h \ "(&(objectClass=$1)($2))" $3 \ | sed -ne"s/^$3: //p" } user_dn="$(do_ldap User "sAMAccountName=$USER" dn)" ACL_GROUPS="$(do_ldap Group "member=$user_dn" cn | paste -sd, -)" export ACL_GROUPS export USERDB_KEYS="$USERDB_KEYS acl_groups" exec "$@" Obviously you will need to adjust the path and connection parameters for ldapsearch to suit your environment; also, I don't use AD, so you may need to adjust the LDAP search. (If you prefer it might be easier to do this in Perl or Python or something rather than shell.) Ben From tlx at leuxner.net Fri Oct 26 12:00:12 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 26 Oct 2012 11:00:12 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <508A4F28.80606@math.univ-paris-diderot.fr> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> Message-ID: <20121026090012.GA31929@nihlus.leuxner.net> On Fri, Oct 26, 2012 at 10:51:52AM +0200, Raphael Ordinas wrote: > service lmtp { > inet_listener lmtp { > address = 172.0.0.1 > port = 2525 > } > } Right, so you are using network sockets with LMTP. Probably does not answer the question why it is not working with the 'submission_host', but is there a reason why the redirects are not reinjected this way? > submission_host = smtp.mydomain.tld Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From busseniu at in.tum.de Fri Oct 26 12:17:44 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Fri, 26 Oct 2012 11:17:44 +0200 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 In-Reply-To: <5085593D.3080403@in.tum.de> References: <5085593D.3080403@in.tum.de> Message-ID: <508A5538.8080604@in.tum.de> Hi, On 22.10.2012 16:33, Christoph Bu?enius wrote: > . list "" INBOX.shared.%.% > > Dovecot 2.1.10 does not list any folders in response to this command. I hope this helps: I bisected this bug and found it was introduced with this changeset: http://hg.dovecot.org/dovecot-2.1/rev/a41f64348d0d changeset: 14453:a41f64348d0d user: Timo Sirainen date: Fri Apr 20 15:18:14 2012 +0300 files: src/lib-storage/list/mailbox-list-fs-iter.c description: layout=fs: Don't assume '/' hierarchy separator when finding mailbox roots. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From tlx at leuxner.net Fri Oct 26 12:31:34 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 26 Oct 2012 11:31:34 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121026090012.GA31929@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> <20121026090012.GA31929@nihlus.leuxner.net> Message-ID: <20121026093134.GB31929@nihlus.leuxner.net> On Fri, Oct 26, 2012 at 11:00:12AM +0200, Thomas Leuxner wrote: > submission_host = smtp.mydomain.tld On second thought, above probably overrides this: # doveconf -a | grep sendmail sendmail_path = /usr/sbin/sendmail ...which may be the culprit. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From r.ordinas at math.univ-paris-diderot.fr Fri Oct 26 12:59:06 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Fri, 26 Oct 2012 11:59:06 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121026090012.GA31929@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> <20121026090012.GA31929@nihlus.leuxner.net> Message-ID: <508A5EEA.2000705@math.univ-paris-diderot.fr> Actually, LMTP inet listener is only used for delivery purpose. I separated the MTA and the MDA on distinct hosts. Incomming mails are received by the MTA which proceed to some check (anti-virus, spams, and aliases) and transport them to the MDA with LMTP. Maybe I misunderstood something, but i don't see why LMTP is involve in a sieve forwarding process (or stuff like non delivery mail return) . According to comments in the "15-lda.conf" file : # Binary to use for sending mails. #sendmail_path = /usr/sbin/sendmail # If non-empty, send mails via this SMTP host[:port] instead of sendmail. submission_host = smtp.mydomain.tld If you don't use the 'submission_host' option, dovecot will forward mail with '/usr/sbin/sendmail' binary which use the forwarders you tell it to use, am i right ? Regards, Raphael Le 26/10/2012 11:00, Thomas Leuxner a ?crit : > On Fri, Oct 26, 2012 at 10:51:52AM +0200, Raphael Ordinas wrote: > >> service lmtp { >> inet_listener lmtp { >> address = 172.0.0.1 >> port = 2525 >> } >> } > Right, so you are using network sockets with LMTP. Probably does not > answer the question why it is not working with the 'submission_host', > but is there a reason why the redirects are not reinjected this way? > >> submission_host = smtp.mydomain.tld > Regards > Thomas From tss at iki.fi Fri Oct 26 13:07:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 26 Oct 2012 13:07:49 +0300 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 In-Reply-To: <508A5538.8080604@in.tum.de> References: <5085593D.3080403@in.tum.de> <508A5538.8080604@in.tum.de> Message-ID: On 26.10.2012, at 12.17, Christoph Bu?enius wrote: > On 22.10.2012 16:33, Christoph Bu?enius wrote: >> . list "" INBOX.shared.%.% >> >> Dovecot 2.1.10 does not list any folders in response to this command. > > I hope this helps: I bisected this bug and found it was introduced with this changeset: > > http://hg.dovecot.org/dovecot-2.1/rev/a41f64348d0d I couldn't reproduce this exactly and I don't see how a41f64348d0d makes any difference .. but I did find another way to reproduce at least a similar bug. Maybe this fixes your problem too? http://hg.dovecot.org/dovecot-2.1/rev/22875bcaa952 From fabio.ferrari at unimore.it Fri Oct 26 13:24:42 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Fri, 26 Oct 2012 12:24:42 +0200 Subject: [Dovecot] Dovecot stops to work - anvil problem Message-ID: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> Hi all, we have a problem about anvil, it seems that when we have a high load the dovecot stops to work. Sometimes it is sufficient to make a dovecot reload, but sometimes we have to restart it. These are the lines related to anvil in the dovecot.log: [root at secchia ~]# grep anvil /var/log/dovecot.log | more Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files Oct 26 11:14:32 imap-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:32 imap-login: Fatal: Couldn't connect to anvil Oct 26 11:14:33 pop3-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:33 pop3-login: Fatal: Couldn't connect to anvil [...] (many lines like these) Oct 26 12:01:10 pop3-login: Fatal: Couldn't connect to anvil Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused And this is the output of the doveconf -n: [root at secchia ~]# doveconf -n # 2.0.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.11.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.8 (Tikanga) xfs auth_cache_size = 1024 auth_cache_ttl = 21600 s auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no hostname = mail.unimore.it info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/cl/mail/vhosts/sms.unimo.it/%Ln/Maildir mail_plugins = $mail_plugins quota mailbox_idle_check_interval = 60 s mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot.masterusers driver = passwd-file master = yes } passdb { args = dovecot driver = pam } plugin { quota = maildir:User quota quota_exceeded_message = Quota exceeded (mailbox is full) quota_rule = *:storage=200MB quota_rule2 = *:messages=100000 quota_rule3 = INBOX.Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = messages=95%% quota-warning 95 %u quota_warning4 = messages=80%% quota-warning 80 %u setting_name = quota } postmaster_address = postmaster at unimore.it quota_full_tempfail = yes service anvil { client_limit = 199999 process_limit = 199999 } service auth { client_limit = 14500 unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } process_limit = 5000 } service imap { process_limit = 5000 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 } } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail user = vmail } user = dovecot } ssl_ca = References: <5085593D.3080403@in.tum.de> <508A5538.8080604@in.tum.de> Message-ID: <508A668D.4070505@in.tum.de> Hello Timo, On 26.10.2012 12:07, Timo Sirainen wrote: > but I did find another way to reproduce at least a similar bug. Maybe this fixes your problem too? http://hg.dovecot.org/dovecot-2.1/rev/22875bcaa952 That does fix the problem, thank you! Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From dale.gallagher at gmail.com Fri Oct 26 14:27:00 2012 From: dale.gallagher at gmail.com (Dale Gallagher) Date: Fri, 26 Oct 2012 13:27:00 +0200 Subject: [Dovecot] dovecot-lda delivery to Maildir/cur as 'seen'? Message-ID: Hi I've added a server-side feature where authenticated customers sending through our SMTP server have their outbound mail copied to their Sent folder (like Gmail does). The delivery script called by qmail calls dovecot-lda to deliver it to the user's Sent folder. The problem now, is that the Sent folder shows the mail as unread, which MUAs flag (and notify, in the case of some). I've searched the docs and mailing list, but can't find an option to tell dovecot-lda to mark the mail being delivered, as seen/read. If I've missed something, please let me know. If not, then I think it might be a good idea to add a feature to dovecot-lda permitting one to specify delivery to the ./cur subfolder of a Maildir, instead of ./new. Thanks From tony.blue.mailinglist at gmx.de Fri Oct 26 14:44:55 2012 From: tony.blue.mailinglist at gmx.de (tony.blue.mailinglist at gmx.de) Date: Fri, 26 Oct 2012 13:44:55 +0200 Subject: [Dovecot] dovecot lda - Permission denied Message-ID: <20121026114455.30440@gmx.net> Hallo, please excuse my bad english. But I am not a native speaker. I changed my cyrus to dovecot (alltogehter: fetchmail - procmail - exim4 - dovecot). But I get (I think from /usr/lib/dovecot/deliver) the following error-message in my syslog: ... Oct 25 23:37:13 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(andy) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) ... Oct 25 23:37:14 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=500(tony) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) ... Dovecot is installed as !include auth-passwdfile.conf.ext. For all users there is a entry in der /etc/dovecot/users. Usaly the user rights are set to 600. I tryed 755, but I get the same errormessage. ... service auth { unix_listener auth-userdb { mode = 0755 user = mailstore group = mailstore } ... If I try "ls /var/run/dovecot/auth-userdb -la" - i get: srwxr-xr-x 1 mailstore mailstore 0 Okt 25 23:36 /var/run/dovecot/auth-userdb How can I solve this problem? Tony From CMarcus at Media-Brokers.com Fri Oct 26 16:37:59 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 26 Oct 2012 09:37:59 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5088B691.7030100@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> <5088B691.7030100@hardwarefreak.com> Message-ID: <508A9237.7080903@Media-Brokers.com> On 2012-10-24 11:48 PM, Stan Hoeppner wrote: > Changing the process priority would not help. Indexing a large mailbox > is an IO bound, not a compute bound, operation. With Linux, changing > from the CFQ to deadline scheduler may help some with low > responsiveness. But the only real solution for such a case where iowait > is bringing the system to its knees is to acquire storage with far > greater IOPS and concurrent IO capability. I.e. a server. Ok, I get it, thanks for elaborating Stan... -- Best regards, Charles From dg at dguhl.org Fri Oct 26 17:01:41 2012 From: dg at dguhl.org (Dennis Guhl) Date: Fri, 26 Oct 2012 16:01:41 +0200 Subject: [Dovecot] dovecot-lda delivery to Maildir/cur as 'seen'? In-Reply-To: References: Message-ID: <20121026140141.GA6769@PC211.ikt.de> On Fri, Oct 26, 2012 at 01:27:00PM +0200, Dale Gallagher wrote: > Hi [..] > The problem now, is that the Sent folder shows the mail as unread, > which MUAs flag (and notify, in the case of some). I've searched the Use Sieve [1] with Imap4flags (RFC 5232) to mark the email as read. Dennis [1] http://wiki2.dovecot.org/Pigeonhole/Sieve [..] From listen at mjh.name Fri Oct 26 21:28:51 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:28:51 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087D457.6040205@sys4.de> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087D457.6040205@sys4.de> Message-ID: <20121026202851.dc0abb3d5e4a4dd5c32d2d6c@mjh.name> On Wed, 24 Oct 2012 13:43:19 +0200 Robert Schetterer wrote: > Am 24.10.2012 13:28, schrieb Milan Holz?pfel: > > The whole mdbox is 6.6 GiB large and I guess that it contains about > > 300k-600k messages. It's an archive of public mailing lists, so I could > > give access to the files. > > > > Can anybody say something about this? May the mdbox be repaired? > > perhaps this helps > > http://wiki2.dovecot.org/Tools/Doveadm/ForceResync > > however upgrading to dovecot latest might be a good idea I tried this command, but all it will do is the "rebuilding indexes" thing that Dovecot's deliver and imapd will also do. (As I mentioned, this fails.) I haven't tried a more recent version of Dovecot so far. Regards, Milan Holz?pfel -- Milan Holz?pfel From listen at mjh.name Fri Oct 26 21:29:15 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:29:15 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087F4B4.2060107@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> Message-ID: <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> On Wed, 24 Oct 2012 09:01:24 -0500 Stan Hoeppner wrote: > On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: > > > I have a problem with an incosistent mdbox: > ... > > four hours after the problem initially appeared, I did a hard reset of > > the system because it was unresponsive. > ... > > Can anybody say something about this? May the mdbox be repaired? > > If the box is truly unresponsive, i.e. hard locked, then the corrupted > indexes are only a symptom of the underlying problem, which is unrelated > to Dovecot, UNLESS, the lack of responsiveness was due to massive disk > access, which will occur when rebuilding indexes on a 6.6GB mailbox. > You need to know the difference so we have accurate information to > troubleshoot with. Thanks for your suggestion. I wasn't looking for a solution for the unresponsiveness, but I failed to make that clear. I was not patient enough to debug the unresponsiveness issue. The box was not hard locked, but any command took very look if it would at all complete. I think that it could be massive swapping, but I wouldn't expect Dovecot to be the cause. After the reboot, Dovecot would happily re-execute the failing index rebuild on each new incoming message, which suggests that Dovecot wasn't the cause for the unresponsiveness. > If the there's a kernel or hardware problem, you should see related > errors in dmesg. Please share those. The kernel had messages like INFO: task cron:2799 blocked for more than 120 seconds. in the dmesg. But again, I didn't mean to ask for a solution to this problem. Regards, Milan Holz?pfel -- Milan Holz?pfel From listen at mjh.name Fri Oct 26 21:30:24 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:30:24 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> On Wed, 24 Oct 2012 13:28:11 +0200 Milan Holz?pfel wrote: > I have a problem with an incosistent mdbox: > [...] > The problem appeared out of nowhere. [...] That's just wrong. Two minutes before the corruption occured for the first time, the machine was booted after power-off without prior shutdown. I didn't notice this until now, sorry for this. The mailbox is on XFS. As far as I remember, XFS in known for leaving NULL bytes at the end of files after a system reset. At least, I found 72 bytes of NULL in a plain text log file on XFS after such an event. Do you think this may be the source of the index corruption? Do you have any other suggestions for recovering the mailbox? Regards, Milan Holz?pfel -- Milan Holz?pfel From fxmulder at gmail.com Fri Oct 26 22:13:33 2012 From: fxmulder at gmail.com (James Devine) Date: Fri, 26 Oct 2012 13:13:33 -0600 Subject: [Dovecot] Overlapping userdb/passdbs Message-ID: I have an ldap server for which each entry includes the email address and the username portion of the email address for authentication. Authentication works by username if the username is unique among all the entries. I need to now add some users which must authenticate even if the username is not unique. I figured one way to do this would be to add a second user/pass db which puts further restrictions on the ldap query to make it unique for those users. This doesn't seem to work however as if the user is found in the first ldap query but the password does not match it does not try the second. I would use the password as part of the query but this setup requires me to allow the client to hash the password. Is there a way to do this? Or maybe I am approaching the problem wrong. From mike at alaadin.org Fri Oct 26 21:47:44 2012 From: mike at alaadin.org (Mike John) Date: Fri, 26 Oct 2012 21:47:44 +0300 Subject: [Dovecot] Changing password for users In-Reply-To: <02a35ba19c559b258dba0de278e31a4d@coptics.org> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> Message-ID: <7827e2e2d9aa524945d00575c3366400@coptics.org> On 2012-10-26 01:17, Mike John wrote: >> Hello, I am using dovecot (2.0.9) and using virtual users using >> passdb >> { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i >> make my virtual users change their passwords using web interface ? >> My >> users already uses squirrelmail to access their mail. is there a >> program to add to squirrelmail to add this function to the clients ? >> or >> should i user different separate website for password changing ? and >> what program/tool can help me with this ? Any ideas is greatly >> appreciated. Mike. Mike, > >> I don't know about forcing users to change their passwords however >> with >> Squirrelmail there are several password change plugins available >> that >> use "poppasswd" to actually c> ssword. Of course poppasswd will > probably need to be modified to go >> against your password data base, in my case it simply uses PAM. The >> version I> sion 1.8.5. Oh you probably want to restrict access to >> the > port from >> the local host only since pas > ansmitted in clear > >> quot > e>Jeff > > I know about poppassd , but it works only for /etc/passwd , > /etc/shadow, but my dovecot virtual users password files > are in different location and i do not know how to modify poppassd, > any > idea how can i do that? and is there another way other than poppassd? i have googled every where, i can not find how to modify poppassd to modify virtual users passwords at /etc/dovecot/passwords , Is there any other way ? i am sure that some one in this mailing list have virtual users and uses modified poppassd or other utils so that his clients can change their password From dave.mehler at gmail.com Fri Oct 26 23:34:46 2012 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 26 Oct 2012 16:34:46 -0400 Subject: [Dovecot] public mailbox not showing up in web client Message-ID: Hello, I'm trying to set up a public mailbox where users can receive notifications out of. I'm not getting any errors from Dovecot 2.1, but nothing is showing up in my user's web clients. In each /home/vmail/public/mailbox folder right now I just have one called testbox I have a dovecot-acl file with: user=testuser1 lr user=user1 lr etc. I'd appreciate any suggestions. Thanks. Dave. # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 hostname = xxx last_valid_gid = 5000 last_valid_uid = 5000 log_path = /var/log/dovecot.error mail_gid = vmail mail_home = /home/vmail/%d/%n/home mail_location = maildir:/home/vmail/%d/%n:LAYOUT=fs mail_plugins = " acl quota zlib" mail_uid = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = yes location = maildir:/home/vmail/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace inbox { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile autocreate = Spam autosubscribe = Spam quota = dict:User quota::proxy::quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } postmaster_address = postmaster at xxx protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = xxx xxx ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_cert = References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> <7827e2e2d9aa524945d00575c3366400@coptics.org> Message-ID: <508AFACD.8050807@whyscream.net> On 26-10-12 20:47, Mike John wrote: > On 2012-10-26 01:17, Mike John wrote: > >>> Hello, I am using dovecot (2.0.9) and using virtual users using passdb >>> { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i >>> make my virtual users change their passwords using web interface ? My >>> users already uses squirrelmail to access their mail. is there a >>> program to add to squirrelmail to add this function to the clients ? or >>> should i user different separate website for password changing ? and >>> what program/tool can help me with this ? Any ideas is greatly >>> appreciated. Mike. Mike, >> >>> I don't know about forcing users to change their passwords however with >>> Squirrelmail there are several password change plugins available that >>> use "poppasswd" to actually c> ssword. Of course poppasswd will >> probably need to be modified to go >>> against your password data base, in my case it simply uses PAM. The >>> version I> sion 1.8.5. Oh you probably want to restrict access to the >> port from >>> the local host only since pas >> ansmitted in clear >> >>> quot >> e>Jeff >> >> I know about poppassd , but it works only for /etc/passwd , >> /etc/shadow, but my dovecot virtual users password files >> are in different location and i do not know how to modify poppassd, any >> idea how can i do that? and is there another way other than poppassd? > > i have googled every where, i can not find how to modify poppassd to > modify virtual users passwords at /etc/dovecot/passwords > , Is there any other way ? i am sure that some one in this mailing list > have virtual users and uses modified poppassd or other utils so that his > clients can change their password Using a database for managing virtual users seems overkill, until you run into issues like this. I have a postgres backend for 20ish users, and I can plugin everything I want. Postfixadmin works geat, and there are many password plugins for squirrelmail/roundcube/etc that work with such a database. Disclaimer: I tried the file-based approach too, but kept building kludges for things that were a lot simpler with a database. In the end, I joined the dark side. -- Tom From jtam.home at gmail.com Sat Oct 27 01:19:31 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 26 Oct 2012 15:19:31 -0700 (PDT) Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: > From: Mike John > >> I know about poppassd , but it works only for /etc/passwd , >> /etc/shadow, but my dovecot virtual users password files >> are in different location and i do not know how to modify poppassd, >> any idea how can i do that? I downloaded and examined it; it's just a wrapper for /usr/bin/passwd, and there doesn't seem an easy way to modify it to use something other than the system password file. Maybe replace "/usr/bin/passwd" with htpasswd? > and is there another way other than poppassd? Write your own PHP script -- it couldn't be more than a few dozen lines of code for a working skeleton. Or Google "php change password htpasswd". Joseph Tam From ben at morrow.me.uk Sat Oct 27 02:09:11 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Sat, 27 Oct 2012 00:09:11 +0100 Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: <20121026230910.GK5388@anubis.morrow.me.uk> At 3PM -0700 on 26/10/12 you (Joseph Tam) wrote: > > > From: Mike John > > > >> I know about poppassd , but it works only for /etc/passwd , > >> /etc/shadow, but my dovecot virtual users password files > >> are in different location and i do not know how to modify poppassd, > >> any idea how can i do that? > > I downloaded and examined it; it's just a wrapper for /usr/bin/passwd, > and there doesn't seem an easy way to modify it to use something other > than the system password file. > > Maybe replace "/usr/bin/passwd" with htpasswd? Try pam_pwdfile with poppwd or some other poppassd that supports PAM. > > and is there another way other than poppassd? > > Write your own PHP script -- it couldn't be more than a few dozen lines > of code for a working skeleton. Or Google "php change password htpasswd". It's not as simple as you seem to think. Quite apart from getting the password-changing itself right (have you considered what happens when two users change their passwords at the same time? when Dovecot tries to read the password file at the same time as you are changing it? when the system crashes when you are halfway through rewriting the password file?), you really shouldn't be running PHP as a user with write access to a password file (even a virtual password file) in any case. Ben From rob0 at gmx.co.uk Sat Oct 27 03:26:46 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Fri, 26 Oct 2012 19:26:46 -0500 Subject: [Dovecot] Changing password for users In-Reply-To: <508AFACD.8050807@whyscream.net> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> <7827e2e2d9aa524945d00575c3366400@coptics.org> <508AFACD.8050807@whyscream.net> Message-ID: <20121027002646.GS3672@harrier.slackbuilds.org> On Fri, Oct 26, 2012 at 11:04:13PM +0200, Tom Hendrikx wrote: > Using a database for managing virtual users seems overkill, > until you run into issues like this. > > I have a postgres backend for 20ish users, and I can plugin > everything I want. Postfixadmin works geat, and there are many > password plugins for squirrelmail/roundcube/etc that work with > such a database. > > Disclaimer: I tried the file-based approach too, but kept > building kludges for things that were a lot simpler with a > database. In the end, I joined the dark side. SQLite gives me the best of both worlds: file-based stability with SQL flexibility and easy backups. There is no Postfixadmin-type solution out there yet, but if you're fine with sqlite3(1) in the console, you won't miss it. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Sat Oct 27 03:49:32 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 19:49:32 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026081120.GI5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> <508A2C80.6010803@hardwarefreak.com> <20121026081120.GI5388@anubis.morrow.me.uk> Message-ID: <508B2F9C.2050706@hardwarefreak.com> You are a well of accessible knowledge Ben. (How have I missed your posts in the past?) On 10/26/2012 3:11 AM, Ben Morrow wrote: > Assuming you have > > mailbox_command = /.../dovecot-lda -a "${RECIPIENT}" I'm setup for system users so it's a simpler, but yes. > or something equivalent in your Postfix configuration, dovecot-lda runs > as a subprocess of local(8) under the uid of the delivered-to user. Of course that makes sense given Postfix is doing the calling. I would have assumed this but my feeble use of tools wasn't showing anything. > Filesystem locking, at least if NFS is not involved, is not that > expensive. Successfully acquiring a flock or fcntl lock takes only a > single syscall which doesn't have to touch the disk, and any form of IPC > is going to need to do that. (Even something like a shared memory region > will need a mutex for synchronisation, and acquiring the mutex has to go > through the kernel.) Thanks for this. I was under the assumption flock/fcntl were more expensive than they are. Probably because all I'd read about them was in relation to NFS (which I don't use, but I read alot like many do). > Dotlocking *is* expensive, because acquiring a dotlock is a complicated > process requiring lots of syscalls, some of which have to write to disk; > and any scheme involving acquiring several locks on the same file is > going to be more so, especially if you can end up getting the first lock > but finding you can't get the second, so then you have to undo the first > and try again. Yeah, I knew dotlocks were the worst due to disk writes, but didn't know the other details. > More importantly, the biggest problem with mbox as a mailbox format is > that any access at all has to lock the whole mailbox. If the LDA is > trying to deliver a new message at the same time as an IMAP user is > fetching a completely different message, or if two instances of the LDA > are trying to deliver at the same time, they will be competing for the > same lock even though they don't really need to be. A file-per-message > format like Maildir avoids this, to the point of being mostly lockless, > but that brings its own efficiency problems; the point of dbox is to > find the compromise between these positions that works best. mbox locking hasn't been problem here as I split the INBOX from the user mailboxes containing IMAP folders (mbox files). We make heavy use of sieve scripts to sort on delivery, so there's not much concurrent access to any one mbox file. The efficiency issue is why I chose mbox over maildir. Users here keep a lot of (list) mail and FTS often. The load on the spindles with maildir is simply too great and would bog down all users. The IOPS benefit of mbox in this scenario outweighs any locking issues. > I wouldn't look too hard at the details of the various ways there are of > locking and parsing mbox files, or the ways in which they can go wrong. > It's enough to make anyone swear off email for life :). Heheh. > The only reason for using mbox is for compatibility with other systems > which use mbox, Not necessarily true. See above. I'm sure I'm not the only one using mbox for this reason. Dovecot is my only app hitting these mbox files. > which means you have to do the locking the same way as > they do (assuming you can work out what that is). If you're going to > change the locking rules you might as well change the file format at the > same time, both to remove the insanity and to make it actually suitable > for use as an IMAP mailstore. That's what Timo did with dbox, so if > you've got your systems to the point where nothing but Dovecot touches > the mail files you should seriously consider switching. If/when I do switch mailbox formats it'll be to mdbox so FTS doesn't drop a big hammer on the spindles. Thanks for the informative discussion Ben. -- Stan From stan at hardwarefreak.com Sat Oct 27 04:45:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 20:45:34 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> Message-ID: <508B3CBE.1000004@hardwarefreak.com> On 10/26/2012 1:29 PM, Milan Holz?pfel wrote: > On Wed, 24 Oct 2012 09:01:24 -0500 > Stan Hoeppner wrote: > >> On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: >> >>> I have a problem with an incosistent mdbox: >> ... >>> four hours after the problem initially appeared, I did a hard reset of >>> the system because it was unresponsive. >> ... >>> Can anybody say something about this? May the mdbox be repaired? >> >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Thanks for your suggestion. I wasn't looking for a solution for the > unresponsiveness, but I failed to make that clear. It's likely all related. If you have already, or will continue to, hard reset the box, you will lose inflight data in the buffer cache, which may very likely corrupt your mdbox files and/or indexes. I'm a bit shocked you'd hard reset a *slow* responding server. Especially one that appears to be unresponsive due to massive disk IO. That's a recipe for disaster... > I was not patient enough to debug the unresponsiveness issue. The box > was not hard locked, but any command took very look if it would at all > complete. I think that it could be massive swapping, but I wouldn't > expect Dovecot to be the cause. This leads me to believe your filesystem root, swap partition, and Dovecot mailbox storage are all on the same disk, or small RAID set. Is this correct? > After the reboot, Dovecot would happily re-execute the failing index > rebuild on each new incoming message, which suggests that Dovecot > wasn't the cause for the unresponsiveness. This operation is a tiny IO pattern compared to the 6.6GB re-indexing operation you mentioned before. So you can't make the simple assumption that "Dovecot wasn't the cause for the unresponsiveness". If fact Dovecot likely instigated the problem, though it likely isn't the "cause". I'll take a stab at that below. >> If the there's a kernel or hardware problem, you should see related >> errors in dmesg. Please share those. > > The kernel had messages like > > INFO: task cron:2799 blocked for more than 120 seconds. Now we're getting some meat on this plate. > in the dmesg. But again, I didn't mean to ask for a solution to this > problem. "blocked for more than 120 seconds" is a kernel warning message, not an error message. We see this quite often on the XFS list. Rarely, this is related to a kernel bug. Most often the cause of this warning is saturated IO. In this case it appears cron blocked for 120s because it couldn't read /var/cron/crontabs/[user] The most likely cause of this is that so many IO requests are piled up in the queue that it took more than 2 minutes for the hardware (disks) to complete them before servicing the cron process' IO requests. Dovecot re-indexing a 6.6GB mailbox, with other IO occurring concurrently, could easily cause this situation if you don't have sufficient spindle IOPS. I.e. this IO pattern will bring a single SATA disk or mirror pair to its knees. If you currently have everything on a single SATA disk or mirror pair, the solution for eliminating the bogging down of the system, and likely the Dovecot issues related to it, is to simply separate your root filesystem, swap, and Dovecot data files onto different physical devices. For instance, moving the root filesystem and swap to a small SSD will prevent the OS unresponsiveness, even if Dovecot is bogged down with IO to the SATA disk. With spinning rust storage, separation of root filesystem, swap, and application data to different storage IO domains is system administration 101 kind of stuff. If you're using SSD this isn't (as) critical as it's pretty hard to saturate the IO limits of an SSD. -- Stan From stan at hardwarefreak.com Sat Oct 27 05:54:21 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 21:54:21 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> Message-ID: <508B4CDD.4070508@hardwarefreak.com> On 10/26/2012 1:30 PM, Milan Holz?pfel wrote: > On Wed, 24 Oct 2012 13:28:11 +0200 > Milan Holz?pfel wrote: > >> I have a problem with an incosistent mdbox: >> [...] >> The problem appeared out of nowhere. [...] > > That's just wrong. Two minutes before the corruption occured for > the first time, the machine was booted after power-off without prior > shutdown. I didn't notice this until now, sorry for this. Ahh, more critical information. Better late than never I guess. > The mailbox is on XFS. As far as I remember, XFS in known for leaving > NULL bytes at the end of files after a system reset. At least, I found > 72 bytes of NULL in a plain text log file on XFS after such an event. > Do you think this may be the source of the index corruption? Very possibly. > Do you have any other suggestions for recovering the mailbox? Other than restoring from a backup, I do not. Others might. But I will offer this suggestion: Never run a server without a properly functioning UPS and shutdown scripts. The system in question isn't a laptop is it? I'm trying to ascertain how many server 'rules' you're breaking before making any more assumptions or giving any more advice. -- Stan From bernics.gabor at penta.hu Sat Oct 27 10:52:29 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Sat, 27 Oct 2012 09:52:29 +0200 Subject: [Dovecot] mail open slowly Message-ID: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> Hello, I have a dovecot IMAP server (relative small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) with 100 Gbyte maildirs. Server works fine but sometimes I can open mails slowly (5-10 sec), typical with new mails. It's intresting when I open an another old mail (with 0 sec wait), after new mail open quickly. Is it dovecot mail indexes or I/O problem? Load and CPU use is small typical 0.10, 10%, I see small IO wait. Debian 6.0, Dovecot 1.2.15, fsync and nmap is disable Best Regards, Gabor From rs at sys4.de Sat Oct 27 12:36:03 2012 From: rs at sys4.de (Robert Schetterer) Date: Sat, 27 Oct 2012 11:36:03 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> Message-ID: <508BAB03.9050709@sys4.de> Am 27.10.2012 09:52, schrieb Bernics G?bor | Penta Uni? Zrt.: > > > Hello, > > I have a dovecot IMAP server (relative small hardware: HP > Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) > with 100 Gbyte maildirs. that seems ok > > Server works fine but sometimes I can open > mails slowly (5-10 sec), typical with new mails. what mailbox type, how much mail in that mailbox how many concurent cons, imap or pop3 ? > > It's intresting when I > open an another old mail (with 0 sec wait), after new mail open > quickly. may client cached > > Is it dovecot mail indexes or I/O problem? perhaps this , perhaps other > > Load and CPU use > is small typical 0.10, 10%, I see small IO wait. > > Debian 6.0, Dovecot > 1.2.15, fsync and nmap is disable show config and logs > > Best Regards, > > Gabor > > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From bernics.gabor at penta.hu Sat Oct 27 14:58:27 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Sat, 27 Oct 2012 13:58:27 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <508BAB03.9050709@sys4.de> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> Message-ID: <5f981c03d54f00df233f82495df72022@penta.hu> Thank you I use dovecot LDA (+sieve) with maildir. conf: http://pastebin.com/9fhYD58g logs: http://pastebin.com/CXct3B6k connections: http://pastebin.com/v24iRz60 "It's intresting when I open an another old mail (with 0 sec wait), after new mail open quickly. may client cached" it's possible. 2012-10-27 11:36 id?pontban Robert Schetterer ezt ?rta: > Am 27.10.2012 09:52, schrieb Bernics G?bor | Penta Uni? Zrt.: > >> Hello, I have a dovecot IMAP server (relative small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) with 100 Gbyte maildirs. > > that seems ok > >> Server works fine but sometimes I can open mails slowly (5-10 sec), typical with new mails. > > what mailbox type, how much mail in that mailbox > how many concurent cons, imap or pop3 ? > Is it dovecot mail indexes or I/O problem? > > perhaps this , perhaps other From rs at sys4.de Sat Oct 27 15:55:04 2012 From: rs at sys4.de (Robert Schetterer) Date: Sat, 27 Oct 2012 14:55:04 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <5f981c03d54f00df233f82495df72022@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> Message-ID: <508BD9A8.8010101@sys4.de> Am 27.10.2012 13:58, schrieb Bernics G?bor | Penta Uni? Zrt.: > > > Thank you > > I use dovecot LDA (+sieve) with maildir. > > conf: in general you should upgrade to 2.1.x with lmtp for better performance use auth cache http://wiki.dovecot.org/Authentication/Caching an example you find in your config > > > http://pastebin.com/9fhYD58g > > logs: > > http://pastebin.com/CXct3B6k i dont see imap here, do more verbose logging > > > connections: > > http://pastebin.com/v24iRz60 to much info for this stage how much traffic concurent in fail stage would be interesting any chance measure iops of the storage ? what mail client ? if tb do debug http://wiki.dovecot.org/Debugging/Thunderbird how much mail does this mailbox have > > "It's intresting when I > open an another old mail (with 0 sec wait), after new mail open quickly. > > > may client cached" > > it's possible. > > 2012-10-27 11:36 id?pontban Robert > Schetterer ezt ?rta: > >> Am 27.10.2012 09:52, schrieb Bernics G?bor | > Penta Uni? Zrt.: >> >>> Hello, I have a dovecot IMAP server (relative > small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with > 2x 7200 SATA disk) with 100 Gbyte maildirs. >> >> that seems ok >> >>> > Server works fine but sometimes I can open mails slowly (5-10 sec), > typical with new mails. >> >> what mailbox type, how much mail in that > mailbox >> how many concurent cons, imap or pop3 ? >> Is it dovecot mail > indexes or I/O problem? >> >> perhaps this , perhaps other > > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stan at hardwarefreak.com Sat Oct 27 21:27:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 27 Oct 2012 13:27:50 -0500 Subject: [Dovecot] mail open slowly In-Reply-To: <5f981c03d54f00df233f82495df72022@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> Message-ID: <508C27A6.30207@hardwarefreak.com> On 10/27/2012 6:58 AM, Bernics G?bor | Penta Uni? Zrt. wrote: > I use dovecot LDA (+sieve) with maildir. > > conf: > > http://pastebin.com/9fhYD58g Next time simply paste "dovecot -n" output into your email. Assuming Dovecot is the only program accessing the maildirs, try: maildir_very_dirty_syncs=yes That may help some. It may not have been a factor in this case, but note that when anyone is doing a full text search on a large mailbox on this hardware with maildir you will see latency, and it is unavoidable. Neither a single 7.2K SATA spindle nor md/RAID1 pair of them, has enough seek capacity to service all the sector requests in a timely fashion. Also, I noticed you disabled fsync. This is a very very bad idea for a mail server. If you lose power, or suffer a kernel/hardware/etc crash, you lose the Linux buffer cache contents. Thus, you may lose emails that haven't been flushed to disk, and possibly get index file corruption if mmap'd pages haven't been flushed. Running with fsync disabled is like having sex with a Bangkok prostitute without a condom while juggling chainsaws while driving drunk at 250kph at night without headlights. fsync does hurt write performance to a degree, especially with maildir storage, but will likely be invisible on a small server with few users/light load. And it will prevent potentially severe problems with file corruption and/or lost emails. -- Stan From stephan at rename-it.nl Sun Oct 28 00:03:20 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 27 Oct 2012 23:03:20 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5088E74C.9030006@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> <5088E74C.9030006@ngtech.co.il> Message-ID: <508C4C18.5010401@rename-it.nl> On 10/25/2012 9:16 AM, Eliezer Croitoru wrote: > My only problem is that it will put the file in the folder but will > not mark the folder with the new file until I actually check the > folder manually. > it's not that much hustle but if there is a way to solve it I will be > more then happy to hear about it. This is most likely a client problem. Have you configured your client to check that folder? Regards, Stephan. From dave at boostpro.com Sat Oct 27 23:00:16 2012 From: dave at boostpro.com (David Abrahams) Date: Sat, 27 Oct 2012 16:00:16 -0400 Subject: [Dovecot] When are search indexes updated? Message-ID: I noticed that occasionally searching in my huge archive mailbox can be really slow, so I tried doveadm index on it and it seemed to do a lot of work, which seemed strange given, for example, that dovecot-lda says it keeps Dovecot index files up-to-date. Then I thought, "maybe these are different files than the search indices." If so, that's not entirely clear from the docs and Wiki. So, questions: * When are search indexes updated? * Are they updated incrementally? * If not, why not? * If so, why would a mailbox's index drift out-of-date, as mine had? BTW, I'm using the clucene search backend. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From stan at hardwarefreak.com Sun Oct 28 04:46:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 27 Oct 2012 21:46:34 -0500 Subject: [Dovecot] When are search indexes updated? In-Reply-To: References: Message-ID: <508C9C8A.8000309@hardwarefreak.com> On 10/27/2012 3:00 PM, David Abrahams wrote: > > I noticed that occasionally searching in my huge archive mailbox can be > really slow, so I tried doveadm index on it and it seemed to do a lot of > work, which seemed strange given, for example, that dovecot-lda says it > keeps Dovecot index files up-to-date. Then I thought, "maybe these are > different files than the search indices." If so, that's not entirely > clear from the docs and Wiki. So, questions: Mailbox and search indexes are separate. Look in your mailbox directory and you'll see them, such as on 1.2.x with mbox: $ la /home/stan/mail/.imap/1-Dovecot total 3.4M drwx------ 2 stan stan 135 Oct 25 21:39 . drwx------ 51 stan stan 4.0K Apr 13 2012 .. -rw------- 1 stan stan 44K Oct 27 13:28 dovecot.index -rw------- 1 stan stan 1.2M Oct 27 21:23 dovecot.index.cache -rw------- 1 stan stan 18K Oct 27 21:23 dovecot.index.log -rw------- 1 stan stan 1.1M May 20 06:32 dovecot.index.search -rw------- 1 stan stan 1.1M May 20 06:32 dovecot.index.search.uids I've not full text searched this folder for quite some time, thus the search indexes are not current, and the next FTS of this mail folder will take much more time than if the FTS indexes were current. > * When are search indexes updated? When the index is stale. > * Are they updated incrementally? > * If not, why not? > * If so, why would a mailbox's index drift out-of-date, as mine had? When a sufficient number of messages are added to an IMAP folder the FTS index becomes stale. This index is not updated in real time. This is why Timo and others recommend cron'ing a script to index folders regularly that are searched regularly. This keeps the indexes up to date and keeps searches fast. If you don't do this or search often, your indexes become stale. Then each time you do an FTS search the first thing that happens is an FTS re-indexing of the mail folder. Only then does it display the search results. > BTW, I'm using the clucene search backend. I've not used Lucene, but I believe the default behavior is similar to the Dovecot 1.2.x FTS indexer. -- Stan From claude.xavier at gmail.com Sun Oct 28 11:28:28 2012 From: claude.xavier at gmail.com (Xavier Claude) Date: Sun, 28 Oct 2012 10:28:28 +0100 Subject: [Dovecot] How to activate antispam plugin Message-ID: <3146470.D7UsahtfX5@coruscant> Hello, I'm using dovecot 2.1.7 from the Debian backports package and I'm trying to get working the antispam plugin with dspam. I have followed the documentation http://wiki2.dovecot.org/Plugins/Antispam but it does not seem to work. The /var/log/dspam folder is empty even after I put mail in the Spam folder. How can I see if the plugin is working and what config option am I missing ? Thank in advance for your help. Here is my config from dovecot -n: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-xenU-6887-i386 i686 Debian 6.0.6 mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } plugin { antispam_backend = dspam antispam_dspam_args = --mode=teft;--deliver=;--user;%u antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = move antispam_spam = Spam antispam_trash = trash;Corbeille;Trash;Deleted Items; Deleted Messages sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } ssl_cert = References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> <5088E74C.9030006@ngtech.co.il> <508C4C18.5010401@rename-it.nl> Message-ID: <508D4F1F.4030603@ngtech.co.il> On 10/27/2012 11:03 PM, Stephan Bosch wrote: > > This is most likely a client problem. Have you configured your client to > check that folder? Yes unless there there is a special thing I dont know yet about in Thunderbird. Thanks, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From afodis.pinon at hotmail.fr Mon Oct 29 11:01:03 2012 From: afodis.pinon at hotmail.fr (Boris PINON) Date: Mon, 29 Oct 2012 10:01:03 +0100 Subject: [Dovecot] Active Directory 2003 user database and passwords with special characters Message-ID: Hello everybody, As explained in the topic, i have troubles with authentication of my users. First of all, sorry for my poor english... I'm running dovecot v1.2.15 on a Debian 6 64bits server up to date. My users database is an Active Directory 2003 (it's important to know that because Active Directory can't retrieve users passwords, you have to bind LDAP with a domain administrator). So, i'm using userdb ldap for authenticated my users and it works ! BUT... When an user having a password with special characters like " ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ", authentication does not work. And in my log file : MY_SERVER | mail/info | dovecot | 2012/08/27 10:42:14 | auth(default): cache(my_user,192.168.7.127): plain(RU0975?*) != 'RU0975??*' As you can see, the character ? has been replaced by ??. My dovecot.conf : protocols = imap imaps pop3 pop3s managesieve shutdown_clients = yes protocol imap { listen = 192.168.7.1:143 ssl_listen = 192.168.7.1:993 mail_plugins = quota imap_quota autocreate imap_client_workarounds = outlook-idle delay-newmail tb-extra-mailbox-sep } protocol pop3 { listen = 192.168.7.1:110 ssl_listen = 192.168.7.1:995 mail_plugins = quota pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_lock_session = no pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol managesieve { listen = 192.168.7.1:4190 login_executable = /usr/lib/dovecot/managesieve-login mail_executable = /usr/lib/dovecot/managesieve managesieve_implementation_string = dovecot } protocol lda { mail_plugins = sieve quota postmaster_address = postmaster at contoso.fr hostname = webmail.contoso.fr sendmail_path = /usr/sbin/sendmail quota_full_tempfail = no auth_socket_path = /var/run/dovecot/auth-master } log_timestamp = "%Y-%m-%d %H:%M:%S " syslog_facility = mail mail_debug = no auth_debug = no auth_debug_passwords = no ssl = required ssl_cert_file = /etc/ssl/certs/webmail.contoso.fr.pem ssl_key_file = /etc/ssl/private/webmail.contoso.fr.key ssl_ca_file = /etc/ssl/certs/VERYSIGN.pem ssl_verify_client_cert = no mail_location = maildir:%h mail_full_filesystem_access = no mail_uid = 500 mail_gid = 8 mail_privileged_group = mail first_valid_uid = 500 last_valid_uid = 500 first_valid_gid = 8 last_valid_gid = 8 login_greeting = Webmail CONTOSO login_process_size = 256 login_process_per_connection = no login_processes_count = 2 login_max_processes_count = 128 login_max_connections = 512 max_mail_processes = 1024 mail_process_size = 256 mail_max_keyword_length = 50 disable_plaintext_auth = yes auth_failure_delay = 2 auth_process_size = 256 auth_username_format = %Lu auth default { mechanisms = plain login auth_cache_size = 2048 passdb ldap { args = /etc/dovecot/dovecot-ldap.conf } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } user = vmail count = 1 socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail group = mail } client { path = /var/run/dovecot/auth-client mode = 0666 user = vmail group = mail } } } dict { } plugin { quota = maildir:User quota quota_warning = bytes=80%% /usr/lib/dovecot/quota-warning 80 quota_warning2 = bytes=95%% /usr/lib/dovecot/quota-warning 95 quota_warning3 = bytes=99%% /usr/lib/dovecot/quota-warning 99 sieve=dovecot.sieve sieve_dir=~/.Sieve sieve_extensions=+imapflags autocreate = Spam autocreate2 = Trash autosubscribe = Spam autosubscribe2 = Trash autosubscribe3 = Sent autosubscribe4 = Drafts } And my dovecot-ldap.conf : # My domain controller uris = ldap://192.168.1.1:3268 dn = CN=ServerOperator,CN=Users,DC=contoso,DC=fr dnpass = MyPassword debug_level = 0 auth_bind = yes ldap_version = 3 base = CN=Users,DC=contoso,DC=fr deref = never scope = subtree user_attrs = mailDirectory=home=/var/spool/mail/%$,mailQuota=quota_rule=*:bytes=%$,=quota_rule2=Trash:storage=100%% user_filter = (&(|(sAMAccountName=%n)(mailAcceptingGeneralID=%u)(mail=%u))(!(|(mailDrop="*|*")(mailDrop="*:*")(mailDrop="*/*")(userParams=noMail)))) pass_filter = (&(sAMAccountName=%n)(!(|(mailDrop="*|*")(mailDrop="*:*")(mailDrop="*/*")(userParams=noMail)))) default_pass_scheme = CRYPT Does anyone else have this problem? If yes, how to solve? Thank you in advance. From tss at iki.fi Mon Oct 29 16:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:18:22 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <20121016231856.GA10851@daniel.localdomain> References: <20121016231856.GA10851@daniel.localdomain> Message-ID: <0CECDB89-90BF-4A2F-97AC-713344F24996@iki.fi> On 17.10.2012, at 2.18, Daniel Parthey wrote: > doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A > shows the following message in the log when iterating the 49th user: > > Oct 17 00:47:17 10.129.3.233 dovecot: doveadm: Error: purge: invalid option -- 'e' > Oct 17 00:47:17 10.129.3.233 dovecot: doveadm(someuser at example-ll.org): Error: doveadm purge: Client sent unknown parameter: ? > > Any ideas on how this error gets triggered? Not sure. There's no valid 'e' option anywhere. I guess one of the non-option parameters begin with "-e" and it think it's an option. But I can't really think of how that would happen with purge either. So it would be helpful to look at what exactly the doveadms are talking to each others. Could you get the network traffic from them? Or strace -s 1000 doveadm purge should show it somewhere too. From tss at iki.fi Mon Oct 29 16:27:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:27:20 +0200 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <20121018060354.GA2528@leningrad.koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> Message-ID: On 18.10.2012, at 9.03, Levent Dane wrote: >> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html > > in Code/dovecot-virtual: > Archive > inthread refs keyword code not deleted I still couldn't reproduce with this. > I tried to take coredump but i didn't compile with debug flags. > http://pastebin.com/CMbiYJeK I think the problem here mainly is that gdb doesn't work very nicely across multiple execs (imap executes doveconf which executes imap again). You can avoid that by getting a core dump the regular way or making the $base_dir/config socket 0666 permissions. > If you can't reproduce this error. Tomorrow, I'll compile with debug flags. A proper gdb backtrace would definitely be the easiest way to solve this. BTW. Is it only STATUS (UNSEEN) that crashes, or also if you simply SELECT the mailbox? From bernics.gabor at penta.hu Mon Oct 29 16:29:22 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Mon, 29 Oct 2012 15:29:22 +0100 Subject: [Dovecot] mail open slowly In-Reply-To: <508C27A6.30207@hardwarefreak.com> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> <508C27A6.30207@hardwarefreak.com> Message-ID: Thanks a lot to everybody First step I will upgrade to dovecot2. I will write my experiences. "Running with fsync disabled is like having sex with a Bangkok prostitute without a condom while juggling chainsaws while driving drunk at 250kph at night without headlights." :-) Gabor 2012-10-27 20:27 id?pontban Stan Hoeppner ezt ?rta: > On 10/27/2012 6:58 AM, Bernics G?bor | Penta Uni? Zrt. wrote: > >> I use dovecot LDA (+sieve) with maildir. conf: http://pastebin.com/9fhYD58g [1] > > Next time simply paste "dovecot -n" output into your email. > > Assuming Dovecot is the only program accessing the maildirs, try: > > maildir_very_dirty_syncs=yes > > That may help some. > > It may not have been a factor in this case, but note that when anyone is > doing a full text search on a large mailbox on this hardware with > maildir you will see latency, and it is unavoidable. Neither a single > 7.2K SATA spindle nor md/RAID1 pair of them, has enough seek capacity to > service all the sector requests in a timely fashion. > > Also, I noticed you disabled fsync. This is a very very bad idea for a > mail server. If you lose power, or suffer a kernel/hardware/etc crash, > you lose the Linux buffer cache contents. Thus, you may lose emails > that haven't been flushed to disk, and possibly get index file > corruption if mmap'd pages haven't been flushed. > > Running with fsync disabled is like having sex with a Bangkok prostitute > without a condom while juggling chainsaws while driving drunk at 250kph > at night without headlights. > > fsync does hurt write performance to a degree, especially with maildir > storage, but will likely be invisible on a small server with few > users/light load. And it will prevent potentially severe problems with > file corruption and/or lost emails. Links: ------ [1] http://pastebin.com/9fhYD58g From tss at iki.fi Mon Oct 29 16:38:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:38:58 +0200 Subject: [Dovecot] Dovecot quota postgres dictionary problems In-Reply-To: <1350632450161-38234.post@n4.nabble.com> References: <1350632450161-38234.post@n4.nabble.com> Message-ID: <794F0BC6-CF9D-4BE7-A0C9-FC0D93E72166@iki.fi> On 19.10.2012, at 10.40, tmihalicek wrote: > I have a strange errors in .err log file, but the postgres seem to be filling > with quota changes, i will also put configs in > > Oct 19 09:23:52 mailstore-node-01 dovecot: imap(test at example.net): Error: > read(/var/run/dovecot/dict) failed: Timeout after 30 seconds > Oct 19 09:24:22 mailstore-node-01 dovecot: imap(test at example.net): Error: > read(/var/run/dovecot/dict) failed: Timeout after 30 seconds dict process is taking too long to give results back. Is PostgreSQL too heavily loaded? > Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Panic: > file dict-client.c: line 270 (client_dict_finish_transaction): assertion > failed: (dict->async_commits > 0) http://hg.dovecot.org/dovecot-2.1/rev/67e9cb0b06ec should fix this crash. From tss at iki.fi Mon Oct 29 16:42:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:42:57 +0200 Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Message-ID: On 19.10.2012, at 23.43, E.B. wrote: > I'm having some problems getting LDA to work without > userdb lookups and have a few related questions. This system has all > users in MySQL, each user with unique UID/GID, no local users at all. > Installation is from apt-get. > > > 1) If LDA is invoked without > lookups, is it correct to assume that the "service auth" and "service > auth-worker" can be completely removed from dovecot master > configuration? (I have tried commenting them out and logging into IMAP, > which seems to work, not sure if anyone else needs the auth service) If you remove them the defaults are simply used. > 2) > If LDA is invoked without lookups, will I be unable to use Dovecot > quota plugin? Does it need to have a user lookup to get quota info? > (haven't added quota support, need to take this one step at a time) You can give quota info also via either environment variables or via -o plugin/quota_rule=xx parameter. > 3) The interesting part -- I am invoking LDA from Maildrop. See: > http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So > when invoked, Maildrop has already dropped to the destination UID/GID > and the needed paths are available in the environment. However, using > as many permutations of calling LDA as I can think of (based on http://wiki2.dovecot.org/LDA ), I always get this: > > (command line usage error. Command output: lda: Fatal: Couldn't lookup our username (uid=2500) ) Set USER environment. > 3.5) > Related question, my users have separate homedir and maildir, both > paths are looked up by Maildrop. I think I need to call LDA with > "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? As long as the home and mail directories point to the same ones as they are when logging in via IMAP/POP3. From tss at iki.fi Mon Oct 29 16:45:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:45:06 +0200 Subject: [Dovecot] Configuring Dovecot & Snarf plugin for the first time In-Reply-To: <50835541.8000808@bubble.org> References: <50835541.8000808@bubble.org> Message-ID: On 21.10.2012, at 4.52, Jeffrey Ross wrote: > However whenever I enable the snarf plugin using the example on the wiki page my email is not loaded and when I remove my configuration for snarf my email re-appears. Based upon what I can tell the snarf plugin is either not loading (but I see it listed in the logs) or simply not working (which is probably because its not configured properly). .. > plugin { > snarf = = /snarf/INBOX > } Looks like you have one too many "="? From tss at iki.fi Mon Oct 29 16:47:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:47:25 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: References: Message-ID: <189B7E53-0495-4D2E-A845-6CEE1304898D@iki.fi> On 18.10.2012, at 11.05, Jan-Frode Myklebust wrote: > I enabled the trash plugin yesterday, adding "trash" to mail_plugins, > and configuring the plugin setting "trash = > /etc/dovecot/dovecot-trash.conf.ext". > > > But I still see users with lots of files in INBOX.Trash getting > bounced because of quota exceeded: .. > # 2.0.14: /etc/dovecot/dovecot.conf There are several fixes to Trash plugin in v2.1. I think it's simply somewhat broken in v2.0. From tss at iki.fi Mon Oct 29 16:49:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:49:58 +0200 Subject: [Dovecot] Auth caching and password changes In-Reply-To: <508526C2.8030403@um.es> References: <508149CC.9070004@um.es> <508526C2.8030403@um.es> Message-ID: <1B474730-A7EF-4607-9A1E-4DD215518E6B@iki.fi> On 22.10.2012, at 13.58, Angel L. Mateo wrote: > My question now is there any way to configure authentication so a mechanism is only use when connections coming from a set of IPs? local/remote {} blocks were supposed to provide this. They don't currently work for auth process settings though. From tss at iki.fi Mon Oct 29 17:04:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:04:25 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <5087FF4F.8050103@papaya-cms.com> References: <5087FF4F.8050103@papaya-cms.com> Message-ID: <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> On 24.10.2012, at 17.46, Alexander Weber wrote: > if address :is "to" "mantis-admin@<*>" > { > fileinto "/home/shared/.automail.Bugtracker/"; > } Use mailbox name, not filesystem path: fileinto "shared/automail/Bugtracker"; From tss at iki.fi Mon Oct 29 17:08:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:08:42 +0200 Subject: [Dovecot] Dovecot sends BYE while fetching X-GM-MSGID In-Reply-To: <1698191351159403@web29e.yandex.ru> References: <1698191351159403@web29e.yandex.ru> Message-ID: On 25.10.2012, at 13.03, Loshkovskyi Andrii wrote: > While using the following set of commands, I am having the error as below: > > FETCH 7 (X-GM-MSGID) > A15 FETCH 7 (X-GM-MSGID) > A15 BAD Error in IMAP command FETCH: Unknown parameter X-GM-MSGID > > Can I somehow disable such errors so that Dovecot won't send BYE on X-GM-MSGID but just proceed with following emails? BYE or BAD? It shouldn't send BYE unless you send 20 consecutive BAD commands. From weber at papaya-cms.com Mon Oct 29 17:11:25 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Mon, 29 Oct 2012 16:11:25 +0100 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> Message-ID: <508E9C9D.8060601@papaya-cms.com> Am 29.10.2012 16:04, schrieb Timo Sirainen: > On 24.10.2012, at 17.46, Alexander Weber wrote: > >> if address :is "to" "mantis-admin@<*>" >> { >> fileinto "/home/shared/.automail.Bugtracker/"; >> } > > Use mailbox name, not filesystem path: > > fileinto "shared/automail/Bugtracker"; > error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. sieve: info: started log at Oct 29 16:10:03. error: msgid=<*>: failed to store into mailbox 'shared/automail/Bugtracker/': Invalid mailbox name. nope, didn't work :/ From tss at iki.fi Mon Oct 29 17:18:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:18:42 +0200 Subject: [Dovecot] Dovecot stops to work - anvil problem In-Reply-To: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> References: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> Message-ID: <51972B14-6973-4510-870D-956F858FC76B@iki.fi> On 26.10.2012, at 13.24, FABIO FERRARI wrote: > Hi all, > > we have a problem about anvil, it seems that when we have a high load the > dovecot stops to work. Sometimes it is sufficient to make a dovecot > reload, but sometimes we have to restart it. > > Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files This is the problem. > And these are the limit settings in the OS: > * soft nofile 131072 > * hard nofile 131072 > > Have someone had the same problem? The OS limits are ok. But you need to make sure that the dovecot processes have enough fds in ulimit. You can check the limits with: cat /proc//limits The "Max open files" soft limit is what you're most likely hitting. Use "ulimit -n 10000" or something before running dovecot binary. And make sure that it changes the limit in the proc. Many init scripts change the ulimit internally. From tss at iki.fi Mon Oct 29 17:18:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:18:46 +0200 Subject: [Dovecot] Can Dovecot authenticate against an external email server? In-Reply-To: <5087035B.7060208@perkel.com> References: <5087035B.7060208@perkel.com> Message-ID: On 23.10.2012, at 23.51, Marc Perkel wrote: > Just wondering if anyone has done this. > > I have a spam filtering service where I am now storing spam for users I filter for. It's a filter and forward service so I don't control the recipient's email server. > > What I would like to do somehow is have the user enter their email address and password and then look up their POP/IMAP server from a database and try to authenticate from it. If sucessful then the user will be able to access their stored spam using Dovecot and Squirrelmail. http://wiki2.dovecot.org/PasswordDatabase/IMAP From tss at iki.fi Mon Oct 29 17:20:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:20:15 +0200 Subject: [Dovecot] dovecot lda - Permission denied In-Reply-To: <20121026114455.30440@gmx.net> References: <20121026114455.30440@gmx.net> Message-ID: <157A66BA-69AB-45AE-927C-21F827B1736B@iki.fi> On 26.10.2012, at 14.44, tony.blue.mailinglist at gmx.de wrote: > Oct 25 23:37:13 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(andy) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) > ... > Oct 25 23:37:14 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=500(tony) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) > ... > > Dovecot is installed as !include auth-passwdfile.conf.ext. For all users there is a entry in der /etc/dovecot/users. > > Usaly the user rights are set to 600. I tryed 755, but I get the same errormessage. 0755 is basically the same as 0600 for sockets, since you disabled writes for others. Use 0777 to give everyone permissions. From tss at iki.fi Mon Oct 29 17:22:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:22:22 +0200 Subject: [Dovecot] Overlapping userdb/passdbs In-Reply-To: References: Message-ID: <98C16420-1D9E-4F37-86D7-9FB91438B843@iki.fi> On 26.10.2012, at 22.13, James Devine wrote: > I have an ldap server for which each entry includes the email address and > the username portion of the email address for authentication. > Authentication works by username if the username is unique among all the > entries. I need to now add some users which must authenticate even if the > username is not unique. I figured one way to do this would be to add a > second user/pass db which puts further restrictions on the ldap query to > make it unique for those users. This doesn't seem to work however as if > the user is found in the first ldap query but the password does not match > it does not try the second. I would use the password as part of the query > but this setup requires me to allow the client to hash the password. Is > there a way to do this? Or maybe I am approaching the problem wrong. You'd need to update this patch: http://dovecot.org/patches/2.0/auth-multi-password-2.0.diff It worked for v1.1 and maybe for v1.2. I never included it mainly because I never had time to check if it had any security issues. From jk at jkart.de Mon Oct 29 17:23:29 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 16:23:29 +0100 Subject: [Dovecot] Out of memory/Managesieve Message-ID: <508E9F71.8050208@jkart.de> Hello, I have here a problem with managesieve. With the login about webmail (roundcube) comes here in the log: --snip dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) --snap I've increased of 2048M and the same above. then with 4096 M happens the following --snip dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) --snap How can one solve then this? Any ideas greatly appreciated. Thanks. -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Backup interessiert niemanden - Auf Restore kommt es an! From tss at iki.fi Mon Oct 29 17:26:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:26:49 +0200 Subject: [Dovecot] When are search indexes updated? In-Reply-To: <508C9C8A.8000309@hardwarefreak.com> References: <508C9C8A.8000309@hardwarefreak.com> Message-ID: <6DC094E4-2D06-4146-A4C2-1717614E30E4@iki.fi> On 28.10.2012, at 4.46, Stan Hoeppner wrote: >> * When are search indexes updated? > > When the index is stale. > >> * Are they updated incrementally? >> * If not, why not? >> * If so, why would a mailbox's index drift out-of-date, as mine had? > > When a sufficient number of messages are added to an IMAP folder the FTS > index becomes stale. This index is not updated in real time. This is > why Timo and others recommend cron'ing a script to index folders > regularly that are searched regularly. This keeps the indexes up to > date and keeps searches fast. If you don't do this or search often, > your indexes become stale. Then each time you do an FTS search the > first thing that happens is an FTS re-indexing of the mail folder. Only > then does it display the search results. Otherwise correct, but "re-indexing" is the wrong word. No already indexed mails are reindexed. Only new mails are added to the index. From tss at iki.fi Mon Oct 29 17:31:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:31:42 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <394FADB5-4E6C-4179-BF30-244390964FA4@iki.fi> On 24.10.2012, at 14.28, Milan Holz?pfel wrote: > Oct 24 10:43:23 two dovecot: imap-login: Login: user=<...>, method=PLAIN, rip=..., lip=..., mpid=4977, TLS > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-24 10:43:23] bytes=115/53726 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: Inconsistency in map index (467,31960 != 467,36768) > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: rebuilding indexes The above problems aren't too bad, since Dovecot fixes itself. > Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Panic: file mdbox-storage-rebuild.c: line 773 (rebuild_update_refcounts): assertion failed: (map_uid < msgs[i]->map_uid) Now this is a bug. > Dovecot 2.0.19-0ubuntu1 But the bug may have already been fixed in v2.1. > The whole mdbox is 6.6 GiB large and I guess that it contains about > 300k-600k messages. It's an archive of public mailing lists, so I could > give access to the files. I'd try first with a recent 2.1 version and if that doesn't fix the crash the easiest way for me to fix it would be to get the files. If you put the files through http://dovecot.org/tools/mdbox-obfuscate.pl they should compress pretty nicely. From tss at iki.fi Mon Oct 29 17:34:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:34:28 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <508E9C9D.8060601@papaya-cms.com> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> <508E9C9D.8060601@papaya-cms.com> Message-ID: <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> On 29.10.2012, at 17.11, Alexander Weber wrote: >> fileinto "shared/automail/Bugtracker"; > > error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. "shared/automail/Bugtracker" isn't the same as "shared/.automail.Bugtracker/". Anyway that specific error message comes from the trailing '/'. From jk at jkart.de Mon Oct 29 17:34:45 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 16:34:45 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508E9F71.8050208@jkart.de> References: <508E9F71.8050208@jkart.de> Message-ID: <508EA215.6000303@jkart.de> am 29.10.12 16:23 schrieb Jim Knuth : > Hello, > > I have here a problem with managesieve. With the login about > webmail (roundcube) comes here in the log: > > --snip > dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): > Out of memory > dovecot: managesieve-login: Fatal: master: service(managesieve-login): > child 10157 returned error 83 (Out of memory (service > managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) > --snap > I've increased of 2048M and the same above. > then with 4096 M happens the following > > --snip > dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: > Bad file descriptor > dovecot: managesieve-login: Fatal: master: service(managesieve-login): > child 9777 killed with signal 6 (core dumps disabled) > --snap > > How can one solve then this? > Any ideas greatly appreciated. Thanks. > Sorry, I've forgotten. OS Debian stable and Dovecot 2.1. -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Nicht Absicht unterstellen, wenn auch Dummheit ausreicht! From tss at iki.fi Mon Oct 29 17:41:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:41:06 +0200 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508E9F71.8050208@jkart.de> References: <508E9F71.8050208@jkart.de> Message-ID: On 29.10.2012, at 17.23, Jim Knuth wrote: > I have here a problem with managesieve. With the login about > webmail (roundcube) comes here in the log: You can always easily reproduce this? Can you get the network traffic logs between Roundcube and Dovecot and reproduce it by sending those same commands manually? > --snip > dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory > dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) Looks like there's a bug somewhere.. Doesn't it log a "raw backtrace"? > --snap > I've increased of 2048M and the same above. > then with 4096 M happens the following > > --snip > dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor > dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) > --snap Probably related to the first error. It would be helpful to get gdb backtraces from both of them, although from the first one you couldn't without patching + recompiling Dovecot. But from the second one I think you can get a core dump with: service managesieve-login { executable = managesieve-login -D } Then you can do something like: gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core bt full From weber at papaya-cms.com Mon Oct 29 17:54:51 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Mon, 29 Oct 2012 16:54:51 +0100 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> <508E9C9D.8060601@papaya-cms.com> <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> Message-ID: <508EA6CB.9010600@papaya-cms.com> Am 29.10.2012 16:34, schrieb Timo Sirainen: > On 29.10.2012, at 17.11, Alexander Weber wrote: > >>> fileinto "shared/automail/Bugtracker"; >> >> error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. > > "shared/automail/Bugtracker" isn't the same as "shared/.automail.Bugtracker/". > > Anyway that specific error message comes from the trailing '/'. > I've tried every combination of this shared subfolder ( pretty wired :S ) but nevermind - i try procmail with sieve after procmail. thanks anyway :) From tss at iki.fi Mon Oct 29 18:53:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 18:53:00 +0200 Subject: [Dovecot] Save/restore IMAP session state Message-ID: <1351529580.13571.93.camel@hurina> Some future Dovecot version will have "imap-idle" processes where IDLEing IMAP connections get moved, so the system wouldn't waste so much memory for all the IDLEing imap processes. A week ago I thought I'd see how easy it would be to implement this. I got a basic proof of concept working as a "X-STATE" command. Save the state: a x-state * STATE AQDLW45QdwAAAAMAAABuAQAAAAAAAFAcffYAPHnpFctbjlDbYQAAcEmzCwAA a OK State exported. Restore the state: b x-state AQDLW45QdwAAAAMAAABuAQAAAAAAAFAcffYAPHnpFctbjlDbYQAAcEmzCwAA b OK State imported. This could also be used to implement quick session state restoring for webmails (as suggested by Michael Slusarz). For getting the imap-idle process there would have to be code that: * triggers the session saving when process is IDLEing * figures out what filesystem paths the imap-idle should be looking at (i.e. paths to selected mailbox's dovecot.index.log file and maybe for e.g. maildir new/) * send the session state string, paths and imap connection fd to imap-idle process via UNIX socket * implement the actual imap-idle process * implement a way for imap-idle process to send back the state and connection fd to restore the imap process The patch is ugly and still missing many things. Anyway I thought I'd include it here just in case someone was really eager to continue implementing it. :) I'm not sure when I'll have time for it. A full patch would probably have to have some session_save()/session_restore() functions in lib-storage API. But a quick and dirty way is possible to implement for v2.1 as well, as long as some IMAP extensions aren't used (most importantly rfc5267). -------------- next part -------------- A non-text attachment was scrubbed... Name: imap-state.diff Type: text/x-patch Size: 11305 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 18:57:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 18:57:32 +0200 Subject: [Dovecot] Save/restore IMAP session state In-Reply-To: <1351529580.13571.93.camel@hurina> References: <1351529580.13571.93.camel@hurina> Message-ID: <567ADA17-F5E6-48DF-9E9D-601267C568FE@iki.fi> On 29.10.2012, at 18.53, Timo Sirainen wrote: > The patch is ugly and still missing many things. Anyway I thought I'd > include it here just in case someone was really eager to continue > implementing it. :) I'm not sure when I'll have time for it. Oh, and of course I forgot one file out of the patch. Here's an updated one. -------------- next part -------------- A non-text attachment was scrubbed... Name: imap-state2.diff Type: application/octet-stream Size: 12948 bytes Desc: not available URL: From guallar at easternrad.com Mon Oct 29 19:57:37 2012 From: guallar at easternrad.com (Josep L. Guallar-Esteve) Date: Mon, 29 Oct 2012 13:57:37 -0400 Subject: [Dovecot] INBOX permissios woes Message-ID: <9cc05811b75ed0f7235dd86d0e5c1dfd@easternrad.com> Hello, I have a dovecot system that uses winbind authentication against Active Directory. I set it up by following the directions in the wiki. That works great. When a new user receives an email, the inbox is created with permissions 600 (rw- --- --) and ownership user:mail , even though I did chmod 02770 /var/spool/mail. And then, when dovecot tries to access the inbox, it throws the error: Oct 29 13:47:59 imap-login: Info: Login: user=, method=PLAIN, rip=10.0.0.6, lip=10.0.0.26, mpid=29047, secured Oct 29 13:47:59 imap(user1): Error: stat(/var/mail/user1) failed: Permission denied Oct 29 13:47:59 imap(user1): Error: stat(/var/mail/user1) failed: Permission denied Accessing users' Sent, Trash, creating new folders.... all that works fine. I've been looking at the documentation, reading the wiki, searching on google, asking on IRC. If you have any hint or documentation that I've must have overlooked, please let me know. Here's my dovecot information: [josep at testmail ]$ dovecot --version 2.0.9 [josep at testmail ]$ dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.11.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain ntlm login auth_use_winbind = yes auth_username_format = %Lu base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log listen = * log_path = /var/log/dovecot.log mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail mbox_write_locks = fcntl passdb { driver = pam } protocols = imap service auth { unix_listener auth-userdb { mode = 0600 } } ssl_cert = References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> Message-ID: <20121029181700.GA4240@leningrad.koli.be> On 10/29, Timo Sirainen wrote: >On 18.10.2012, at 9.03, Levent Dane wrote: > >>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >> >> in Code/dovecot-virtual: >> Archive >> inthread refs keyword code not deleted > >I still couldn't reproduce with this. I think the problem is mail-search.c is corrupting the index files. >> I tried to take coredump but i didn't compile with debug flags. >> http://pastebin.com/CMbiYJeK > >I think the problem here mainly is that gdb doesn't work very nicely across multiple execs (imap executes doveconf which executes imap again). You can avoid that by getting a core dump the regular way or making the $base_dir/config socket 0666 permissions. I compiled with -ggdb flag. I'm getting this informations /var/log/messages: http://pastebin.com/bpkvp4Ak and from gdb: http://pastebin.com/HY0mVYBS I'm using mutt for imap access. When I pressed '%' key which runs function, the dovecot got seqfault. >> If you can't reproduce this error. Tomorrow, I'll compile with debug flags. > >A proper gdb backtrace would definitely be the easiest way to solve this. > >BTW. Is it only STATUS (UNSEEN) that crashes, or also if you simply SELECT the mailbox? I tried simple SELECT and it still crashed. As I said, the problem is dovecot.index files. Somehow, mail-search.c corrupts this file. -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 From tss at iki.fi Mon Oct 29 20:23:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 20:23:14 +0200 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <20121029181700.GA4240@leningrad.koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> <20121029181700.GA4240@leningrad.koli.be> Message-ID: <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> On 29.10.2012, at 20.17, Levent Dane wrote: > On 10/29, Timo Sirainen wrote: >> On 18.10.2012, at 9.03, Levent Dane wrote: >> >>>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >>> >>> in Code/dovecot-virtual: >>> Archive >>> inthread refs keyword code not deleted >> >> I still couldn't reproduce with this. > > I think the problem is mail-search.c is corrupting the index files. Not that itself, but yeah looks like if virtual plugin assert-crashes in mail-search.c it leaves the indexes so that the next access will segfault. > I compiled with -ggdb flag. I'm getting this informations > /var/log/messages: http://pastebin.com/bpkvp4Ak > and from gdb: http://pastebin.com/HY0mVYBS Better backtrace than last time, but still no debug information in the backtrace. Maybe that got stripped somewhere between compiling and installing? You can check with "file ..../imap" to see if it's there. Also backtrace from both the mail-search.c assert crash and the segfault would be useful. From calestyo at scientia.net Mon Oct 29 22:31:48 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 21:31:48 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? Message-ID: <1351542708.3435.25.camel@fermat.scientia.net> Hi. For mbox, http://wiki2.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata lists a numer of mail headers: - X-IMAPbase - X-IMAP - X-UID - Status - X-Status - X-Keywords - Content-Length that are treated specially by dovecot. It also suggests, that these should be stripped by the LDA (I guess in order that someone sending you such mail cannot set the status or keywords, or even "attack you" by setting a bogus Content-Length). I wondered, when using maildir, are there any headers that dovecote would treat specially, too? And which I therefore should strip? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 22:39:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 22:39:51 +0200 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? Message-ID: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. From tss at iki.fi Mon Oct 29 22:40:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 22:40:46 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351542708.3435.25.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> Message-ID: <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> On 29.10.2012, at 22.31, Christoph Anton Mitterer wrote: > For mbox, > http://wiki2.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata lists a > numer of mail headers: > - X-IMAPbase > - X-IMAP > - X-UID > - Status > - X-Status > - X-Keywords > - Content-Length > that are treated specially by dovecot. > > It also suggests, that these should be stripped by the LDA (I guess in > order that someone sending you such mail cannot set the status or > keywords, or even "attack you" by setting a bogus Content-Length). Right. > I wondered, when using maildir, are there any headers that dovecote > would treat specially, too? > And which I therefore should strip? No. Maildir metadata is stored elsewhere. From calestyo at scientia.net Mon Oct 29 22:54:09 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 21:54:09 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste Message-ID: <1351544049.3435.47.camel@fermat.scientia.net> Hi. I recently mentioned in several posts, that I'd tended to use mbox rather than maildir, because you don't loose so much space (due to always allocating full blocks per maildir file and thus per mail). I made some tests of my archive, which consists of some 3,4 million mails at a total of 42GB). Most of these mails are probably normal sized, but there are also some with bigger attachments. For those who are interested here are the results: I used a 53687091200 B image file (via loop device) and tested ext4 only. btrfs is IMHO not yet ready, I have had often issues with XFS (corruptions), reiser4 is more or less dead and reiser3 is said to have issues (see e.g. its wikipedia article, even though it has that mode for small files which would fit nicely). As you see the number of mails increased a bit, cause I tested over several days... but this is only a very small increase so it shouldn't change the numbers a lot. 1) Original mbox archives (right now in Evolution) mbox exact space: 38122676224 (does not include meta-data) mbox guess space: 44625670144 (includes Evolution meta-data which is several GBs) mbox num mails: 3412999 (occurances of From_ lines) In the following: - image file, 1B-blocks, Used_begin, Used_end, Available_begin, Available_end result out of df -B 1 - mdir exact used space is the sum of du -B 1 for each regular file (i.e. each mdir file) - mdir guess used space du -B 1 on the root dir of the filesystem - mdir num mails: find . type -f | wc -l on the root dir of the filesystem 2) EXT4 with 4096 blocks: image file: 53687091200 1B-blocks: 52844687360 Used_begin: 188555264 Used_end: 45198778368 Available_begin: 49971777536 Available_end: 2444972032 mdir exact used space: 44810866688 mdir guess used space: 45010243584 mdir num mails: 3423296 delta: 6.688190464 G delta / mail: 1953 B 3) EXT4 with 2048 blocks: image file: 53687091200 1B-blocks: 50324295680 Used_begin: 82857984 Used_end: 41598846976 Available_begin: 47557083136 Available_end: 6041094144 mdir exact used space: 41323991040 mdir guess used space: 41516007424 mdir num mails: 3425033 delta: 3.201314816 G delta / mail: 934 B 4) EXT4 with 1024 blocks: image file: 53687091200 1B-blocks: 50314834944 Used_begin: 38287360 Used_end: 39909360640 Available_begin: 47592193024 Available_end: 7721119744 mdir exact used space: 39683908608 mdir guess used space: 39871086592 mdir num mails: 3425033 delta: 1.561232384 G delta / mail: 455 B As you can see, the delta per mail is rather close to the statistically expected values of 2048B, 1024B and 512B. In the end I probably changed my opinion. ~7GB of wasted block space for all my mails is actually quite a lot, but in days of cheap disk space it's acceptable. And with mbox one has IMHO the major disadvantage that mailservers (including dovecot) store some meta-data _in_ it (i.e. in the mails themselves) , which I don't like a lot. I still think about reports that mbox is much faster with full text search (which sounds reasonable)... but therefore one needs probably and database backend anyway. HTH, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 23:00:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:00:56 +0200 Subject: [Dovecot] v2.2.alpha1 released Message-ID: <1351544456.13571.102.camel@hurina> http://dovecot.org/releases/2.2/alpha/dovecot-2.2.alpha1.tar.gz http://dovecot.org/releases/2.2/alpha/dovecot-2.2.alpha1.tar.gz.sig I wanted to start stabilizing v2.2 release some months ago already, but I somehow got stuck adding more and more features. Now it looks like all of the necessary API changes are done, so everything I'm planning on near future can still be added to v2.2 without major changes. The redesigned dsync hasn't had much testing yet, so avoid running it with important mails. Would be nice if people started testing and reporting any bugs. I was going to create some kind of a test suite for testing all the possible syncing combinations and also doing some random stress testing, but I haven't had time for that yet. The new dsync supports doing very fast syncs by saving the resulting state and giving it as command line parameter to the next sync. So for example doveadm sync -s "" > new-state saves the state and doveadm sync -s `cat new-state` continues from the saved state. The replicator code doesn't yet support this. * When creating home directories, the permissions are copied from the parent directory if it has setgid-bit set. For full details, see http://wiki2.dovecot.org/SharedMailboxes/Permissions * "doveadm auth" command was renamed to "doveadm auth test" * IMAP: ID command now advertises server name as Dovecot by default. It was already trivial to guess this from command replies. + Implemented IMAP MOVE and BINARY extensions + Implemented IMAP CATENATE, URLAUTH and URLAUTH=BINARY extensions (by Stephan Bosch). + Implemented IMAP NOTIFY extension. Requires mailbox_list_index=yes to be enabled. + Redesigned and rewritten dsync. The new design makes the syncing faster, more reliable and more featureful. The new dsync protocol isn't backwards compatible with old dsync versions (but is designed to be forwards compatible with future versions). + All mailbox formats now support per-user message flags for shared mailboxes by using a private index. It can be enabled by adding :INDEXPVT= to mail location. This should be used instead of :INDEX also for Maildir/mbox to improve performance. + Improved mailbox list indexes. They should be usable now, although still disabled by default. + Added LAYOUT=index. The mailbox directories are created using their GUIDs in the filesystem, while the actual GUID <-> name mapping exists only in the index. + LMTP proxy: Implemented XCLIENT extension for passing remote IP address through proxy. From pw at wk-serv.de Mon Oct 29 23:05:42 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 29 Oct 2012 22:05:42 +0100 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <5083D963.3000700@wk-serv.de> References: <5083D963.3000700@wk-serv.de> Message-ID: <508EEFA6.1020506@wk-serv.de> Hi, no one here who is able to reply to my questions? Regards Patrick Patrick Westenberg schrieb: > Hi everyone, > > short version: > Is there no built in failover mechanism for the director service to > handle a backend failure? > > Long version: > I have a frontend server running the director service and two backends. > Due to maintenance I had to shut down one of the backends which caused > connection errors for the users being directed to this backend. > > I was very surprised as I expected the director to redirect these users > to the remaining backend. > > Am I wrong or is the director not working as expected? > > Regards > Patrick > > > > # 2.1.6: /usr/local/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 > auth_mechanisms = plain login > director_mail_servers = 172.17.1.1 172.17.1.2 > director_servers = 172.17.1.3 172.17.1.4 > lmtp_proxy = yes > log_path = /var/log/dovecot.log > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > protocols = imap pop3 lmtp sieve > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-userdb { > user = dovecot > } > } > service director { > fifo_listener login/proxy-notify { > mode = 0666 > } > inet_listener { > address = 172.17.1.3 > port = 9090 > } > unix_listener director-userdb { > mode = 0600 > } > unix_listener login/director { > mode = 0666 > } > } > service imap-login { > executable = imap-login director > } > service lmtp { > inet_listener lmtp { > address = 172.17.1.3 > port = 24 > } > } > service managesieve-login { > executable = managesieve-login director > inet_listener sieve { > port = 4190 > } > } > service pop3-login { > executable = pop3-login director > } > ssl_cert = ssl_key = protocol !smtp { > passdb { > args = proxy=y nopassword=y starttls=any-cert > driver = static > } > } > protocol smtp { > passdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > } > protocol lmtp { > auth_socket_path = director-userdb > } From tss at iki.fi Mon Oct 29 23:06:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:06:53 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351544049.3435.47.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> Message-ID: <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> On 29.10.2012, at 22.54, Christoph Anton Mitterer wrote: > I recently mentioned in several posts, that I'd tended to use mbox > rather than maildir, because you don't loose so much space (due to > always allocating full blocks per maildir file and thus per mail). .. > In the end I probably changed my opinion. > ~7GB of wasted block space for all my mails is actually quite a lot, but > in days of cheap disk space it's acceptable. > And with mbox one has IMHO the major disadvantage that mailservers > (including dovecot) store some meta-data _in_ it (i.e. in the mails > themselves) , which I don't like a lot. > I still think about reports that mbox is much faster with full text > search (which sounds reasonable)... but therefore one needs probably and > database backend anyway. There is of course mdbox also, which gives the best of both mbox and maildir (and some of its own new annoyances). From calestyo at scientia.net Mon Oct 29 23:09:11 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:09:11 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> Message-ID: <1351544951.3435.61.camel@fermat.scientia.net> Hi Timo. On Mon, 2012-10-29 at 22:40 +0200, Timo Sirainen wrote: > > I wondered, when using maildir, are there any headers that dovecote > > would treat specially, too? > > And which I therefore should strip? > > No. Maildir metadata is stored elsewhere. Great... and I expect that this ("no headers from the maildir files are interpreted") applies also, when one "imports" mails the first time. With import I don't mean via IMAP, but plainly moving e.g. a maildir++ tree under dovecots mail location. Then dovecot usually starts to generate all it's metadata,... and I expect that things like status and keywords are left simply unset... and things like UID and UIDVALIDITY are freshly initialised and not tried to be converted from the maildir files, right? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 23:11:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:11:15 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <508EEFA6.1020506@wk-serv.de> References: <5083D963.3000700@wk-serv.de> <508EEFA6.1020506@wk-serv.de> Message-ID: <6DFB1CD2-5FE6-405A-B2A8-545938A11F98@iki.fi> People already replied and pointed to poolmon. There is no built-in failure handling, because it's not possible to implement it in a way that works well for everyone. Although I think poolmon could also itself use a bit of tweaking. For example if all hosts became very heavily loaded, poolmon would now probably drop all of them immediately if one if its check connections failed. On 29.10.2012, at 23.05, Patrick Westenberg wrote: > Hi, > > no one here who is able to reply to my questions? > > Regards > Patrick > > > > Patrick Westenberg schrieb: >> Hi everyone, >> >> short version: >> Is there no built in failover mechanism for the director service to >> handle a backend failure? >> >> Long version: >> I have a frontend server running the director service and two backends. >> Due to maintenance I had to shut down one of the backends which caused >> connection errors for the users being directed to this backend. >> >> I was very surprised as I expected the director to redirect these users >> to the remaining backend. >> >> Am I wrong or is the director not working as expected? >> >> Regards >> Patrick >> >> >> >> # 2.1.6: /usr/local/etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 >> auth_mechanisms = plain login >> director_mail_servers = 172.17.1.1 172.17.1.2 >> director_servers = 172.17.1.3 172.17.1.4 >> lmtp_proxy = yes >> log_path = /var/log/dovecot.log >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> protocols = imap pop3 lmtp sieve >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0666 >> user = postfix >> } >> unix_listener auth-userdb { >> user = dovecot >> } >> } >> service director { >> fifo_listener login/proxy-notify { >> mode = 0666 >> } >> inet_listener { >> address = 172.17.1.3 >> port = 9090 >> } >> unix_listener director-userdb { >> mode = 0600 >> } >> unix_listener login/director { >> mode = 0666 >> } >> } >> service imap-login { >> executable = imap-login director >> } >> service lmtp { >> inet_listener lmtp { >> address = 172.17.1.3 >> port = 24 >> } >> } >> service managesieve-login { >> executable = managesieve-login director >> inet_listener sieve { >> port = 4190 >> } >> } >> service pop3-login { >> executable = pop3-login director >> } >> ssl_cert = > ssl_key = > protocol !smtp { >> passdb { >> args = proxy=y nopassword=y starttls=any-cert >> driver = static >> } >> } >> protocol smtp { >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> } >> protocol lmtp { >> auth_socket_path = director-userdb >> } > From tss at iki.fi Mon Oct 29 23:13:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:13:36 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351544951.3435.61.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> Message-ID: <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> On 29.10.2012, at 23.09, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 22:40 +0200, Timo Sirainen wrote: >>> I wondered, when using maildir, are there any headers that dovecote >>> would treat specially, too? >>> And which I therefore should strip? >> >> No. Maildir metadata is stored elsewhere. > > Great... and I expect that this ("no headers from the maildir files are > interpreted") applies also, when one "imports" mails the first time. > > With import I don't mean via IMAP, but plainly moving e.g. a maildir++ > tree under dovecots mail location. Yeah. > Then dovecot usually starts to generate all it's metadata,... and I > expect that things like status and keywords are left simply unset... and > things like UID and UIDVALIDITY are freshly initialised and not tried to > be converted from the maildir files, right? Flags are stored in the maildir filenames, so they're always preserved. Keywords, UIDs, UIDVALIDITY etc is preserved if you copy the dovecot-* files with it (which is a good idea to do). From calestyo at scientia.net Mon Oct 29 23:15:30 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:15:30 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> Message-ID: <1351545330.3435.66.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:06 +0200, Timo Sirainen wrote: > There is of course mdbox also, which gives the best of both mbox and maildir (and some of its own new annoyances). Thanks, Timo,... I forgot to mention that. For me _personally_ two things speak against using it: a) To be honest, "you must not lose the dbox index files, they can't be regenerated without data loss"[0] made me a bit scared ;-) b) ext* has no integrity checking (by hash sums) so I used to create my own that puts SHA512 hashes into the inodes of files (as USER_XATTRS). This of course, works only when you have a storage format where files don't change anymore once written,... which can't work with formats having multiple mails per file. Thanks, Chris. btw: What are the actual advantages of sdbox over maildir? [0] http://wiki2.dovecot.org/MailboxFormat/dbox -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Mon Oct 29 23:20:27 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:20:27 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> Message-ID: <1351545627.3435.71.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:13 +0200, Timo Sirainen wrote: > > Great... and I expect that this ("no headers from the maildir files > are > > interpreted") applies also, when one "imports" mails the first time. > > > > With import I don't mean via IMAP, but plainly moving e.g. a maildir > ++ > > tree under dovecots mail location. > > Yeah. So that means: From a "security" point of view, when using maildir (!) there's no need to remove such headers, cause dovcote ignores them (on maildir) always. Right?! I just wondered because when I looked through my mail archive (currently as mentioned, under Evolution)... many emails already had X-UID and X-IMAP* headers.... (set by the remote side, not by Evolution) ... and in no case these should be able to mess around in my dovecot :) > UIDs, UIDVALIDITY etc is preserved if you copy the dovecot-* files > with it (which is a good idea to do). I'll have a question on that too, but ask it under a separate mail in a few minutes,.. cause it doesn't fit this thread anymore ;) Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From limon at koli.be Mon Oct 29 23:26:48 2012 From: limon at koli.be (Levent Dane) Date: Mon, 29 Oct 2012 16:26:48 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> <20121029181700.GA4240@leningrad.koli.be> <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> Message-ID: <20121029212648.GA4292@leningrad.koli.be> On 10/29, Timo Sirainen wrote: >On 29.10.2012, at 20.17, Levent Dane wrote: > >> On 10/29, Timo Sirainen wrote: >>> On 18.10.2012, at 9.03, Levent Dane wrote: >>> >>>>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >>>> >>>> in Code/dovecot-virtual: >>>> Archive >>>> inthread refs keyword code not deleted >>> >>> I still couldn't reproduce with this. >> >> I think the problem is mail-search.c is corrupting the index files. > >Not that itself, but yeah looks like if virtual plugin assert-crashes in mail-search.c it leaves the indexes so that the next access will segfault. > >> I compiled with -ggdb flag. I'm getting this informations >> /var/log/messages: http://pastebin.com/bpkvp4Ak >> and from gdb: http://pastebin.com/HY0mVYBS > >Better backtrace than last time, but still no debug information in the backtrace. Maybe that got stripped somewhere between compiling and installing? You can check with "file ..../imap" to see if it's there. > >Also backtrace from both the mail-search.c assert crash and the segfault would be useful. I think I get correct backtrace. I attached this mail and uploaded pastebin. http://pastebin.com/L41e6AXY -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 -------------- next part -------------- Oct 29 16:21:40 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) Oct 29 16:21:40 widder dovecot: imap(limon): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x451b1) [0xb76911b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4521f) [0xb769121f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7660d4e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x3b845) [0xb770a845] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_search_result_update_flags+0xe3) [0xb77320d3] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_sync_search_results_update+0x69) [0xb77394f9] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mailbox_sync_deinit+0x1f5) [0xb7738855] -> /usr/lib/dovecot/lib20_fts_plugin.so(+0xa0a6) [0xb74970a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x3a) [0xb770f7fa] -> /usr/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0xbf2) [0xb7487ac2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x39) [0xb770f769] -> dovecot/imap(imap_sync_init+0x54) [0x8060294] -> dovecot/imap() [0x8052262] -> dovecot/imap(cmd_idle+0xc3) [0x80523f3] -> dovecot/imap(command_exec+0x3d) [0x80591cd] -> dovecot/imap() [0x805815f] -> dovecot/imap() [0x8058230] -> dovecot/imap(client_handle_input+0x12d) [0x805847d] -> dovecot/imap(client_input+0x5f) [0x8058daf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x42) [0xb769ff92] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd3) [0xb76a0f43] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0xb769fa30] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2d) [0xb768880d] -> dovecot/imap(main+0x2b1) [0x8061c71] -> /lib/libc.so.6(__libc_start_main+0xe7) [0xb74de573] -> dovecot/imap() [0x804fa51] Oct 29 16:21:40 widder dovecot: imap(limon): Fatal: master: service(imap): child 8060 killed with signal 6 (core dumped) -------------- next part -------------- #0 0xf57fe416 in __kernel_vsyscall () No symbol table info available. #1 0xb74f1a1a in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb74f3014 in abort () from /lib/libc.so.6 No symbol table info available. #3 0xb76911c5 in default_fatal_finish (type=, status=) at failures.c:191 backtrace = 0x8df75a8 "/usr/lib/dovecot/libdovecot.so.0(+0x451b1) [0xb76911b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4521f) [0xb769121f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7660d4e] -> /usr/lib/dovecot/libdo"... #4 0xb769121f in i_internal_fatal_handler (ctx=0xbfaff584, format=0xb7767320 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfaff5a4 "?v\267Z") at failures.c:649 status = 0 #5 0xb7660d4e in i_panic (format=0xb7767320 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = 0xbfaff5a4 "?v\267Z" #6 0xb770a845 in mail_search_args_init_sub (args=, arg=0x8e6cee8, change_uidsets=false, search_saved_uidset=0x0) at mail-search.c:90 thread_args = keywords = {0x8e6cf40 "lists", 0x0} __FUNCTION__ = "mail_search_args_init_sub" #7 0xb77320d3 in search_result_update_search (changed_uids_arr=0x8e87030, result=0x8e58ed0) at index-search-result.c:69 search_ctx = changed_uids = 0x8e46b30 next_uid = 29224 ret = t = mail = changed_count = 1 changed_idx = 0 #8 index_search_result_update_flags (result=0x8e58ed0, uids=0x8e87030) at index-search-result.c:131 search_arg = {next = 0x8e6cee8, type = SEARCH_UIDSET, value = {subargs = 0x0, seqset = {arr = { buffer = 0x8e740e8, element_size = 8}, v = 0x8e740e8, v_modifiable = 0x8e740e8}, str = 0x0, time = 0, size = 0, flags = 0, search_flags = 0, date_type = 0, thread_type = MAIL_THREAD_NONE, keywords = 0x0, modseq = 0x0, search_args = 0x0, search_result = 0x0, mailbox_glob = 0x0}, context = 0x0, hdr_field_name = 0x0, match_not = 0, match_always = 0, nonmatch_always = 0, fuzzy = 0, result = 0} ret = 0 __FUNCTION__ = "index_search_result_update_flags" #9 0xb77394f9 in search_result_update (result=0x8e58ed0, ctx=0x8e87010) at index-sync-search.c:75 No locals. #10 index_sync_search_results_update (ctx=0x8e87010) at index-sync-search.c:88 results = 0x9060740 i = count = 3 #11 0xb7738855 in index_mailbox_sync_deinit (_ctx=0x8e87010, status_r=0xbfaffa3c) at index-sync.c:386 ctx = 0x8e87010 sync_rec = {seq1 = 3077094660, seq2 = 148987872, type = 0} delayed_expunges = false ret = 0 #12 0xb74970a6 in fts_sync_deinit (ctx=0x8e87010, status_r=0xbfaffa3c) at fts-storage.c:584 box = 0x9060580 fbox = 0x9060898 flist = 0x8e16060 ret = 0 #13 0xb770f7fa in mailbox_sync_deinit (_ctx=0xbfaffa40, status_r=0xbfaffa3c) at mail-storage.c:1347 ctx = box = 0x9060580 errormsg = error = ret = #14 0xb7487ac2 in virtual_sync_backend_box_sync (sync_flags=, bbox=0x8e632a8, ctx=0x8e8dda8) at virtual-sync.c:973 uidmap = sync_rec = {seq1 = 22114, seq2 = 22114, type = MAILBOX_SYNC_TYPE_FLAGS} idx1 = vuid = sync_ctx = 0x0 sync_status = {sync_delayed_expunges = 0} idx2 = vseq = 149273152 #15 virtual_sync_backend_box (bbox=0x8e632a8, ctx=0x8e8dda8) at virtual-sync.c:1067 sync_flags = status = {messages = 3077174859, recent = 148890672, unseen = 152726112, uidvalidity = 3215980904, uidnext = 3077353460, first_unseen_seq = 149046960, first_recent_uid = 149450720, last_cached_seq = 543664, highest_modseq = 13217152038154990465, keywords = 0x8df63a0, permanent_flags = 3077174635, nonpermanent_modseqs = 0, permanent_keywords = 0, allow_new_keywords = 1} ret = #16 virtual_sync_backend_boxes (ctx=0x8e8dda8) at virtual-sync.c:1399 bboxes = 0x9079798 i = count = 1 #17 virtual_sync (flags=0, mbox=0x8e62e18) at virtual-sync.c:1496 ctx = 0x8e8dda8 index_sync_flags = ret = #18 virtual_storage_sync_init (box=0x8e62e18, flags=0) at virtual-sync.c:1516 mbox = 0x8e62e18 sync_ctx = ret = #19 0xb770f769 in mailbox_sync_init (box=0x8e62e18, flags=0) at mail-storage.c:1324 _data_stack_cur_id = 4 ctx = #20 0x08060294 in imap_sync_init (client=0x8e17628, box=0x8e62e18, imap_flags=0, flags=0) at imap-sync.c:142 ctx = 0x8e5ba40 __FUNCTION__ = "imap_sync_init" #21 0x08052262 in idle_sync_now (box=, ctx=0x8e17eb8) at cmd-idle.c:145 __FUNCTION__ = "idle_sync_now" #22 0x080523f3 in cmd_idle (cmd=0x8e17e30) at cmd-idle.c:276 client = 0x8e17628 ctx = 0x8e17eb8 #23 0x080591cd in command_exec (cmd=0x8e17e30) at imap-commands.c:148 hook = 0x8dff260 ret = #24 0x0805815f in client_command_input (cmd=0x8e17e30) at imap-client.c:682 client = 0x8e17628 command = __FUNCTION__ = "client_command_input" #25 0x08058230 in client_command_input (cmd=0x8e17e30) at imap-client.c:733 client = 0x8e17628 command = __FUNCTION__ = "client_command_input" #26 0x0805847d in client_handle_next_command (remove_io_r=, client=0x8e17628) at imap-client.c:774 size = 12 #27 client_handle_input (client=0x8e17628) at imap-client.c:786 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #28 0x08058daf in client_input (client=0x8e17628) at imap-client.c:825 cmd = output = 0x8e16bfc bytes = 12 __FUNCTION__ = "client_input" #29 0xb769ff92 in io_loop_call_io (io=0x8f49090) at ioloop.c:379 ioloop = 0x8dfe400 t_id = 2 #30 0xb76a0f43 in io_loop_handler_run (ioloop=0x8dfe400) at ioloop-epoll.c:213 ctx = 0x8dfe5e0 events = event = 0x8dfe620 list = 0x8e16c90 io = tv = {tv_sec = 1791, tv_usec = 756031} events_count = 148991120 msecs = 1 ret = 1 i = j = call = #31 0xb769fa30 in io_loop_run (ioloop=0x8dfe400) at ioloop.c:398 No locals. #32 0xb768880d in master_service_run (service=0x8dfe330, callback=0x80612f0 ) at master-service.c:543 No locals. #33 0x08061c71 in main (argc=1, argv=0x8dfe1c0) at main.c:389 set_roots = {0x80645e0, 0x0} login_set = {auth_socket_path = 0x8df6060 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x8061720 , failure_callback = 0x8061430 } service_flags = storage_service_flags = username = c = From tss at iki.fi Mon Oct 29 23:39:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:39:33 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351545627.3435.71.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> Message-ID: <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> On 29.10.2012, at 23.20, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:13 +0200, Timo Sirainen wrote: >>> Great... and I expect that this ("no headers from the maildir files >> are >>> interpreted") applies also, when one "imports" mails the first time. >>> >>> With import I don't mean via IMAP, but plainly moving e.g. a maildir >> ++ >>> tree under dovecots mail location. >> >> Yeah. > So that means: From a "security" point of view, when using maildir (!) > there's no need to remove such headers, cause dovcote ignores them (on > maildir) always. Right?! Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). > I just wondered because when I looked through my mail archive (currently > as mentioned, under Evolution)... many emails already had X-UID and > X-IMAP* headers.... (set by the remote side, not by Evolution) ... and > in no case these should be able to mess around in my dovecot :) If you migrated from mbox format it could have brought those headers to maildir. They're anyway not used for anything by Dovecot. From tss at iki.fi Mon Oct 29 23:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:42:28 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351545330.3435.66.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> Message-ID: <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: > btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. From jk at jkart.de Mon Oct 29 23:43:08 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 22:43:08 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> Message-ID: <508EF86C.5070202@jkart.de> am 29.10.12 16:41 schrieb Timo Sirainen : > On 29.10.2012, at 17.23, Jim Knuth wrote: > >> I have here a problem with managesieve. With the login about >> webmail (roundcube) comes here in the log: > > You can always easily reproduce this? Can you get the network traffic logs between Roundcube and Dovecot and reproduce it by sending those same commands manually? > >> --snip >> dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) > > Looks like there's a bug somewhere.. Doesn't it log a "raw backtrace"? > >> --snap >> I've increased of 2048M and the same above. >> then with 4096 M happens the following >> >> --snip >> dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) >> --snap > > Probably related to the first error. It would be helpful to get gdb backtraces from both of them, although from the first one you couldn't without patching + recompiling Dovecot. But from the second one I think you can get a core dump with: > > service managesieve-login { > executable = managesieve-login -D > } > > Then you can do something like: > > gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/managesieve-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/managesieve-login...done. (no debugging symbols found)...done. /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. (gdb) > bt full bt full No stack. (gdb) > -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning. [Churchill] From tss at iki.fi Mon Oct 29 23:46:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:46:05 +0200 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508EF86C.5070202@jkart.de> References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> Message-ID: On 29.10.2012, at 23.43, Jim Knuth wrote: > ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core > /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. You'll of course need to have the core file first. Instead of: >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) It should say (core dumped). Besides the executable change, you'll need to run "ulimit -c unlimited" just before dovecot binary. From calestyo at scientia.net Mon Oct 29 23:52:54 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:52:54 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> Message-ID: <1351547574.3435.74.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:39 +0200, Timo Sirainen wrote: > Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). Great... I think it would worth adding all this to: http://wiki2.dovecot.org/MailboxFormat/Maildir Is the wiki open for public editing (after registering an account)? Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Mon Oct 29 23:54:42 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:54:42 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> Message-ID: <1351547682.3435.76.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: > > btw: What are the actual advantages of sdbox over maildir? > > * Not moving files from new/ to cur/ directory > * Not renaming files when changing message flags > * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) > > Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. Oh that's quite some advantage... And I guess the interior of the files is the same? I.e. just the plain mail without any changes or quoting? For sdbox, does that part with "loosing the indexes means game over" ;) , too? Thanks, Chris -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jk at jkart.de Mon Oct 29 23:58:42 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 22:58:42 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> Message-ID: <508EFC12.4000509@jkart.de> am 29.10.12 22:46 schrieb Timo Sirainen : > On 29.10.2012, at 23.43, Jim Knuth wrote: > >> ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core >> /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. > > You'll of course need to have the core file first. Instead of: > >>> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) > > It should say (core dumped). Besides the executable change, you'll need to run "ulimit -c unlimited" just before dovecot binary. > If I run "ulimit -c unlimited" no problems more with Managesieve Login over Roundcube: Oct 29 22:50:46 srv1 dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=9354, secured, session= Oct 29 22:50:46 srv1 dovecot: managesieve(web1p1): Disconnected: Logged out bytes=120/177 Oct 29 22:53:16 srv1 dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=9418, secured, session= Oct 29 22:53:16 srv1 dovecot: managesieve(web1p1): Disconnected: Logged out bytes=44/145 But the same: srv1:~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/managesieve-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/managesieve-login...done. (no debugging symbols found)...done. /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. (gdb) bt full No stack. (gdb) q -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Ein Tag an dem Du nicht l?chelst, ist ein verlorener Tag. (Charly Chaplin) From tss at iki.fi Tue Oct 30 00:05:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 00:05:42 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351547682.3435.76.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <1351547682.3435.76.camel@fermat.scientia.net> Message-ID: On 29.10.2012, at 23.54, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: >>> btw: What are the actual advantages of sdbox over maildir? >> >> * Not moving files from new/ to cur/ directory >> * Not renaming files when changing message flags >> * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) >> >> Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. > > Oh that's quite some advantage... > > And I guess the interior of the files is the same? I.e. just the plain > mail without any changes or quoting? Yes, but it's in dbox format so it contains also some extra metadata (not in the mail headers). > For sdbox, does that part with "loosing the indexes means game > over" ;) , too? You'll lost message flags then. Both sdbox and mdbox keep dovecot.index.backup files and repairing tries very hard to preserve everything from the indexes it sees, so I don't think it's a big concern as long as the system behaves properly. From tss at iki.fi Tue Oct 30 00:08:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 00:08:28 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351547574.3435.74.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> Message-ID: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> On 29.10.2012, at 23.52, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:39 +0200, Timo Sirainen wrote: >> Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). > > Great... I think it would worth adding all this to: > http://wiki2.dovecot.org/MailboxFormat/Maildir Well, that isn't really maildir-specific. It's pop3 specific that is done with all mailbox formats. pop3_reuse_xuidl setting's comments should probably warn about the possibility of receiving unwanted X-UIDL headers in new mails. > Is the wiki open for public editing (after registering an account)? Yes. You don't even need to register. From calestyo at scientia.net Tue Oct 30 00:16:55 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 23:16:55 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> Message-ID: <1351549015.3435.80.camel@fermat.scientia.net> On Tue, 2012-10-30 at 00:08 +0200, Timo Sirainen wrote: > > Great... I think it would worth adding all this to: > > http://wiki2.dovecot.org/MailboxFormat/Maildir > > Well, that isn't really maildir-specific. It's pop3 specific that is > done with all mailbox formats. pop3_reuse_xuidl setting's comments > should probably warn about the possibility of receiving unwanted > X-UIDL headers in new mails. No I meant _everything_.. i.e. that dovecote never interprets these message headers when using maildir... unless for that one case when using POP3 on maildir and when pop3_reuse_xuidl is set to yes. I'll make some chances and post you the diff links here, so you can check this and correct if something's wrong. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From janfrode at tanso.net Tue Oct 30 00:26:29 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 29 Oct 2012 23:26:29 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> +1 Better to be lenient, than to confuse users by accepting some but not other messages. I believe most larger mail providers has a max message size of around 64MB or less, so allowing the final message to exceed quota by about that sounds reasonable to me. -jf From calestyo at scientia.net Tue Oct 30 00:31:20 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 23:31:20 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> Message-ID: <1351549880.3435.81.camel@fermat.scientia.net> I think it should be configurable by how much (either a fixed space or relative to the quota) the last mail may be larger than the quota.... but then... +1 as well :) Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Tue Oct 30 01:13:45 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 00:13:45 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> Message-ID: <1351552425.3435.83.camel@fermat.scientia.net> Please have a look at: http://master.wiki2.dovecot.org/MailboxFormat/mbox?action=diff&rev2=17&rev1=16 http://master.wiki2.dovecot.org/MailboxFormat/Maildir?action=diff&rev2=45&rev1=44 whether it's correct. Oh and... I'd assume that everything I've added for maildir also applies to the dbox formats? If so, I'd add the text there, too. Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From bob at computerisms.ca Tue Oct 30 01:23:16 2012 From: bob at computerisms.ca (Bob Miller) Date: Mon, 29 Oct 2012 16:23:16 -0700 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <1351552996.2097.57.camel@worklian> +1 to one last mail, though it would be nice if the over percentage could be configurable... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. > From sven at svenhartge.de Tue Oct 30 01:36:08 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 30 Oct 2012 00:36:08 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail > that is over 1MB, Dovecot rejects the mail. But smaller mails aren't > rejected probably for days. So user might not even realize that they > didn't receive one of the mails. Also having a user "almost over > quota" is a rather strange state I think. > So what do you think about v2.2 allowing delivery of one last mail > even if it brings the user over quota? Except add a limit that if the > message size is as much as the user's entire quota limit it wouldn't > be added (or 50% or ..?). Also IMAP wouldn't allow this, since user > would get an error anyway. I could make this also optional, but if > nobody really wants to keep the old behavior there's really no point > in adding the option. Yes, please add this new option. If possible with configurable limit. I'd rather have a user go directly over quota with one final mail than have a situation where half the mails get delivered and the other half is rejected. >From a 1st level support stand point this new behavior is easier to explain than the way it is now. By looking into my new crytal ball I can see the following happening: A user with 300KBytes under his quota gets a mail with 500KBytes in size. This of course bounces. He is then called by the sender who complains about the full mailbox. The user then sends himself a test mail (Subject: Test, Body: Test) which is delivered, because it is rather small and fits inside the few bytes left. The user then is confused. (And I have to use some of my precious time to explain to the user the inner workings of the mail system. ;)) So I'd very much appreciate such an option. Gr??e, Sven. -- Sigmentation fault. Core dumped. From noel.butler at ausics.net Tue Oct 30 01:43:30 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 30 Oct 2012 09:43:30 +1000 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <1351554210.7884.5.camel@tardis> On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? +1 only if configurable, and with an additional configurable quota percentage value option for those that do enable the function. In 99.9% of cases I could never see a service provider wanting this, but some small private businesses perhaps might see a benefit in it. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From sven at svenhartge.de Tue Oct 30 01:48:32 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 30 Oct 2012 00:48:32 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1351554210.7884.5.camel@tardis> Message-ID: Noel Butler wrote: > On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: >> So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? > +1 only if configurable, and with an additional configurable quota > percentage value option for those that do enable the function. > In 99.9% of cases I could never see a service provider wanting this, > but some small private businesses perhaps might see a benefit in it. If your user quota is 1GiB (which is not big, if you look at todays user quotas even at freemail providers) and the max mail size 30MiB, then a users max mailbox size would then be 1054MiB. Not an unreasonable price to pay for an easier to understand error condition, IMHO. Gr??e, Sven. -- Sigmentation fault. Core dumped. From calestyo at scientia.net Tue Oct 30 02:16:05 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 01:16:05 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <1351547682.3435.76.camel@fermat.scientia.net> Message-ID: <1351556165.3435.88.camel@fermat.scientia.net> On Tue, 2012-10-30 at 00:05 +0200, Timo Sirainen wrote: > > And I guess the interior of the files is the same? I.e. just the plain > > mail without any changes or quoting? > Yes, but it's in dbox format so it contains also some extra metadata (not in the mail headers). Yeah of course... but the important point here is the "not in the mail headers" part :) So I've added the following changes, please double check :) http://master.wiki2.dovecot.org/MailboxFormat/dbox?action=diff&rev2=30&rev1=29 > > For sdbox, does that part with "loosing the indexes means game > > over" ;) , too? > You'll lost message flags then. Both sdbox and mdbox keep > dovecot.index.backup files and repairing tries very hard to preserve > everything from the indexes it sees, so I don't think it's a big > concern as long as the system behaves properly. Yeah... sounds not too bad... :) Off topic: Have you ever thought about adding a "real" DB backend? Nothing against dbox... :) ... and I have no performance comparison of dbox with what could be done with a DBMS... but the advantage of the later would be that you get all fancy features from database systems for free... like fast indexing, online replication, etc. p.. One might even reuse something like AOX for this. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From noel.butler at ausics.net Tue Oct 30 02:27:58 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 30 Oct 2012 10:27:58 +1000 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1351554210.7884.5.camel@tardis> Message-ID: <1351556878.7884.20.camel@tardis> On Tue, 2012-10-30 at 00:48 +0100, Sven Hartge wrote: > Noel Butler wrote: > > On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > > >> So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? > > > +1 only if configurable, and with an additional configurable quota > > percentage value option for those that do enable the function. > > > In 99.9% of cases I could never see a service provider wanting this, > > but some small private businesses perhaps might see a benefit in it. > > If your user quota is 1GiB (which is not big, if you look at todays user > quotas even at freemail providers) and the max mail size 30MiB, then a > users max mailbox size would then be 1054MiB. > > Not an unreasonable price to pay for an easier to understand error > condition, IMHO. > Sven , That's nice when it's one or ten, but you need to look at the big picture, what about 300K users, all doing the same. Also, as to mail sizes, in decades gone by with dialup it was 5mb, now days with DSL, Cable, FTTN etc, many that I know of use 50mb mail sizes because that takes mere seconds now days. Don't forget, in some countries, hardware is still incredibly (criminally) overpriced, a 600G drive from HP in the U.S. is about 350 odd last time I looked, probably lot cheaper now, in this country (AU), the same drive today is still around 800, and that was when our dollar was 1.07 to the U.S. 1.00, even with taxes and customs and transport, some so and so's are still making an absolute massive killing in profits. Of course the more appropriate way would be like most of us do now, send the warning messages, if the users can not be bothered to keep an eye on their quota or act when they get mailbox almost/now full warnings, why is it our problem :) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From calestyo at scientia.net Tue Oct 30 02:42:25 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 01:42:25 +0100 Subject: [Dovecot] mbox2mdir... what about UIDs/etc? (was: how to best import Evolution/Thunderbird mail into dovecot?) In-Reply-To: References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1351557745.3435.106.camel@fermat.scientia.net> Hi again :) In the meantime I made some checks[0] on how much storage one looses by using maildir (compared to mbox)... and decided that it's much but I can live with it. This of course doesn't solve my problems that I have a possibly a mix of different mbox subformats, a mix of different mail status formats (Thunderbird and Evolution)... and some 17k mails that suffered from From_ line corruption (due to Evolution, getmail and postfix either incorrectly quoting them or even intentionally using mboxo)... so I'll still need some scripting in the end. Which I'll base upon mb2md[1] respectively it's Dovecot-izsed version[2]. I diffed the two, and it seems the only differences are that the later handles the following in addition: 1) keywords (via X-IMAP, X-IMAPbase and X-Keywords) 2) UIDs, UIDVALITIDYs and UIDLASTs (via the X-IMAP, X-IMAPbase and X-UID mail headers of the mboxes 3) ,S= and ,W= tags (Guess that's it right?) Now I have some questions: to 1) I never used keywords on mails myself so far,... so if any X-Keywords headers exist, these were sent from remote. So I guess I _really want_ to ignore them (and not let remote people set my local keywords), right? to 2) I haven't had time yet to read into the IMAP4 RFC (though I'll need to do so soon),... but AFAIU the UIDs, UIDVALITIDYs and UIDLASTs are used for the server/clients to identify which message they talk about and avoid unnecessary reloading and to assure statuses are set on the right message, etc. All mails that I migrate were only used locally by one client. So I guess I can fully ignore any UID/UIDVALITIDY/UIDLAST preservation, right? So in principle I can use plain mb2md (without the dovecot mods)... and simply convert all my mboxes to maildir, put them in the dovecot mail (having the mails in the ../new dirs) location and start dovecot, right? Now will dovecot itself assign fresh consecutive UIDs to all maildir files? Or will I get into troubles? to 3) If dovecot can make use of these,.. I'm happy with having them set, but analogous to (2): If I use plain mb2md (without the dovecot mods)... and simply convert all my mboxes to maildir, put them in the dovecot mail (having the mails in the ../new dirs) location and start dovecot.... Can I make dovecot to calculate these fields by itself when it loads? Thanks, Chris. [0] http://dovecot.org/pipermail/dovecot/2012-October/069130.html [1] http://batleth.sapienti-sat.org/projects/mb2md/ [2] http://dovecot.org/tools/mb2md.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jtam.home at gmail.com Tue Oct 30 04:09:12 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 29 Oct 2012 19:09:12 -0700 (PDT) Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: Ben Morrow wrote: >> Maybe replace "/usr/bin/passwd" with htpasswd? > > Try pam_pwdfile with poppwd or some other poppassd that supports PAM. That's it! I was trying to remember the name of this PAM module. >>> and is there another way other than poppassd? >> >> Write your own PHP script -- it couldn't be more than a few dozen lines >> of code for a working skeleton. Or Google "php change password htpasswd". > > It's not as simple as you seem to think. Quite apart from getting the > password-changing itself right (have you considered what happens when > two users change their passwords at the same time? when Dovecot tries to > read the password file at the same time as you are changing it? when the > system crashes when you are halfway through rewriting the password > file?), you really shouldn't be running PHP as a user with write access > to a password file (even a virtual password file) in any case. I did consider it, and you're right, it is tricky to get it absolutely right. If robusteness and security was of utmost importance, I would abandon PHP too. I was scaling the solution to the OP's technical ability and apparent size of their operation -- if poppwd passes muster, this wouldn't be too far off. Joseph Tam From tony.blue.mailinglist at gmx.de Tue Oct 30 07:33:22 2012 From: tony.blue.mailinglist at gmx.de (tony.blue.mailinglist at gmx.de) Date: Tue, 30 Oct 2012 06:33:22 +0100 Subject: [Dovecot] dovecot-lda not correct folder Message-ID: <508F66A2.7010809@gmx.de> Hello, i use dovecot with maildir. The maildir-folder looks like this: vmail/mail/user1/cur vmail/mail/user1/.Sent vmail/mail/user1/.optionalfolder Procmail should put some definded mails in the "optionalfolder" My uses configuration does not do this: ... DELIVERMAIL="/usr/lib/dovecot/dovecot-lda" IMAP="$DELIVERMAIL -e -d $LOGNAME -m INBOX" ZUSATZORDNER="$DELIVERMAIL -e -d $LOGNAME -m .optionalfolder" ... dovecot-lda puts the mails for the optionalfolder always in the .cur (INBOX). What?s the correct dovecot-lda parameter to put the mails in the optionalfolder? Thank you! Tony From slusarz at curecanti.org Tue Oct 30 09:19:07 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 30 Oct 2012 01:19:07 -0600 Subject: [Dovecot] Save/restore IMAP session state In-Reply-To: <1351529580.13571.93.camel@hurina> References: <1351529580.13571.93.camel@hurina> Message-ID: <20121030011907.Horde.5xjiGoF5lbhQj39rg9FXuZA@bigworm.curecanti.org> Quoting Timo Sirainen : > A week ago I thought I'd see > how easy it would be to implement this. I got a basic proof of concept > working as a "X-STATE" command. [snip] > This could also be used to implement quick session state restoring for > webmails (as suggested by Michael Slusarz). Wow. We must have some sort of crazy mind-meld going on: I have been working on this concept the last few days with the idea of generating some sort of draft proposal to provoke further discussion going forward. I would say great minds think alike, but that would be giving myself too much credit. So this provided the necessary motivation to finish the draft concept. Now that I have (somewhat) figured out the RFC 2629 XML format for doing this kind of thing, hopefully this has is presented in a somewhat coherent format. The draft, which is significantly more comprehensive than your suggestion, can be found here: https://raw.github.com/slusarz/horde-sandbox/master/imap-state-draft/draft-imap-state-00.txt Not sure if this should remain the forum for discussing this concept, or if we should move to private messages (or even to the imap-protocol list). Let me know your thoughts on this (or anyone else with an interest). michael From crohmann at netcologne.de Tue Oct 30 09:53:06 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 30 Oct 2012 08:53:06 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <508F8762.4040109@netcologne.de> On 29.10.2012 21:39, Timo Sirainen wrote: > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. Great idea. This makes being over quota a stable state and makes it easier for users to understand their "problem". Regards Christian From zybi at talex.pl Tue Oct 30 11:17:03 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Tue, 30 Oct 2012 10:17:03 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <508F9B0F.30108@talex.pl> Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. > This will finally make possible to reject RCPT TO: before the message size is known instead of accepting the message and sending a bounce later (bouncing SPAM is not good). -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From Ralf.Hildebrandt at charite.de Tue Oct 30 11:42:36 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 30 Oct 2012 10:42:36 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> Message-ID: <20121030094236.GG25787@charite.de> * Jan-Frode Myklebust : > > > +1 > > Better to be lenient, than to confuse users by accepting some but not other messages. Amen to that! +1 -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From cr at sys4.de Tue Oct 30 12:11:14 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 11:11:14 +0100 Subject: [Dovecot] copymail deleted Message-ID: Hi, I had enabled an option in dovecot. mail_attachment_dir = /var/mail/virtual/copymail/attachments After a while I checked /var/mail/virtual and did some cleanup. I did not remember that copymail was specified in dovecot and erased it. Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: file_istream.stat(/var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6) failed: No such file or directory Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: istream-concat: Failed to get size of stream /var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: read() failed: Invalid argument (FETCH for mailbox INBOX UID 196) Oct 30 10:56:05 mx0 dovecot: imap(hidden): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-30 10:56:05] in=150 out=950 I have Bacula and have restored most of the stuff, but obviously not all files. That is not too important. But I do not know, how to tell dovecot that it may "forget" about files that produce a "No such file or directory" error. Can I do some "rescan/rebuild" in dovecot? Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Tue Oct 30 12:19:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 12:19:31 +0200 Subject: [Dovecot] copymail deleted In-Reply-To: References: Message-ID: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> On 30.10.2012, at 12.11, Christian R??ner wrote: > Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: file_istream.stat(/var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6) failed: No such file or directory > > I have Bacula and have restored most of the stuff, but obviously not all files. That is not too important. But I do not know, how to tell dovecot that it may "forget" about files that produce a "No such file or directory" error. > > Can I do some "rescan/rebuild" in dovecot? Currently you can't in any easy way. The easiest fix for now I think would be to write a script that reads through dbox files, parses the attachment metadata lines and recreates the missing files with the original size (e.g. sparse-0-filled). The dbox parsing can be done easily with: doveadm dump m.1 | grep ^msg.ext-ref The format is: 1*( ) If the options="-" then the byte count is the final size. If options="B" then byte count is the base64-encoded size while the original file has to be base64-decoded size. From ef at math.uni-bonn.de Tue Oct 30 12:42:36 2012 From: ef at math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) Date: Tue, 30 Oct 2012 11:42:36 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <20121030104235.GL15430@trav.math.uni-bonn.de> Sounds like a reasonable idea, but one has to keep in mind that file system quotas never work that way. So that change would make quota=fs behave differently from the rest. So it should at least be configurable, I think. From sheng-wei.lim at proximityjobs.com Tue Oct 30 11:51:55 2012 From: sheng-wei.lim at proximityjobs.com (sheng-wei.lim) Date: Tue, 30 Oct 2012 17:51:55 +0800 Subject: [Dovecot] Problem about SSL for Dovecot. Message-ID: <000001cdb684$34e16f20$9ea44d60$@proximityjobs.com> Hi All, With the below setting (cropped), the ssl certificate(signed by godaddy) don?t seems to work. It will still ask me if I want to accept this certificate. I have use the same certificate for apache host and postfix it works without any prompt. dovecot version : 2.0.19 # OS: Linux 3.2.0-32-generic-pae i686 Ubuntu 12.04.1 LTS Dovecot config: ssl = required ssl_cert = References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> Message-ID: <508FB360.5090704@Media-Brokers.com> On 2012-10-29 5:42 PM, Timo Sirainen wrote: > On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: > >> btw: What are the actual advantages of sdbox over maildir? > * Not moving files from new/ to cur/ directory > * Not renaming files when changing message flags > * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) > > Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. > I had been wanting to ask about this too... So... what are the disadvantages? -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Oct 30 13:03:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 30 Oct 2012 07:03:02 -0400 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351544049.3435.47.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> Message-ID: <508FB3E6.6030304@Media-Brokers.com> On 2012-10-29 4:54 PM, Christoph Anton Mitterer wrote: > In the end I probably changed my opinion. > ~7GB of wasted block space for all my mails is actually quite a lot, but > in days of cheap disk space it's acceptable. > And with mbox one has IMHO the major disadvantage that mailservers > (including dovecot) store some meta-data_in_ it (i.e. in the mails > themselves) , which I don't like a lot. > I still think about reports that mbox is much faster with full text > search (which sounds reasonable)... but therefore one needs probably and > database backend anyway. What makes the most sense for me is to use mbox (or mdbox) for longer term storage that you may be offloading to slower storage systems, and use maildir (or sdbox) for the new mails... Would work great as long as you have a reliable method for archiving older mails out to your slower storage. This is what I plan on doing someday... -- Best regards, Charles From simon.buongiorno at gmail.com Tue Oct 30 13:11:37 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Tue, 30 Oct 2012 07:11:37 -0400 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <20121030094236.GG25787@charite.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> <20121030094236.GG25787@charite.de> Message-ID: On Oct 30, 2012 5:43 AM, "Ralf Hildebrandt" wrote: > > * Jan-Frode Myklebust : > > > > > > +1 > > > > Better to be lenient, than to confuse users by accepting some but not other messages. > > Amen to that! +1 Surely the answer is that as soon as any mail is rejected an over-quota message is injected? That way, the quota remains as it currently is, but the user will a) be aware that he's over or nearly over quota, b) that a mail was rejected for being too big (if you inject the right over-quota message). Simon From calestyo at scientia.net Tue Oct 30 13:30:29 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 12:30:29 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <508FB360.5090704@Media-Brokers.com> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <508FB360.5090704@Media-Brokers.com> Message-ID: <1351596629.7808.5.camel@heisenberg.scientia.net> On Tue, 2012-10-30 at 07:00 -0400, Charles Marcus wrote: > So... what are the disadvantages? I (but I'm no expert) would guess that it's a dovecot-only format. No support from most other tools,... I'd guess you cannot use e.g. maildrop with it, or can you? I personally was always a bit worried, when meta-data is put in the mail... now AFAIU dbox does _not_ do this... and you can cleanly extract each unmodified mail from the dbox fail (single or multi), right? Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From calestyo at scientia.net Tue Oct 30 13:31:40 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 12:31:40 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <508FB3E6.6030304@Media-Brokers.com> References: <1351544049.3435.47.camel@fermat.scientia.net> <508FB3E6.6030304@Media-Brokers.com> Message-ID: <1351596700.7808.6.camel@heisenberg.scientia.net> On Tue, 2012-10-30 at 07:03 -0400, Charles Marcus wrote: > What makes the most sense for me is to use mbox (or mdbox) for longer > term storage that you may be offloading to slower storage systems, and > use maildir (or sdbox) for the new mails... Was also something I thought about... still the more I think about it, the more I hate, that with mbox meta-data is stored in the mails. > Would work great as long as you have a reliable method for archiving > older mails out to your slower storage. I still hope for some DB backend ;) Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From cgregoir99 at yahoo.com Tue Oct 30 14:01:47 2012 From: cgregoir99 at yahoo.com (Christian Gregoire) Date: Tue, 30 Oct 2012 12:01:47 +0000 (GMT) Subject: [Dovecot] POP3 Proxy : user format not accepted Message-ID: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Hello, I want to use Dovecot as a POP3 proxy (http://wiki.dovecot.org/HowTo/ImapProxy). All is working fine on my sample platform, except that I have plenty (several thousands) of users that login using local_part#domain, instead of local_part at domain, which is an old setting on my POP3 server. And in that case, Dovecot returns 'Authentication failed'. Here is my proxy table : mysql> select * from tbl_proxy; +--------------------+-------------+--------------------+ | user ? ? ? ? ? ? ? | host ? ? ? ?| destuser ? ? ? ? ? | +--------------------+-------------+--------------------+ | christian at mydom.fr | 10.10.100.1 | christian at mydom.fr | | christian#mydom.fr | 10.10.100.1 | christian at mydom.fr | +--------------------+-------------+--------------------+ If I login on the Dovecot proxy with the '@' version, everything is fine : root : ~> telnet?10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian at mydom.fr +OK pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) And MySQL logs show the query : 121030 12:55:28 ? ? 3 Query ? ? SELECT NULL AS password, host, destuser, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS proxy, 'Y' AS nopassword FROM tbl_proxy WHERE user = 'christian at mydom.fr' If I login on the Dovecot proxy with the '#' version, it fails : root : ~> telnet 10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian#mydom.fr +OK pass azerty42 -ERR Authentication failed. And nothing shows up the the MySQL logs. If I login directly on the POP server with the same credentials, no problem : root : ~> telnet?10.10.100.1 110 Trying 10.10.100.1... Connected to?10.10.100.1?(10.10.100.1). Escape character is '^]'. +OK Welcome to POP3 Server V 2.06. Authenticate yourself. user christian#mydom.fr +OK Password required for christian#mydom.fr pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) Any idea ? Is the '#' not internally supported in the user login by Dovecot ? Thanks in advance. Christian From tss at iki.fi Tue Oct 30 15:08:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 15:08:37 +0200 Subject: [Dovecot] POP3 Proxy : user format not accepted In-Reply-To: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> References: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Message-ID: On 30.10.2012, at 14.01, Christian Gregoire wrote: > Any idea ? Is the '#' not internally supported in the user login by Dovecot ? See auth_username_chars setting. From chris at dotchristopher.com Tue Oct 30 15:03:55 2012 From: chris at dotchristopher.com (Chris Smith) Date: Tue, 30 Oct 2012 14:03:55 +0100 Subject: [Dovecot] Dovecot does not update acl_shared_dict file Message-ID: <20121030140355.Horde.LhzrQUVMXLlQj9A7c15yx4A@www.dotchristopher.com> Hi all, Firstly, thanks for all your effort with this software. Much appreciated. I am having a slight issues trying to enable reading of other users mailboxes. The docs are a little sparse for those that aren't mailadmin heros, I wonder if anyone could please help me see where I am going wrong. I would like to allow some users to list and read the mailboxes of others. E.g: When User1 logs in, they are presented with a list of their own folders, and those of User2, User3, etc to which they have (e.g. read/list) access I followed the instructions on the wiki (for Version 1.X)- there are two concepts here: 1. Grant some [e.g. list/read] access on a mailbox folder to a particular user 2. Enable the acl_shared_dict to allow dovecot to track (and display to IMAP clients) the folders to which they have access. This will take the form of a BDB file: /var/lib/dovecot/shared-mailboxes.db This file can only be updated by using the SETACL command. *This is where I have the problem*. The SETACL command does nothing. No matter how hard I try, I cannot get dovecot to update this file. The only indication I have that the file exists and is readable by dovecot is the disappearance from the logs of the line: No acl_shared_dict setting - shared mailbox listing is disabled So I can see that dovecot knows the file is there because it does not complain any more. The file is there (because I created it myself, although it is empty: size = 0), and in a moment of weakness I made sure it could be updated (this will be changed back!): ls -l /var/lib/dovecot/shared-mailboxes.db -rwxrwxrwx 1 dovecot dovecot 0 2012-10-30 12:27 /var/lib/dovecot/shared-mailboxes.db But if I try to update the ACL, absolutely nothing happens: [ > command ] [ < reply ] > telnet localhost 143 < * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=CRAM-MD5] Email server > a login [User2] [pass] < a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH ACL RIGHTS=texk] Logged in > a SETACL Inbox [User 1] rl < a OK Setacl complete. At this time, in the dovecot log (with debug enabled): 2012-10-30 13:55:24 IMAP([User2]): Info: Namespace : Using permissions from /home/mailboxes/[domain]/[User1]: mode=0770 gid=-1 2012-10-30 13:55:24 IMAP([User2]): Info: acl vfile: reading file /home/mailboxes/[domain]/[User1]/dovecot-acl 2012-10-30 13:55:24 IMAP([User2]): Info: acl vfile: reading file /home/mailboxes/[domain]/[USer1]/dovecot-acl Can anyone please help me track down what I'm doing wrong here?! It's driving me mad! Thanks for your time if you've read this far!! As requested: dovecot --version 1.2.9 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot info_log_path: /var/log/dovecot.info log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps imap listen: 127.0.0.1:143 ssl_listen: 37.235.54.98 ssl_cert_file: /etc/ssl/dovecot.crt ssl_key_file: /etc/ssl/private/dovecot.key login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_greeting: Email server valid_chroot_dirs: /var/spool/vmail mail_location: maildir:/home/mailboxes/%d/%n mail_debug: yes mbox_write_locks: fcntl dotlock mail_plugins: acl imap_acl lda: postmaster_address: [ valid at ddress ] auth default: mechanisms: plain cram-md5 verbose: yes passdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: passwd-file args: /etc/dovecot/users socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix master: path: /var/spool/postfix/private/auth-master mode: 384 user: postfix group: postfix plugin: acl: vfile acl_shared_dict: file:/var/lib/dovecot/shared-mailboxes.db acl: vfile:/etc/dovecot/acls From cr at sys4.de Tue Oct 30 15:28:22 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 14:28:22 +0100 Subject: [Dovecot] copymail deleted In-Reply-To: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> Message-ID: <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> > The format is: > > 1*( ) > > If the options="-" then the byte count is the final size. If options="B" then byte count is the base64-encoded size while the original file has to be base64-decoded size. Ok, so far I have "grep'ed" this here: msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 1443213 550635 B76 56/f2/56f25e225385902f3fc5185dc3d0103f59b34d14-b134401e794009503a0400002cb72ff6 1994019 477177 B76 c4/36/c436874b56cf3cd105e82f9243c7eac53c467f32-b234401e794009503a0400002cb72ff6 2561522 1075531 B76 77/af/77af1045a783308dbbf2f8a464c5136a0407e720-b334401e794009503a0400002cb72ff6 3715582 1195635 B76 99/33/99339b17a21ce052cd8f47f1d88c6e869cc1650b-b434401e794009503a0400002cb72ff6 4966686 715386 B76 fe/df/fedf23091720d3fa649af3bd6537e66304b8061a-b534401e794009503a0400002cb72ff6 5805913 788086 B76 ab/36/ab36f53a443f1855bc13caaba9e01e9464b2921f-b634401e794009503a0400002cb72ff6 6684258 906273 B76 10/70/1070d21039bc3f305bb948315a01344eefb2a465-b734401e794009503a0400002cb72ff6 7590707 204613 B76 39/44/394402c057791482f79351363f025ae0a7caf1b0-b834401e794009503a0400002cb72ff6 7795492 1349911 B76 41/bd/41bd01b4880065e5136cafbd1d191a1f8a1ead55-b934401e794009503a0400002cb72ff6 9271435 1504539 B76 c6/71/c671c1367e843741a2cc8f083a37231522d37640-ba34401e794009503a0400002cb72ff6 10877759 357555 B76 58/f5/58f582d2644025b843cf991f5cf783d27f9d90c9-bb34401e794009503a0400002cb72ff6 11826037 890683 B76 82/da/82dabbe06f269e7c79417db3b570246a648d2139-bc34401e794009503a0400002cb72ff6 msg.ext-ref = 118947 317624 B76 ad/9b/ad9be52e11433cd0337cda13bf0a458fd0fd948d-df905c0cd33d0950ae7800002cb72ff6 436770 139669 B76 78/15/781526d896a0530a5e76ebce65f2eb690d102dd3-e0905c0cd33d0950ae7800002cb72ff6 576610 457829 B76 61/3a/613a70c8515c572a04211fb0c63828d9c9acfb70-e1905c0cd33d0950ae7800002cb72ff6 1107667 410786 B76 7f/6b/7f6b7ee9b08a73600d98e8583aae343a90e76b96-e2905c0cd33d0950ae7800002cb72ff6 1611186 816686 B76 ff/ff/ffff9362c5356d8bedb17bd56edf0524bd0ae7b3-e3905c0cd33d0950ae7800002cb72ff6 2516232 643918 B76 4f/aa/4faa153fada5ceea79016cf2eadc1d05110f3f2e-e4905c0cd33d0950ae7800002cb72ff6 3291363 1036359 B76 e6/f3/e6f342bf28e8edfd3214666aaa52f0c067bae22b-e5905c0cd33d0950ae7800002cb72ff6 4418344 668813 B76 20/78/2078c98fb9bcadeeaa49bc38dc31548142fc71b1-e6905c0cd33d0950ae7800002cb72ff6 5154786 502218 B76 40/f4/40f4af3ad2077493caa34faabb201531609b50c4-e7905c0cd33d0950ae7800002cb72ff6 5782912 628591 B76 cc/a9/cca98a2a325f1be9a398d62890836cf11f267c4b-e8905c0cd33d0950ae7800002cb72ff6 6518382 526201 B76 17/47/1747a90b58c50c3d01da7f3a6601f7073cd5b163-e9905c0cd33d0950ae7800002cb72ff6 7140759 517776 B76 04/af/04afe7deb8e6ee99153433d2845da417e54cd042-ea905c0cd33d0950ae7800002cb72ff6 7769983 2317979 B76 05/13/0513bcfceff303125f233ad2c01c5ba2ed96c6a2-eb905c0cd33d0950ae7800002cb72ff6 10214312 3097649 B76 35/e4/35e46902b3e6473b9689a92acd71e58fb7165a8f-ec905c0cd33d0950ae7800002cb72ff6 msg.ext-ref = 75027 1291257 B76 b9/dc/b9dcd6899ae65e5c11b122d7bfc3be9fefc21024-5df010068b3f0950c27d00002cb72ff6 1441078 1131344 B76 f6/e6/f6e63f000d6501be472629747448057b122104c1-5ef010068b3f0950c27d00002cb72ff6 2572595 2218094 B76 93/96/9396c5eaeac2615119e55c67fa8f010332ba0fd3-5ff010068b3f0950c27d00002cb72ff6 4790862 2211695 B76 cc/a5/cca5607fb739306f3628a19575dc41432f74a22d-60f010068b3f0950c27d00002cb72ff6 7002730 2614603 B76 66/10/661002c8039997174e34b9ef31d0e693a556eebe-61f010068b3f0950c27d00002cb72ff6 9617506 2760312 B76 8c/65/8c656fe835af26c175337cd318daca8ae8e00369-62f010068b3f0950c27d00002cb72ff6 12377991 2341764 B76 19/c8/19c83e0bf1284e74e49feecaf95506266201551d-63f010068b3f0950c27d00002cb72ff6 15209343 406758 B76 b6/62/b66216837cc48422e22e7a9a22631f840a49ef78-64f010068b3f0950c27d00002cb72ff6 15616301 136877 B76 06/9f/069f5ab86dc9e8e9972f3f5c0dda03c1f3103730-65f010068b3f0950c27d00002cb72ff6 15753350 971075 B76 a7/7c/a77c36690ff0f0f774b82efaf15f93535ba027e9-66f010068b3f0950c27d00002cb72ff6 16849194 1197333 B76 4f/28/4f2881be6d0e8a7f53c0e226c0dbb148b05674c7-67f010068b3f0950c27d00002cb72ff6 18168424 850768 B76 92/72/9272e1ea7ceb79df6222686bf157f957fa9851c1-68f010068b3f0950c27d00002cb72ff6 19019393 135641 B76 60/fd/60fdcd7851c8f0a21f342aaafce9e49a3e00e1aa-69f010068b3f0950c27d00002cb72ff6 19155207 897179 B76 63/59/6359abf4f9e806e3990e0d6590e519924c838fa5-6af010068b3f0950c27d00002cb72ff6 20169966 1022612 B76 f8/65/f8654367f5df050d23565644e83c8c50abb69c39-6bf010068b3f0950c27d00002cb72ff6 But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. Thanks a lot Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Tue Oct 30 15:42:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 15:42:49 +0200 Subject: [Dovecot] copymail deleted In-Reply-To: <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> Message-ID: On 30.10.2012, at 15.28, Christian R??ner wrote: > msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 > But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. They are all base64 yes, the B76 means that all the encoded lines will be 76 chars long. So the file size above needs to be 1282212, divided by 77 (76+LF) = 16652 full lines and 8 bytes over. Base64 encodes 3 byte blocks into 4 byte chars, so the original data has (16652*76+8)/4*3 = 949170 bytes (or 1-2 bytes less, but that makes no difference because it's padded anyway). So if you create /attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 that is 949170 bytes long, and do the same for the rest of the attachments, you should be able to read this mail without errors. You can easily create the files without wasting space with: dd if=/dev/zero of=foo bs=1 seek=949169 count=1 From cr at sys4.de Tue Oct 30 16:44:01 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 15:44:01 +0100 Subject: [Dovecot] copymail deleted In-Reply-To: References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> Message-ID: <09B9ED24-9319-48A7-85D4-0FF7D12F6296@sys4.de> Hi, >> msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 > >> But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. > > They are all base64 yes, the B76 means that all the encoded lines will be 76 chars long. So the file size above needs to be 1282212, divided by 77 (76+LF) = 16652 full lines and 8 bytes over. Base64 encodes 3 byte blocks into 4 byte chars, so the original data has (16652*76+8)/4*3 = 949170 bytes (or 1-2 bytes less, but that makes no difference because it's padded anyway). > > So if you create /attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 that is 949170 bytes long, and do the same for the rest of the attachments, you should be able to read this mail without errors. > > You can easily create the files without wasting space with: > dd if=/dev/zero of=foo bs=1 seek=949169 count=1 Thanks. I have calculated both other files and recreated zero padded files. Now I am going to watch the log file and see, if errors are gone. One last question: If the user now opens a mail, where the attachments are broken and he/she removes the mail, are the created hand-made files be removed automatically? Thanks in advance Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From dmiller at amfes.com Tue Oct 30 17:00:19 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 30 Oct 2012 08:00:19 -0700 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 Message-ID: I'm compiling as I normally do. The config line for Dovecot is: configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap followed by make & make install Then a 'configure' for Pigeonhole, followed by make, yields: libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sieve-dump sieve-dump.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_new_null at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_peer_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_load_verify_locations at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_rsa_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_desc_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_new_index at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_client_CA_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_ext_d2i at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_accept at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_cert at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_server_method at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OBJ_txt2nid at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_write at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_num at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_get_write_guarantee at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_cipher_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_push at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_get_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_type_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_COMP_get_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_by_id at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_write at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `i2d_DHparams at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_ENTRY_get_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_info_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_subject_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_init at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_clear_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_load_error_strings at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_RSA at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_bits at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_INFO_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_set_flags at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_new at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_dup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_index_by_NID at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `d2i_DHparams at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_text_by_NID at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OPENSSL_add_all_algorithms_noconf at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_DSA at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_cleanup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_entry at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_type at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_library_init at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_cipher at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_cipher_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_connect at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_error_string_n at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_CTX_get_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_verify at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_bio_pair at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_client_method at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_length at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_get_cert_store at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_dh_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_ciphers at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_mem_buf at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data_X509_STORE_CTX_idx at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_pop_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_value at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `GENERAL_NAME_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_version at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RAND_bytes at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_new at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_pending at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_bio at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_load_builtin_engines at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_X509 at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_cleanup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_crl at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_read at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_finish at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_read at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_oneline at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_free_strings at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_PKEY_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_compression at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_verify_cert_error_string at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_ctrl at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_last_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RSA_generate_key at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_state_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_X509_INFO_read_bio at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_generate_parameters at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_free at OPENSSL_1.0.0' collect2: ld returned 1 exit status make[3]: *** [sieve-dump] Error 1 make[3]: *** Waiting for unfinished jobs.... -- Daniel From rob0 at gmx.co.uk Tue Oct 30 18:23:18 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 30 Oct 2012 11:23:18 -0500 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <20121030162318.GB3672@harrier.slackbuilds.org> On Mon, Oct 29, 2012 at 10:39:51PM +0200, Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver > mail that is over 1MB, Dovecot rejects the mail. But smaller mails > aren't rejected probably for days. So user might not even realize > that they didn't receive one of the mails. Also having a user > "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail > even if it brings the user over quota? Except add a limit that if > the message size is as much as the user's entire quota limit it > wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, > since user would get an error anyway. I could make this also > optional, but if nobody really wants to keep the old behavior > there's really no point in adding the option. I think the thing to do is to adjust the admin's thinking about it. Yes, if the current mailstore is under quota, by all means, you should accept the next email up to the maximum size the server accepts. No exception, just take it. You control $quota and $maxMsg. Set your quota with that in mind, where $(($quota - 1 + $maxMsg)) total is something you can live with. That said, I have been fortunate to never have to set up a quota. Storage is cheap. An occasional cron job can point out individual users who might be beyond what you'd consider reasonable, and to those users, apply a LART. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From cgregoir99 at yahoo.com Tue Oct 30 22:57:36 2012 From: cgregoir99 at yahoo.com (Christian Gregoire) Date: Tue, 30 Oct 2012 20:57:36 +0000 (GMT) Subject: [Dovecot] POP3 Proxy : user format not accepted In-Reply-To: References: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Message-ID: <1351630656.22457.YahooMailNeo@web172405.mail.ir2.yahoo.com> Indeed, I set its value to empty to allow all characters and it now works. Thanks a lot Timo. ________________________________ De?: Timo Sirainen ??: Christian Gregoire ; Dovecot Mailing List Envoy? le : Mardi 30 octobre 2012 14h08 Objet?: Re: [Dovecot] POP3 Proxy : user format not accepted On 30.10.2012, at 14.01, Christian Gregoire wrote: > Any idea ? Is the '#' not internally supported in the user login by Dovecot ? See auth_username_chars setting. From stephan at rename-it.nl Tue Oct 30 23:46:02 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 30 Oct 2012 22:46:02 +0100 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: References: Message-ID: <50904A9A.8030703@rename-it.nl> On 10/30/2012 4:00 PM, Daniel L. Miller wrote: > I'm compiling as I normally do. The config line for Dovecot is: > > configure --with-ldap --with-ssl --with-bzlib --with-zlib > --with-stemmer --with-lucene --with-ldap > > followed by make & make install > > Then a 'configure' for Pigeonhole, followed by make, yields: What kind of system are you compiling this on? Regards, Stephan. From cliff at clamjuice.org Wed Oct 31 00:29:21 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Tue, 30 Oct 2012 18:29:21 -0400 Subject: [Dovecot] Unable to get Managesieve working Message-ID: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> I have Roundcube webmail (v. 0.8.2) running with the managesieve plugin (v. 5.1). I am able to create sieve rules without any problems in Roundcube, but incoming mail is not being processed with the rule I specify. I suspect there is something that isn't configured correctly in dovecot, but unfortunately I am unable to find a resolution online and my very limited knowledge of dovecot isn't allowing me to solve this problem on my own. I am including my dovecot config below. Please keep my knowledge level in mind when suggesting options and let me know if there is any other information I can provide to help troubleshoot the problem. Thanks for any assistance! # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key ssl_cipher_list: ALL:!LOW:!SSLv2 login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_location: maildir:~/mail:LAYOUT=fs:INBOX=~/mail/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve auth default: mechanisms: plain login passdb: driver: pam passdb: driver: pam userdb: driver: passwd userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: sieve: ~/.dovecot.sieve sieve_dir: ~/mail/sieve From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 01:37:09 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 00:37:09 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> Message-ID: <20121030233709.GA14111@daniel.localdomain> Hi Cliff, Cliff Dunn wrote: > I have Roundcube webmail (v. 0.8.2) running with the managesieve > plugin (v. 5.1). I am able to create sieve rules without any > problems in Roundcube, but incoming mail is not being processed with > the rule I specify. managesieve is the service for managing your sieve rules, it does not process any emails. In order to actually sort mails into folders, you need to add the "sieve" plugin to your mail_plugins list. http://wiki.dovecot.org/LDA/Sieve/Dovecot protocol lda { .. # Support for dynamically loadable plugins. mail_plugins is a space separated # list of plugins to load. mail_plugins = sieve # ... other plugins like quota } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 02:09:57 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 01:09:57 +0100 Subject: [Dovecot] lmtp out of memory - raw backtrace Message-ID: <20121031000957.GA15191@daniel.localdomain> Hi, Our setup: - 4 hosts with director and mailbox instance - delivery via director lmtp into mailbox lmtp - mailbox format: mdbox - storage on NFS - OS: Linux 2.6.32-44-server x86_64 Ubuntu 10.04.4 LTS - Dovecot 2.1.10 - Pigeonhole 0.3.3 We're getting strange "out of memory" lmtp errors/backtrace with dovecot 2.1.10 accompanied by high load caused caused by a lot of lmtp deliveries to one user. First action would be to increase vsz_limit to a higher value, but I just want to make sure there is no bug - before blindly increasing this limit. I think it should not crash and corrupt mdbox, even if memory limit is reached: Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Fatal: pool_system_realloc(16777216): Out of memory Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4271a) [0x7f6dcbae971a] -> /usr/lib/dovecot/libdovecot.so.0(+0x42766) [0x7f6dcbae 9766] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f6dcbabd158] -> /usr/lib/dovecot/libdovecot.so.0(+0x53690) [0x7f6dcbafa690] -> /usr/lib/dovecot/libdovecot.so.0(+0x3e6f5) [0x7f6dcbae56f5] -> /usr/lib/dove cot/libdovecot.so.0(buffer_write+0x7c) [0x7f6dcbae5e7c] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8033) [0x7f6dcbdda033] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_record_map_move_to_private+0x3 c) [0x7f6dcbdda4ec] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_get_atomic_map+0x18) [0x7f6dcbde9c88] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb7cf9) [0x7f6dcbde9cf9] -> /usr/lib/dovecot/lib dovecot-storage.so.0(mail_index_sync_record+0x7e6) [0x7f6dcbdea626] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x23c) [0x7f6dcbdeae4c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_m ap+0xa8) [0x7f6dcbddc2b8] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb42da) [0x7f6dcbde62da] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb458a) [0x7f6dcbde658a] -> /usr/lib/dovecot/libdovecot-storage.so.0( mail_index_sync_begin_to+0x4f) [0x7f6dcbde6ecf] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1e) [0x7f6dcbde6f4e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_atomic_lock+0x5e) [0x 7f6dcbd6115e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_transaction_save_commit_pre+0x46) [0x7f6dcbd64fb6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9d4f3) [0x7f6dcbdcf4f3] -> /usr/lib/dovecot/libdov ecot-storage.so.0(mail_index_transaction_commit_full+0x9f) [0x7f6dcbddd97f] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x8a) [0x7f6dcbdcf18a] -> /usr/lib/dovecot/modules/lib10_quota_plug in.so(+0xba7f) [0x7f6dca4eca7f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x3d) [0x7f6d Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Fatal: master: service(lmtp): child 26698 returned error 83 (Out of memory (service lmtp { vsz_limit=256 MB }, you may need to increase it)) Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1 at example.org): Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1 at example.org): Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 10:32:27 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: Corrupted dbox file /mail/dovecot/example.org/user1/mail/storage/m.3577 (around offset=1844402): EOF reading msg header (got 0/30 bytes) Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Cron/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/postfix/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Dovecot/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:46 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: 2ZCXNsuej1CcUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHO029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: 2ZCXNsuej1CcUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: PFBjN8uej1CdUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHP029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: spYhE82ej1CiUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHQ029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: sOscMs2ej1CrUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHS029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: spYhE82ej1CiUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: PFBjN8uej1CdUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: sOscMs2ej1CrUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: +jXqE+uej1AiVAAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300933.q9U9URHi029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: +jXqE+uej1AiVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: CxDMM+uej1AnVAAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300933.q9U9URHl029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: CxDMM+uej1AnVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Regards Daniel -- https://plus.google.com/103021802792276734820 From cliff at clamjuice.org Wed Oct 31 02:17:56 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Tue, 30 Oct 2012 20:17:56 -0400 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <20121030233709.GA14111@daniel.localdomain> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> Message-ID: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Ok, so when I add the mail_plugins = sieve I get: sudo service dovecot restart Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found from directory /usr/lib/dovecot/modules/imap Error: imap dump-capability process returned 89 Fatal: Invalid configuration in /etc/dovecot/dovecot.conf failed I am assuming something is missing here? On 2012-10-30 19:37, Daniel Parthey wrote: > Hi Cliff, > > Cliff Dunn wrote: >> I have Roundcube webmail (v. 0.8.2) running with the managesieve >> plugin (v. 5.1). I am able to create sieve rules without any >> problems in Roundcube, but incoming mail is not being processed with >> the rule I specify. > > managesieve is the service for managing your sieve rules, > it does not process any emails. > > In order to actually sort mails into folders, you need to add > the "sieve" plugin to your mail_plugins list. > > http://wiki.dovecot.org/LDA/Sieve/Dovecot > > protocol lda { > .. > # Support for dynamically loadable plugins. mail_plugins is a space > separated > # list of plugins to load. > mail_plugins = sieve # ... other plugins like quota > } > > Regards > Daniel From tss at iki.fi Wed Oct 31 02:20:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 31 Oct 2012 02:20:34 +0200 Subject: [Dovecot] lmtp out of memory - raw backtrace In-Reply-To: <20121031000957.GA15191@daniel.localdomain> References: <20121031000957.GA15191@daniel.localdomain> Message-ID: On 31.10.2012, at 2.09, Daniel Parthey wrote: > - Dovecot 2.1.10 > - Pigeonhole 0.3.3 > > We're getting strange "out of memory" lmtp errors/backtrace with dovecot 2.1.10 > accompanied by high load caused caused by a lot of lmtp deliveries to one user. > > First action would be to increase vsz_limit to a higher value, but I just > want to make sure there is no bug - before blindly increasing this limit. Recent changes in hg should help with this for LMTP. I'll probably release 2.1.11 somewhat soon. > I think it should not crash and corrupt mdbox, even if memory limit is reached: It should crash, because it reached the enforced vsz limit. If you don't want it to crash you can disable the limit, at the cost of potentially eating all of your memory. > Oct 30 10:32:27 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: Corrupted dbox file /mail/dovecot/example.org/user1/mail/storage/m.3577 (around offset=1844402): EOF reading msg header (got 0/30 bytes) This does look like something that should be fixed. It's not exactly a corruption so much as finding a partially written mail during rescan (because of the crash), but still it can probably be avoided. From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 02:45:12 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 01:45:12 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Message-ID: <20121031004511.GA15928@daniel.localdomain> Cliff Dunn wrote: > >In order to actually sort mails into folders, you need to add > >the "sieve" plugin to your mail_plugins list. > > > >http://wiki.dovecot.org/LDA/Sieve/Dovecot > > > >protocol lda { > > mail_plugins = sieve > >} > Ok, so when I add the mail_plugins = sieve I get: > sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found > from directory /usr/lib/dovecot/modules/imap > Error: imap dump-capability process returned 89 > Fatal: Invalid configuration in /etc/dovecot/dovecot.conf > failed > > I am assuming something is missing here? I guess you are missing the sieve plugin completely, did you install the dovecot-sieve plugin? Where did you get sieve from and how did you install it? Did you follow all the compile instructions at http://wiki.dovecot.org/LDA/Sieve/Dovecot or did you install some package? What do the following commands say? # find /usr/lib/dovecot | grep sieve # apt-cache policy dovecot-sieve # dpkg -l "dovecot*" Regards Daniel -- https://plus.google.com/103021802792276734820 From dmiller at amfes.com Wed Oct 31 03:11:50 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 30 Oct 2012 18:11:50 -0700 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: <50904A9A.8030703@rename-it.nl> References: <508FEB83.5010209@amfes.com> <50904A9A.8030703@rename-it.nl> Message-ID: On 30.10.2012 14:46, Stephan Bosch wrote: > On 10/30/2012 4:00 PM, Daniel L. Miller wrote: > >> I'm compiling as I normally do. The config line for Dovecot is: configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap followed by make & make install Then a 'configure' for Pigeonhole, followed by make, yields: > > What kind of system are you compiling this on? AMD Opteron 4180, Ubuntu Precision, Linux 3.2.0-25, 64-bit. libssl-dev 1.0.1c-3ubuntu1 libc6 2.15 gcc 4.6.3 -- Daniel From calestyo at scientia.net Wed Oct 31 03:13:03 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 02:13:03 +0100 Subject: [Dovecot] maildir S= and W= Message-ID: <1351645983.24721.0.camel@fermat.scientia.net> Hi. Even new mails delivered by my MDA don't get the ,S= and ,W= fields set... (but when I "upload" a mail via IMAP to dovecot, they are set) Is there some place in dovecot where I need to enable this? Or would it be the MDA that has to calculate and set this already when placing a file in ./new? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5113 bytes Desc: not available URL: From calestyo at scientia.net Wed Oct 31 03:50:23 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 02:50:23 +0100 Subject: [Dovecot] maildir and end-of-line encoding Message-ID: <1351648223.24721.4.camel@fermat.scientia.net> Hi. I just wondered, the following: My MDA may get mails that use LF or CR/LF end of line encodings and deliver them into maildirs. I couldn't find any information about, whether one should or must convert all into one format, cause AFAIK at least on the IMAP side, CR/LF is always used? How does this work on the maildir/backend side of dovcot? Can it work with both and simply automatically convert LF into CR/LF? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5113 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 09:20:43 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 08:20:43 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351645983.24721.0.camel@fermat.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> Message-ID: <5090D14B.9080805@skye.it> Il 31/10/2012 02:13, Christoph Anton Mitterer ha scritto: > Hi. > > Even new mails delivered by my MDA don't get the ,S= and ,W= fields > set... > (but when I "upload" a mail via IMAP to dovecot, they are set) > > > Is there some place in dovecot where I need to enable this? Or would it > be the MDA that has to calculate and set this already when placing a > file in ./new? The MDA should calculate and set this, dovecot always add these informations, as you can see when upload file via IMAP. Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From skdovecot at smail.inf.fh-brs.de Wed Oct 31 11:08:10 2012 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 31 Oct 2012 10:08:10 +0100 (CET) Subject: [Dovecot] backtrace for non-existant %{ldap:attr} on login Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I'm fetching the user and auth data from LDAP, this is the string: pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,mailUidNumber=userdb_uid,mailGidNumber=userdb_gid,mailLocationDovecot=userdb_mail,uid=userdb_user,=userdb_quota_rule=*:bytes=%{ldap:mailQuotaBytes}, =userdb_quota_rule2=Trash:bytes=+%{ldap:mailQuotaBytesTrash} If mailQuotaBytesTrash or mailQuotaBytes is not present, the LOGIN process does not work: 1 login user pwd 1 NO [UNAVAILABLE] Temporary authentication failure. [mailsrv2:2012-10-31 08:56:51] * OK Waiting for authentication process to respond.. If I add those two attributes, the user can login successfully. dovecot-2.1.10/sbin/dovecot --version 2.1.10 (9cdeab12f3e1) The log entries: 2012-10-31 09:56:51 auth: Panic: pool_data_stack_realloc(): stack frame changed 2012-10-31 09:56:51 auth: Error: Raw backtrace: /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x4857a) [0x7f2c0528c57a] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x485c6) [0x7f2c0528c5c6] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f2c0525feaf] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x58f2e) [0x7f2c0529cf2e] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x442f5) [0x7f2c052882f5] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x68) [0x7f2c05288728] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(str_vprintfa+0x6d) [0x7f2c052a796d] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(str_printfa+0x88) [0x7f2c052a7ac8] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x42682e] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(var_expand_with_funcs+0x6fb) [0x7f2c052ac48b] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb](db_ldap_result_iterate_next+0x12f) [0x42734f] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x428974] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x428f5f] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x4282a1] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f2c05298756] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f2c0529978f] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f2c052986f8] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f2c052840d3] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb](main+0x376) [0x41bba6] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f2c04449c8d] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x40cf89] 2012-10-31 09:56:51 auth: Fatal: master: service(auth): child 15865 killed with signal 6 (core dumps disabled) 2012-10-31 09:56:51 imap-login: Warning: Auth connection closed with 1 pending requests (max 0 secs, pid=15869, EOF) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUJDqe2oxLS8a3A9mAQI+YQf/Qd4IIeM35Hmmpl1IMcQwJFK4854G5Dku yK+GsWhE2gxI6KaLO6DSI/kpN79qhQRkHsUAHzoPiZ7kQpZprNaEP/CIPkTzw//i HyC2Odpfa8fWUOqtH5Cp6X5spF6hQa4mmQyzgXguF9bdjZkNu4vQ78wRuQGG4eHi BOkNL0b93DsN3NSIoXDpiAiJn1aleTLe7mYkfsGewjb+AN+FpR4hLbf32yRRn8J0 Fkn8agEijixGMXEBD4ZprTbX9NbUr92YPfRycMnA2A00MUUlv/iKlqXRpMLafRjL bnHL/QE80xHoKKJUR96/RjruciIolAtlyYvhTU9ibRpLYm5Hcd9bZg== =iPvg -----END PGP SIGNATURE----- From fabio.ferrari at unimore.it Wed Oct 31 13:12:41 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Wed, 31 Oct 2012 12:12:41 +0100 Subject: [Dovecot] Dovecot stops to work - anvil problem In-Reply-To: <51972B14-6973-4510-870D-956F858FC76B@iki.fi> References: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> <51972B14-6973-4510-870D-956F858FC76B@iki.fi> Message-ID: <0d40f13a4256adae8f084c385dcd0fd3.squirrel@webmail2.unimore.it> Thank you very much for your help, I cross mi fingers but it seems that this was the problem. Fabio Ferrari > On 26.10.2012, at 13.24, FABIO FERRARI wrote: > >> Hi all, >> >> we have a problem about anvil, it seems that when we have a high load >> the >> dovecot stops to work. Sometimes it is sufficient to make a dovecot >> reload, but sometimes we have to restart it. >> >> Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files > > This is the problem. > >> And these are the limit settings in the OS: >> * soft nofile 131072 >> * hard nofile 131072 >> >> Have someone had the same problem? > > The OS limits are ok. But you need to make sure that the dovecot processes > have enough fds in ulimit. You can check the limits with: > > cat /proc//limits > > The "Max open files" soft limit is what you're most likely hitting. Use > "ulimit -n 10000" or something before running dovecot binary. And make > sure that it changes the limit in the proc. Many init scripts change the > ulimit internally. > > From tss at iki.fi Wed Oct 31 13:26:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 31 Oct 2012 13:26:11 +0200 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: References: Message-ID: On 30.10.2012, at 17.00, Daniel L. Miller wrote: > I'm compiling as I normally do. The config line for Dovecot is: > > configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap > > followed by make & make install > > Then a 'configure' for Pigeonhole, followed by make, yields: > libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sieve-dump sieve-dump.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot > libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot > /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_new_null at OPENSSL_1.0.0' > /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_error at OPENSSL_1.0.0' I think this is a Dovecot bug, fixed by: http://hg.dovecot.org/dovecot-2.1/rev/7d931927e4ac You could also do this by adding -lssl -lcrypto manually to the installed dovecot-config and running configure again for pigeonhole. From calestyo at scientia.net Wed Oct 31 13:41:51 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 12:41:51 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <5090D14B.9080805@skye.it> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> Message-ID: <1351683711.7825.0.camel@heisenberg.scientia.net> On Wed, 2012-10-31 at 08:20 +0100, Alessio Cecchi wrote: > The MDA should calculate and set this, dovecot always add these > informations, as you can see when upload file via IMAP. Ah thanks,... do you know whether it's possible to have them set by maildrop? I couldn't find anything on this. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 14:04:27 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 13:04:27 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351683711.7825.0.camel@heisenberg.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> Message-ID: <509113CB.7020402@skye.it> Il 31/10/2012 12:41, Christoph Anton Mitterer ha scritto: > On Wed, 2012-10-31 at 08:20 +0100, Alessio Cecchi wrote: >> The MDA should calculate and set this, dovecot always add these >> informations, as you can see when upload file via IMAP. > Ah thanks,... do you know whether it's possible to have them set by > maildrop? I couldn't find anything on this. My maildrop (2.4) version, working with qmail and vpopmail, add S= by default. Probably you are running an old version without Maildir++ support: http://www.inter7.com/courierimap/README.maildirquota.html ============== Delivering to a Maildir++ Delivering to a Maildir++ is like delivering to a Maildir, with the following exceptions: Follow the usual Maildir conventions for naming the filename used to store the message, except that append ,S=nnnnn to the name of the file, where nnnnn is the size of the file. This eliminates the need to stat() most messages when calculating the quota. If the size of the message is not known at the beginning, append ,S=nnnnn when renaming the message from tmp to new. ============== Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From cliff at clamjuice.org Wed Oct 31 14:47:10 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Wed, 31 Oct 2012 08:47:10 -0400 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <20121031004511.GA15928@daniel.localdomain> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> <20121031004511.GA15928@daniel.localdomain> Message-ID: <34eb54a518ab2a0cdff5709b9e8bfac6@mail.clamjuice.org> I didn't follow those instructions as I assumed that it would be installed with Dovecot from the Debian repositories. See output of commands below. And thanks again for the help! # find /usr/lib/dovecot | grep sieve /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.la /usr/lib/dovecot/managesieve /usr/lib/dovecot/managesieve-login # apt-cache policy dovecot-sieve N: Unable to locate package dovecot-sieve # dpkg -l "dovecot*" Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-============================-============================-======================================================================== un dovecot (no description available) ii dovecot-common 1:1.2.15-7 secure mail server that supports mbox and maildir mailboxes ii dovecot-imapd 1:1.2.15-7 secure IMAP server that supports mbox and maildir mailboxes On 2012-10-30 20:45, Daniel Parthey wrote: > Cliff Dunn wrote: >> >In order to actually sort mails into folders, you need to add >> >the "sieve" plugin to your mail_plugins list. >> > >> >http://wiki.dovecot.org/LDA/Sieve/Dovecot >> > >> >protocol lda { >> > mail_plugins = sieve >> >} >> Ok, so when I add the mail_plugins = sieve I get: >> sudo service dovecot restart >> Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found >> from directory /usr/lib/dovecot/modules/imap >> Error: imap dump-capability process returned 89 >> Fatal: Invalid configuration in /etc/dovecot/dovecot.conf >> failed >> >> I am assuming something is missing here? > > I guess you are missing the sieve plugin completely, > did you install the dovecot-sieve plugin? > Where did you get sieve from and how did you install it? > > Did you follow all the compile instructions at > http://wiki.dovecot.org/LDA/Sieve/Dovecot or > did you install some package? > > What do the following commands say? > > # find /usr/lib/dovecot | grep sieve > # apt-cache policy dovecot-sieve > # dpkg -l "dovecot*" > > Regards > Daniel From stephan at rename-it.nl Wed Oct 31 15:03:23 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 31 Oct 2012 14:03:23 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Message-ID: <5091219B.6000709@rename-it.nl> Op 10/31/2012 1:17 AM, Cliff Dunn schreef: > Ok, so when I add the mail_plugins = sieve I get: > sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found from > directory /usr/lib/dovecot/modules/imap > Error: imap dump-capability process returned 89 > Fatal: Invalid configuration in /etc/dovecot/dovecot.conf > failed > > I am assuming something is missing here? You should only put the mail_plugins=sieve inside de protocol lda {} section. Adding the Sieve plugin to IMAP makes no sense. Regards, Stephan. From calestyo at scientia.net Wed Oct 31 15:55:32 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 14:55:32 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <509113CB.7020402@skye.it> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> <509113CB.7020402@skye.it> Message-ID: <1351691732.8425.1.camel@heisenberg.scientia.net> On Wed, 2012-10-31 at 13:04 +0100, Alessio Cecchi wrote: > maildrop (2.4) Ah thanks... yeah I had an old version.. > add S= by > default. > http://www.inter7.com/courierimap/README.maildirquota.html AFAIU that... ,W= is however not set, right? :( thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 16:09:51 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 15:09:51 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351691732.8425.1.camel@heisenberg.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> <509113CB.7020402@skye.it> <1351691732.8425.1.camel@heisenberg.scientia.net> Message-ID: <5091312F.1080603@skye.it> Il 31/10/2012 14:55, Christoph Anton Mitterer ha scritto: > On Wed, 2012-10-31 at 13:04 +0100, Alessio Cecchi wrote: >> maildrop (2.4) > Ah thanks... yeah I had an old version.. > > >> add S= by >> default. >> http://www.inter7.com/courierimap/README.maildirquota.html > AFAIU that... ,W= is however not set, right? :( Yes, W= is not set by maildrop, only dovecot-lda set W=, anyway S=size is sufficient for speed up the system. If the message was stored with CR+LF linefeeds, and are the same. More info here: http://wiki2.dovecot.org/MailboxFormat/Maildir Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From jpdalbec at ysu.edu Wed Oct 31 17:15:56 2012 From: jpdalbec at ysu.edu (John Dalbec) Date: Wed, 31 Oct 2012 11:15:56 -0400 Subject: [Dovecot] IMAP proxy between Office 365 client/Sun Messaging Server Message-ID: <509140AC.80602@ysu.edu> I would like to be able to migrate messages from existing end-user accounts on Sun Messaging Server to Office 365 using an administrator login and password. Unfortunately the migration tool for Office 365 doesn't support SASL AUTHENTICATE PLAIN login. The online documentation I've found suggests that I should be able to configure dovecot as an IMAP proxy and have it log in to Sun Messaging Server with AUTHENTICATE PLAIN and encode_base64("user\0admin\0adminpw") in response to a login from the Office 365 migration tool. I'd like to configure dovecot to run only the IMAP proxy if possible. I was thinking of setting all accounts to use the same (strong) password in the proxy. Would anyone be willing to share a sample configuration? Thanks, John Dalbec ellucian Luminis system administrator Youngstown State University From rs at sys4.de Wed Oct 31 17:37:53 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 31 Oct 2012 16:37:53 +0100 Subject: [Dovecot] IMAP proxy between Office 365 client/Sun Messaging Server In-Reply-To: <509140AC.80602@ysu.edu> References: <509140AC.80602@ysu.edu> Message-ID: <509145D1.1070504@sys4.de> Am 31.10.2012 16:15, schrieb John Dalbec: > I would like to be able to migrate messages from existing end-user > accounts on Sun Messaging Server to Office 365 using an administrator > login and password. Unfortunately the migration tool for Office 365 > doesn't support SASL AUTHENTICATE PLAIN login. > > The online documentation I've found suggests that I should be able to > configure dovecot as an IMAP proxy and have it log in to Sun Messaging > Server with AUTHENTICATE PLAIN and encode_base64("user\0admin\0adminpw") > in response to a login from the Office 365 migration tool. I'd like to > configure dovecot to run only the IMAP proxy if possible. I was > thinking of setting all accounts to use the same (strong) password in > the proxy. > > Would anyone be willing to share a sample configuration? > > Thanks, > John Dalbec > ellucian Luminis system administrator > Youngstown State University perhaps look in this http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy http://wiki.dovecot.org/HowTo/ImapProxy Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From micah at riseup.net Wed Oct 31 21:15:57 2012 From: micah at riseup.net (Micah Anderson) Date: Wed, 31 Oct 2012 15:15:57 -0400 Subject: [Dovecot] Error: Internal quota calculation error Message-ID: <87d2zyxxjm.fsf@minnow.riseup.net> Hello, I'm using 2.1.7 with seive and mysql quotas. We had an outage the other day where the database server where quotas are stored was not available for a short period of time. In dovecot land, the following types of errors occured in that scenario: Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: Internal quota calculation error Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: sieve: msgid=<20122132765181x.ABCCE457 at example.com>: failed to store into mailbox 'Trash': Internal error occurred. Refer to server log for more information. [2012-10-26 22:19:01] Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: sieve: script /maildir/e/example/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /maildir/e/example/.dovecot.sieve.log may reveal additional details) I expect that there would be quota calculation errors as dovecot could not reach the database server, but what worried me was the 'failed to store into mailbox' message from sieve. The 'Trash' mailbox in this particular seive script is the correct location for the message to be filed into, but the worrisome message is the 'failed with unsuccessful implicit keep'. I looked through all the message-ids that reported this error and I found that the messages were properly delivered in the end, so this might be some issue interacting between sieve, dovecot and quota and just causing a scary message that can be ignored? thanks for any information you can provide, dovecot is great! micah -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available URL: From tlx at leuxner.net Wed Oct 31 21:46:27 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 31 Oct 2012 20:46:27 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508EFC12.4000509@jkart.de> References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> <508EFC12.4000509@jkart.de> Message-ID: Am 29.10.2012 um 22:58 schrieb Jim Knuth : > /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. Error says it all? So not really helpful. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4364 bytes Desc: not available URL: From jk at jkart.de Wed Oct 31 21:50:11 2012 From: jk at jkart.de (Jim Knuth) Date: Wed, 31 Oct 2012 20:50:11 +0100 Subject: [Dovecot] (Solved) Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> <508EFC12.4000509@jkart.de> Message-ID: <509180F3.7020608@jkart.de> am 31.10.12 20:46 schrieb Thomas Leuxner : > Am 29.10.2012 um 22:58 schrieb Jim Knuth : > >> /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. > > Error says it all? So not really helpful. > yes, I know ;) But no more problems. I dont no why *bg* -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Ich schaue mir meine Filme nie an. Sie sind mir zu brutal. (Charles Bronson) From stan at hardwarefreak.com Mon Oct 1 00:37:17 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 30 Sep 2012 16:37:17 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <5068BB8D.8020302@hardwarefreak.com> On 9/30/2012 8:02 AM, Charles Marcus wrote: > Hi Timo/everyone, > > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? > > The reason I ask is, we are changing ISPs, and I would like to see in > the logs when an external connection is coming from our OLD ISP > connection, and when it is coming through our new one. Traffic monitoring during a switchover of this nature is typically done at the edge router, not inside an individual server application. Is your router able to compile or export a daily traffic report per physical port, or raw data to your network monitoring software, showing packets/connections for TCP/UDP. Most can. This would give you the information you seek, including all traffic for both the new and old ISP, not just IMAP. -- Stan From eugene at raptor.kiev.ua Mon Oct 1 02:07:25 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 02:07:25 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50641AE7.6040201@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> Message-ID: On Thu, 27 Sep 2012 12:22:47 +0300, Davide wrote: > Hi to all, sorry in advance for my poor english, this is the first time > that i wrote to a list if i make mistake .... excuseme. > My problem is this: i have dovecot 2.1.8 installed and functioning from > 2 years one week ago i have installed crm114 for my last spam detection > filter "version 20100106-BlameMichelson (TRE 0.8.0 (BSD))" > My mail system is qmail that through .qmail default correctly tag with > X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-27CA1CFB > X-CRM114-CacheID: sfid-20120927_105129_798028_B0035817 > X-CRM114-Status: GOOD ( 28.64 ) headers ... > "Operation failed over folder 'UNSURE'. Server for account > davide.marchi at mail.cgilfe.it said: [CANNOT] Failed to call crm114 > binary.." Did you see anything in syslog? If not, let's begin with the attached patch. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix -------------- next part -------------- A non-text attachment was scrubbed... Name: crm_debug.patch Type: application/octet-stream Size: 694 bytes Desc: not available URL: From eugene at raptor.kiev.ua Mon Oct 1 02:16:25 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 02:16:25 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: References: <50641AE7.6040201@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich wrote: > Did you see anything in syslog? If not, let's begin with the attached > patch. Ouch... too sleepy. Here's the correct patch. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix -------------- next part -------------- A non-text attachment was scrubbed... Name: crm_debug.patch Type: application/octet-stream Size: 716 bytes Desc: not available URL: From tss at iki.fi Mon Oct 1 05:41:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 01 Oct 2012 05:41:00 +0300 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <1349059260.18782.42.camel@innu> On Sun, 2012-09-30 at 09:02 -0400, Charles Marcus wrote: > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? Dovecot only sees one remote IP address (%r) and one local IP address (% l) for connections. %r is already logged, but you can add %l if that helps. Other than that, I can't really help. From david.ledger at ivdcs.co.uk Mon Oct 1 10:58:30 2012 From: david.ledger at ivdcs.co.uk (David Ledger) Date: Mon, 1 Oct 2012 08:58:30 +0100 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <5068582A.6030507@brightok.net> References: <506842CB.8080501@Media-Brokers.com> <5068582A.6030507@brightok.net> Message-ID: At 09:33 -0500 30/9/12, Jack Bates wrote: >On 9/30/2012 8:02 AM, Charles Marcus wrote: >> Hi Timo/everyone, >> >> Currently we are logging the remote IP, but is there a way to show >>the IP address that the NAT connection is coming from? >> >> The reason I ask is, we are changing ISPs, and I would like to see >>in the logs when an external connection is coming from our OLD ISP >>connection, and when it is coming through our new one. >> >> We have a Watchguard firewall, and I have both External >>connections setup and working, and have just pointed our DNS >>records to the new public IP, and would like to be able to see >>which WAN connection/IP they are coming from. > >You could bind 2 internal IP Addresses to the server and have each >NAT translation go to a different internal IP. > >Jack From my remembrances of the packet layout there is nowhere in the packet for the pre-NAT address to live. The only place the mapping is stored is in the internal tables of the NAT router which has to know where to send the reply packets. David -- David Ledger - Freelance Unix Sysadmin in the UK. david.ledger at ivdcs.co.uk www.ivdcs.co.uk From CMarcus at Media-Brokers.com Mon Oct 1 11:58:15 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 01 Oct 2012 04:58:15 -0400 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <1349059260.18782.42.camel@innu> References: <506842CB.8080501@Media-Brokers.com> <1349059260.18782.42.camel@innu> Message-ID: <50695B27.8090802@Media-Brokers.com> On 2012-09-30 10:41 PM, Timo Sirainen wrote: > Dovecot only sees one remote IP address (%r) and one local IP address (% > l) for connections. %r is already logged, but you can add %l if that > helps. Other than that, I can't really help. Yeah, but that is the IP of the mail server, and since I have only one, it doesn't help any. No worries, I did see how to see this on my perimeter firewall (thanks Stan), so I can see what I'm looking for now. Thanks Timo, -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From davide.marchi at mail.cgilfe.it Mon Oct 1 12:00:14 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:00:14 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_011708_663506_8DEC5391) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) Message-ID: <50695B9E.3090502@mail.cgilfe.it> Thank you very much for the reply, i' have installed the supplied patch with the following command: - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) i compile the plugin and all go easy i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell me ..blah..blah.. [CANNOT] Failed to call crm114 binary I cant see in any log what's the problem .... Il 01/10/2012 01:16, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich > wrote: > >> Did you see anything in syslog? If not, let's begin with the attached >> patch. > > Ouch... too sleepy. Here's the correct patch. > From davide.marchi at mail.cgilfe.it Mon Oct 1 12:04:30 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:04:30 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) Message-ID: <50695C9E.5040101@mail.cgilfe.it> I downloaded and applaied the patch in the message 01:16 of 716 bytes: diff -r 7f94cc6b4d8e src/crm114.c --- a/src/crm114.c Fri May 11 04:05:59 2012 +0300 +++ b/src/crm114.c Mon Oct 01 02:15:40 2012 +0300 @@ -56,11 +56,17 @@ * really only needs the signature. */ if (pipe(pipes)) + { + i_debug("Failed to create pipes"); return -1; + } pid = fork(); if (pid < 0) + { + i_debug("Couldn't fork"); return -1; + } if (pid) { @@ -117,6 +123,7 @@ argv[i++] = spam ? cfg->spam : cfg->non_spam; + i_debug("Executing '%s %s'", cfg->binary, t_strarray_join(argv, " ")); execv(cfg->binary, (char *const *) argv); /* fall through if reaver can't be found */ i_debug("executing %s failed: %d (uid=%d, gid=%d)", cfg->binary, errno, Il 01/10/2012 11:00, Davide ha scritto: > Thank you very much for the reply, i' have installed the supplied patch > with the following command: > > - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) > i compile the plugin and all go easy > > i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell > me ..blah..blah.. [CANNOT] Failed to call crm114 binary > I cant see in any log what's the problem .... > > Il 01/10/2012 01:16, Eugene Paskevich ha scritto: >> On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich >> wrote: >> >>> Did you see anything in syslog? If not, let's begin with the attached >>> patch. >> >> Ouch... too sleepy. Here's the correct patch. >> From davide.marchi at mail.cgilfe.it Mon Oct 1 12:08:33 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:08:33 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695C9E.5040101@mail.cgilfe.it> (sfid-20121001_110525_429072_9195F28B) References: <50641AE7.6040201@mail.cgilfe.it> (sfid-20121001_011708_663506_8DEC5391) <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_110105_323231_C69C15B3) <50695C9E.5040101@mail.cgilfe.it> (sfid-20121001_110525_429072_9195F28B) Message-ID: <50695D91.3090204@mail.cgilfe.it> I have recived an empty message in response Il 01/10/2012 11:04, Davide ha scritto: > I downloaded and applaied the patch in the message 01:16 of 716 bytes: > > diff -r 7f94cc6b4d8e src/crm114.c > --- a/src/crm114.c Fri May 11 04:05:59 2012 +0300 > +++ b/src/crm114.c Mon Oct 01 02:15:40 2012 +0300 > @@ -56,11 +56,17 @@ > * really only needs the signature. > */ > if (pipe(pipes)) > + { > + i_debug("Failed to create pipes"); > return -1; > + } > > pid = fork(); > if (pid < 0) > + { > + i_debug("Couldn't fork"); > return -1; > + } > > if (pid) > { > @@ -117,6 +123,7 @@ > > argv[i++] = spam ? cfg->spam : cfg->non_spam; > > + i_debug("Executing '%s %s'", cfg->binary, t_strarray_join(argv, " ")); > execv(cfg->binary, (char *const *) argv); > /* fall through if reaver can't be found */ > i_debug("executing %s failed: %d (uid=%d, gid=%d)", cfg->binary, > errno, > > > Il 01/10/2012 11:00, Davide ha scritto: >> Thank you very much for the reply, i' have installed the supplied patch >> with the following command: >> >> - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) >> i compile the plugin and all go easy >> >> i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell >> me ..blah..blah.. [CANNOT] Failed to call crm114 binary >> I cant see in any log what's the problem .... >> >> Il 01/10/2012 01:16, Eugene Paskevich ha scritto: >>> On Mon, 01 Oct 2012 02:07:25 +0300, Eugene Paskevich >>> wrote: >>> >>>> Did you see anything in syslog? If not, let's begin with the attached >>>> patch. >>> >>> Ouch... too sleepy. Here's the correct patch. >>> From fabio.ferrari at unimore.it Mon Oct 1 12:15:14 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Mon, 1 Oct 2012 11:15:14 +0200 Subject: [Dovecot] Problem with process_limit Message-ID: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Hello, i have a problem with the process_limit configuration. Occasionally, it happens that the dovecot.log shows this line: master: Warning: service(imap): process_limit reached, client connections are being dropped So I checked, the process number, with the command: ps auxwww | grep imap | grep -v login | wc -l and it shows 1024. Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line process_limit = 1500 I checked if the dovecot had accepted the change with the command doveconf -n | grep process_limit and it shows process_limit = 1500 But it is dropping the connections when they reach 1024 anyway, the configuration parameter is totally ignored. Can anyone help? Am I editing the right parameter? thanks in advance Fabio Ferrari P.S. The version of the dovecot is dovecot-2.0.1-1_118.el5 on Red Hat Enterprise Linux Server release 5.8 (Tikanga). From dovecot at tvetc.de Mon Oct 1 12:17:48 2012 From: dovecot at tvetc.de (Karim 'Kasi Mir' Senoucci) Date: Mon, 01 Oct 2012 11:17:48 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders Message-ID: <50695FBC.6080403@tvetc.de> Hello everyone, I recently updated to dovecot 2.0.19 (in fact, I updated the whole system to Ubuntu 12.04 LTS, I am using the dovecot from the Ubuntu packages) and just today found out that the dovecot-lda for some reason doesn't auto-create missing folders anymore as it did with my old installation (1.0.10). I have a "system users" layout and send my mails through a user-specific procmail filter. Every delivery in those filters is done via dovecot-lda using a line list this: > |$DELIVER -d archive -m lists.hylafax-`date "+%Y%m"` where $DELIVER is set to > DELIVER="/usr/bin/sudo /usr/lib/dovecot/dovecot-lda" (I use the sudo because some of the procmail lines deliver the mail to folders in a different user's mailbox.) This works fine for all users, unless the mail folder to be delivered to doesn't exist yet. Here's an example of such a delivery attempt: > Oct 01 10:57:09 lda: Debug: auth input: archive > system_groups_user=archive uid=1002 gid=1002 home=/home/archive > Oct 01 10:57:09 lda(archive): Debug: Effective uid=1002, gid=1002, > home=/home/archive > Oct 01 10:57:09 lda(archive): Debug: Namespace : type=private, > prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes > location=maildir:/home/archive/Maildir > Oct 01 10:57:09 lda(archive): Debug: maildir++: > root=/home/archive/Maildir, index=, control=, > inbox=/home/archive/Maildir, alt= > Oct 01 10:57:09 lda(archive): Debug: none: root=, index=, control=, > inbox=, alt= > Oct 01 10:57:09 lda(archive): Debug: Destination address: xxx at xxxx.de > (source: user at hostname) > Oct 01 10:57:09 lda(archive): Debug: Namespace : > /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using > default permissions > Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions > from /home/archive/Maildir: mode=0755 gid=-1 > Oct 01 10:57:09 lda(archive): Debug: Namespace : > /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using > default permissions > Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions > from /home/archive/Maildir: mode=0755 gid=-1 > Oct 01 10:57:09 lda(archive): Info: > msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: save failed > to open mailbox lists.hylafax-201210: Mailbox doesn't exist: > lists.hylafax-201210 > Oct 01 10:57:09 lda(archive): Info: > msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: saved mail to > INBOX Can anyone tell me what goes wrong here and how to fix it? From what I could find out dovecot-lda should auto-create those mailbox folders, but somehow it doesn't. Any help is appreciated. Thanks in advance Kasi Mir From eugene at raptor.kiev.ua Mon Oct 1 12:20:06 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 12:20:06 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50695B9E.3090502@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 12:00:14 +0300, Davide wrote: > Thank you very much for the reply, i' have installed the supplied patch > with the following command: > > - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) > i compile the plugin and all go easy > > i move my wrongly tagged mail from UNSURE to INBOX and thunderbird tell > me ..blah..blah.. [CANNOT] Failed to call crm114 binary > I cant see in any log what's the problem .... Weird... Did you configure anything specific about logging? doveconf -n might show. You should also check your syslog configuration as to where debugging logging should go to. Otherwise I have no idea. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From robert at schetterer.org Mon Oct 1 12:46:58 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 01 Oct 2012 11:46:58 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders In-Reply-To: <50695FBC.6080403@tvetc.de> References: <50695FBC.6080403@tvetc.de> Message-ID: <50696692.3050700@schetterer.org> Am 01.10.2012 11:17, schrieb Karim 'Kasi Mir' Senoucci: > Hello everyone, > I recently updated to dovecot 2.0.19 (in fact, I updated the whole > system to Ubuntu 12.04 LTS, I am using the dovecot from the Ubuntu > packages) and just today found out that the dovecot-lda for some reason > doesn't auto-create missing folders anymore as it did with my old > installation (1.0.10). > > I have a "system users" layout and send my mails through a user-specific > procmail filter. Every delivery in those filters is done via dovecot-lda > using a line list this: > >> |$DELIVER -d archive -m lists.hylafax-`date "+%Y%m"` > > where $DELIVER is set to > >> DELIVER="/usr/bin/sudo /usr/lib/dovecot/dovecot-lda" > > (I use the sudo because some of the procmail lines deliver the mail to > folders in a different user's mailbox.) > > This works fine for all users, unless the mail folder to be delivered to > doesn't exist yet. Here's an example of such a delivery attempt: > >> Oct 01 10:57:09 lda: Debug: auth input: archive >> system_groups_user=archive uid=1002 gid=1002 home=/home/archive >> Oct 01 10:57:09 lda(archive): Debug: Effective uid=1002, gid=1002, >> home=/home/archive >> Oct 01 10:57:09 lda(archive): Debug: Namespace : type=private, >> prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes >> location=maildir:/home/archive/Maildir >> Oct 01 10:57:09 lda(archive): Debug: maildir++: >> root=/home/archive/Maildir, index=, control=, >> inbox=/home/archive/Maildir, alt= >> Oct 01 10:57:09 lda(archive): Debug: none: root=, index=, control=, >> inbox=, alt= >> Oct 01 10:57:09 lda(archive): Debug: Destination address: xxx at xxxx.de >> (source: user at hostname) >> Oct 01 10:57:09 lda(archive): Debug: Namespace : >> /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using >> default permissions >> Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions >> from /home/archive/Maildir: mode=0755 gid=-1 >> Oct 01 10:57:09 lda(archive): Debug: Namespace : >> /home/archive/Maildir/.lists.hylafax-201210 doesn't exist yet, using >> default permissions >> Oct 01 10:57:09 lda(archive): Debug: Namespace : Using permissions >> from /home/archive/Maildir: mode=0755 gid=-1 >> Oct 01 10:57:09 lda(archive): Info: >> msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: save failed >> to open mailbox lists.hylafax-201210: Mailbox doesn't exist: >> lists.hylafax-201210 >> Oct 01 10:57:09 lda(archive): Info: >> msgid=<201210010954.02813.hylafax_resp at earthshod.co.uk>: saved mail to >> INBOX > > Can anyone tell me what goes wrong here and how to fix it? From what I > could find out dovecot-lda should auto-create those mailbox folders, but > somehow it doesn't. Any help is appreciated. > > Thanks in advance > Kasi Mir my bet goes here http://wiki2.dovecot.org/LDA parameters lda_mailbox_autocreate lda_mailbox_autosubscribe -- Best Regards MfG Robert Schetterer From davide.marchi at mail.cgilfe.it Mon Oct 1 12:54:01 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 11:54:01 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_112051_451409_99CBB428) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> (sfid-20121001_112051_451409_99CBB428) Message-ID: <50696839.7090901@mail.cgilfe.it> this is my dovecot configuration for antispam plugin logging: antispam_verbose_debug = 1 antispam_debug_target = syslog and this is info_log_path = /var/log/dovecot/dovecot.log log_path = /var/log/dovecot/dovecot-err.log debug_log_path = Il 01/10/2012 11:20, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:00:14 +0300, Davide > wrote: > >> Thank you very much for the reply, i' have installed the supplied >> patch with the following command: >> >> - patch -p1 -i ../crm_debug.patch (i'm in the cloned base directory) >> i compile the plugin and all go easy >> >> i move my wrongly tagged mail from UNSURE to INBOX and thunderbird >> tell me ..blah..blah.. [CANNOT] Failed to call crm114 binary >> I cant see in any log what's the problem .... > > Weird... Did you configure anything specific about logging? > doveconf -n might show. You should also check your syslog configuration > as to where debugging logging should go to. Otherwise I have no idea. > From dovecot at tvetc.de Mon Oct 1 13:01:39 2012 From: dovecot at tvetc.de (Karim 'Kasi Mir' Senoucci) Date: Mon, 01 Oct 2012 12:01:39 +0200 Subject: [Dovecot] Problem: dovecot-lda doesn't auto-create folders In-Reply-To: <50696692.3050700@schetterer.org> References: <50695FBC.6080403@tvetc.de> <50696692.3050700@schetterer.org> Message-ID: <50696A03.3080605@tvetc.de> Hello everyone, Am 01.10.2012 11:46, schrieb Robert Schetterer: >> Can anyone tell me what goes wrong here and how to fix it? From what I >> could find out dovecot-lda should auto-create those mailbox folders, but >> somehow it doesn't. Any help is appreciated. > my bet goes here > > parameters > > lda_mailbox_autocreate > lda_mailbox_autosubscribe D*mn, I fell into the trap again. I was searching for such parameters before posting my question here, but couldn't find them - because I was looking in the 1.x docmentation, not the 2.x one. Thanks for pointing that out to me. Greetings Kasi Mir From eugene at raptor.kiev.ua Mon Oct 1 13:07:48 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 13:07:48 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <50696839.7090901@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 12:54:01 +0300, Davide wrote: > this is my dovecot configuration for antispam plugin logging: > antispam_verbose_debug = 1 > antispam_debug_target = syslog Neither my plugin nor (I believe) main dovecot reads those two. You can remove them. > and this is > info_log_path = /var/log/dovecot/dovecot.log > log_path = /var/log/dovecot/dovecot-err.log > debug_log_path = Did you notice that patch is mainly for i_debug() invocations? :-) Try sitting debug_log_path to something meaningful. And of course restart dovecot. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From davide.marchi at mail.cgilfe.it Mon Oct 1 13:57:14 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 12:57:14 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_120839_719900_279FDE63) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> (sfid-20121001_120839_719900_279FDE63) Message-ID: <5069770A.5030007@mail.cgilfe.it> I deleted antispam_verbose_debug = 1 antispam_debug_target = syslog from dovecot.conf and i added debug_log_path = /var/log/dovecot/dovecot_debug.log restarted dovecot and now i see in the debug log 2012-10-01 12:33:31 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib90_antispam_plugin.so but nothing inherit [CANNOT] Failed to call crm114 binary Il 01/10/2012 12:07, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:54:01 +0300, Davide > wrote: > >> this is my dovecot configuration for antispam plugin logging: >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog > > Neither my plugin nor (I believe) main dovecot reads those two. You can > remove them. > >> and this is >> info_log_path = /var/log/dovecot/dovecot.log >> log_path = /var/log/dovecot/dovecot-err.log >> debug_log_path = > > Did you notice that patch is mainly for i_debug() invocations? :-) > Try sitting debug_log_path to something meaningful. And of course > restart dovecot. > From davide.marchi at mail.cgilfe.it Mon Oct 1 13:58:23 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 12:58:23 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_120839_719900_279FDE63) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> (sfid-20121001_120839_719900_279FDE63) Message-ID: <5069774F.30209@mail.cgilfe.it> Parameters i.e. antispam_crm_binary = /opt/crm114/mailreaver.crm must be enclosed on ""? so antispam_crm_binary = "/opt/crm114/mailreaver.crm" Il 01/10/2012 12:07, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 12:54:01 +0300, Davide > wrote: > >> this is my dovecot configuration for antispam plugin logging: >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog > > Neither my plugin nor (I believe) main dovecot reads those two. You can > remove them. > >> and this is >> info_log_path = /var/log/dovecot/dovecot.log >> log_path = /var/log/dovecot/dovecot-err.log >> debug_log_path = > > Did you notice that patch is mainly for i_debug() invocations? :-) > Try sitting debug_log_path to something meaningful. And of course > restart dovecot. > From eugene at raptor.kiev.ua Mon Oct 1 14:00:22 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 14:00:22 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069774F.30209@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069774F.30209@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 13:58:23 +0300, Davide wrote: > Parameters i.e. antispam_crm_binary = /opt/crm114/mailreaver.crm > must be enclosed on ""? so > antispam_crm_binary = "/opt/crm114/mailreaver.crm" No. -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From eugene at raptor.kiev.ua Mon Oct 1 14:01:14 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 14:01:14 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069770A.5030007@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 13:57:14 +0300, Davide wrote: > I deleted > antispam_verbose_debug = 1 > antispam_debug_target = syslog > from dovecot.conf and i added > > debug_log_path = /var/log/dovecot/dovecot_debug.log > > restarted dovecot and now i see in the debug log > > 2012-10-01 12:33:31 imap: Debug: Module loaded: > /usr/local/lib/dovecot/lib90_antispam_plugin.so > > but nothing inherit [CANNOT] Failed to call crm114 binary Could you please post the contents of the debug file somewhere? -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From davide.marchi at mail.cgilfe.it Mon Oct 1 14:48:31 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 13:48:31 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_130205_017099_DD5462AE) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> (sfid-20121001_130205_017099_DD5462AE) Message-ID: <5069830F.4060401@mail.cgilfe.it> Can i attach compressed log in a post? I can produce output replacing crm binary with a script bash to ouput command,user etc etc Il 01/10/2012 13:01, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 13:57:14 +0300, Davide > wrote: > >> I deleted >> antispam_verbose_debug = 1 >> antispam_debug_target = syslog >> from dovecot.conf and i added >> >> debug_log_path = /var/log/dovecot/dovecot_debug.log >> >> restarted dovecot and now i see in the debug log >> >> 2012-10-01 12:33:31 imap: Debug: Module loaded: >> /usr/local/lib/dovecot/lib90_antispam_plugin.so >> >> but nothing inherit [CANNOT] Failed to call crm114 binary > > Could you please post the contents of the debug file somewhere? > From davide.marchi at mail.cgilfe.it Mon Oct 1 14:53:08 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 13:53:08 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_134931_162985_54BB8C15) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> (sfid-20121001_130205_017099_DD5462AE) <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_134931_162985_54BB8C15) Message-ID: <50698424.8080208@mail.cgilfe.it> Restarting Dovecot instead to use doveadm reload i hav strange error: managesieve(root): Fatal: getcwd() failed: No such file or directory doveconf: Error: managesieve-login: dump-capability process returned 89 Il 01/10/2012 13:48, Davide ha scritto: > Can i attach compressed log in a post? > I can produce output replacing crm binary with a script bash to ouput > command,user etc etc > > > Il 01/10/2012 13:01, Eugene Paskevich ha scritto: >> On Mon, 01 Oct 2012 13:57:14 +0300, Davide >> wrote: >> >>> I deleted >>> antispam_verbose_debug = 1 >>> antispam_debug_target = syslog >>> from dovecot.conf and i added >>> >>> debug_log_path = /var/log/dovecot/dovecot_debug.log >>> >>> restarted dovecot and now i see in the debug log >>> >>> 2012-10-01 12:33:31 imap: Debug: Module loaded: >>> /usr/local/lib/dovecot/lib90_antispam_plugin.so >>> >>> but nothing inherit [CANNOT] Failed to call crm114 binary >> >> Could you please post the contents of the debug file somewhere? >> From eugene at raptor.kiev.ua Mon Oct 1 15:04:50 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Mon, 01 Oct 2012 15:04:50 +0300 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: <5069830F.4060401@mail.cgilfe.it> References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> Message-ID: On Mon, 01 Oct 2012 14:48:31 +0300, Davide wrote: > Can i attach compressed log in a post? If it's of the sane size :-) But I'm actually interested in the output which was triggered by the mail move itself. > I can produce output replacing crm binary with a script bash to ouput > command,user etc etc Wait a second... So the script is ran correctly but reaver isn't? That probably means that either reaver dislikes its arguments or its stdin... -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From jbates at brightok.net Mon Oct 1 15:23:38 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 07:23:38 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: References: <506842CB.8080501@Media-Brokers.com> <5068582A.6030507@brightok.net> Message-ID: <50698B4A.7090604@brightok.net> On 10/1/2012 2:58 AM, David Ledger wrote: > At 09:33 -0500 30/9/12, Jack Bates wrote: >> On 9/30/2012 8:02 AM, Charles Marcus wrote: >>> Hi Timo/everyone, >>> >>> Currently we are logging the remote IP, but is there a way to show >>> the IP address that the NAT connection is coming from? >>> >>> The reason I ask is, we are changing ISPs, and I would like to see >>> in the logs when an external connection is coming from our OLD ISP >>> connection, and when it is coming through our new one. >>> >>> We have a Watchguard firewall, and I have both External connections >>> setup and working, and have just pointed our DNS records to the new >>> public IP, and would like to be able to see which WAN connection/IP >>> they are coming from. >> >> You could bind 2 internal IP Addresses to the server and have each >> NAT translation go to a different internal IP. >> >> Jack > > From my remembrances of the packet layout there is nowhere in the > packet for the pre-NAT address to live. The only place the mapping is > stored is in the internal tables of the NAT router which has to know > where to send the reply packets. > > David > > Public IP1 -> 192.168.0.33 Public IP2 -> 192.168.0.34 Now we just track the internal address in our logs, since each public IP is mapped to a different internal IP. Jack From jbates at brightok.net Mon Oct 1 15:35:03 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 07:35:03 -0500 Subject: [Dovecot] Problem with process_limit In-Reply-To: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: <50698DF7.60209@brightok.net> On 10/1/2012 4:15 AM, FABIO FERRARI wrote: > Hello, > > i have a problem with the process_limit configuration. > > Occasionally, it happens that the dovecot.log shows this line: > master: Warning: service(imap): process_limit reached, client connections > are being dropped > > So I checked, the process number, with the command: > ps auxwww | grep imap | grep -v login | wc -l > and it shows 1024. > > Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line > process_limit = 1500 > > I checked if the dovecot had accepted the change with the command > doveconf -n | grep process_limit > and it shows > process_limit = 1500 > > But it is dropping the connections when they reach 1024 anyway, the > configuration parameter is totally ignored. > > Can anyone help? Am I editing the right parameter? > > thanks in advance > > Fabio Ferrari > > P.S. The version of the dovecot is dovecot-2.0.1-1_118.el5 on Red Hat > Enterprise Linux Server release 5.8 (Tikanga). Don't forget to change the operating system limit as well. I added these lines to my /etc/sysconfig/dovecot on rhel6. I compiled dovecot myself, but I package similar to the system version. ulimit -n 4096 ulimit -u 5120 This increases the filehandles allowed by dovecot to 4096 and increases the number of processes per user to 5120. This is a proxy server, so I needed to support much larger numbers. than a silly 1024. Jack From davide.marchi at mail.cgilfe.it Mon Oct 1 18:46:53 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Mon, 01 Oct 2012 17:46:53 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_143220_258050_A12B6D1E) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_143220_258050_A12B6D1E) Message-ID: <5069BAED.8020206@mail.cgilfe.it> I made an experiment because iwasn't able to output some error to syslog or stderr about my configuration... I've installed antispam plugin in a parallel server with the program of johannes Bergs that seem now upgraded to function with 2.X and i have same problem ("Failed to call crm114 binary.." in antispam dovecot from your program, Eugene, and "Failed to call reaver.." from Johannes program) but now i'm able to output to log with this config: antispam_debug_target = syslog antispam_debug_target = stderr antispam_verbose_debug = 1 antispam_backend = crm114 antispam_spam_pattern_ignorecase = spam* antispam_trash_pattern_ignorecase = trash* antispam_unsure_pattern_ignorecase = unsure* antispam_signature = X-CRM114-CacheID antispam_signature_missing = error antispam_crm_binary = /usr/local/bin/piper_log.sh antispam_crm_spam_arg = --spam antispam_crm_notspam_arg = --good # antispam_crm_binary = /opt/crm114/mailreaver.crm antispam_crm_args = -u;%h;--fileprefix=/opt/crm114/ # antispam_signature = X-CRM114-CacheID in dovecot-err.log now i have 2012-10-01 17:39:35 imap: Error: antispam: plugin initialising (2.0-4-g83b0b4b-dirty) 2012-10-01 17:39:35 imap: Error: antispam: verbose debug enabled 2012-10-01 17:39:35 imap: Error: antispam: "SPAM" is exact match spam folder 2012-10-01 17:39:35 imap: Error: antispam: "UNSURE" is exact match unsure folder 2012-10-01 17:39:35 imap: Error: antispam: "Trash" is exact match trash folder 2012-10-01 17:39:35 imap: Error: antispam: reaver binary set to /usr/local/bin/piper_log.sh 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg -u 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg /home/vpopmail/domains/mail.cgilfe.it/davide.marchi 2012-10-01 17:39:35 imap: Error: antispam: reaver extra arg --fileprefix=/opt/crm114/ 2012-10-01 17:39:35 imap: Error: antispam: signature header line is "X-CRM114-CacheID" 2012-10-01 17:39:39 imap: Error: antispam: plugin initialising (2.0-4-g83b0b4b-dirty) 2012-10-01 17:39:39 imap: Error: antispam: verbose debug enabled . . . 2012-10-01 17:39:42 imap: Error: antispam: mail copy: from trash: 0, to trash: 0 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_spam(SPAM): 1 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_spam(INBOX): 0 2012-10-01 17:39:42 imap: Error: antispam: mailbox_is_unsure(SPAM): 0 2012-10-01 17:39:42 imap: Error: antispam: mail copy: src spam: 1, dst spam: 0, src unsure: 0 (i moved a mail from SPAM to INBOX) this is the output for "call command crm args" /opt/crm114/mailreaver.crm --good -u /home/vpopmail/domains/mail.cgilfe.it/davide.marchi --fileprefix=/opt/crm114/ if i exec this command with user vpopmail in console the command is successful Il 01/10/2012 14:04, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 14:48:31 +0300, Davide > wrote: > >> Can i attach compressed log in a post? > > If it's of the sane size :-) > But I'm actually interested in the output which was triggered by the > mail move itself. > >> I can produce output replacing crm binary with a script bash to ouput >> command,user etc etc > > Wait a second... So the script is ran correctly but reaver isn't? > That probably means that either reaver dislikes its arguments or its > stdin... > From fabiodepin at gmail.com Mon Oct 1 22:20:50 2012 From: fabiodepin at gmail.com (Fabio Depin) Date: Mon, 1 Oct 2012 16:20:50 -0300 Subject: [Dovecot] BUG to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2 Message-ID: Hello, Today I needed to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2. When running 'make' getting the following error: -------------------------------------------------- -------- db-checkpassword.c: In function 'sigchld_handler': db-checkpassword.c: 426: error: assignment of read-only member '__in' db-checkpassword.c: 429: error: assignment of read-only member '__in' db-checkpassword.c: 431: error: assignment of read-only member '__in' db-checkpassword.c: 432: error: assignment of read-only member '__in' make [3]: ** [db-checkpassword.o] Error 1 make [3]: ** Waiting for other processes to finish. mv-f .deps / auth-worker-server.Tpo .deps / auth-worker-server.Po make [3]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src/auth ' make [2]: ** [all-recursive] Error 1 make [2]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src ' make [1]: ** [all-recursive] Error 1 make [1]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7 ' make: ** [all] Error 2 -------------------------------------------------- -------- -To work did the following: -------------------------------------------------- -------- 417a418 + Int stat = status-> status; 426c427 - If (WIFSIGNALED (status-> status)) { --- + If (WIFSIGNALED (stat)) { 429c430 - Dec2str (status-> pid), WTERMSIG (status-> status)); --- + Dec2str (status-> pid), WTERMSIG (stat)); 431.432 c432, 433 -} Else if (WIFEXITED (status-> status)) { - Request-> exit_status WEXITSTATUS = (status-> status); --- +} Else if (WIFEXITED (stat)) { + Request-> exit_status WEXITSTATUS = (stat); -------------------------------------------------- -------- With this change worked perfectly ntanto in debian 4 with gcc 4.1.2, as in debian 6 with gcc 4.4.5. -I wonder if I made the change may affect any function of dovecot, or if it is correct. Thank you for your attention. Fabio Depin From joe at netmusician.org Mon Oct 1 22:34:25 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 01 Oct 2012 15:34:25 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <506453CE.7000608@gmail.com> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> <506453CE.7000608@gmail.com> Message-ID: <5069F041.6060904@netmusician.org> Are performance issues a possible symptom of this problem? It was mentioned that this happens after disconnects, but does this break IDLE? > Birta Levente > September 27, 2012 9:25 AM > > Hi Timo > > I just want to ask you: this issue is still in your task list? > If you need more debug information please tell me how can I give you. > > Thanks, > Levi > > Timo Sirainen > September 24, 2012 10:58 AM > > Show your doveconf -n output at least. As for debugging information, > that would depend on how you installed Dovecot? From some RPM or sources? > > Birta Levente > September 24, 2012 10:55 AM > > > I have the same problem, but on centos 6.3 64bit. How can I give you > the debug information? > > Levi > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: postbox-contact.jpg Type: image/jpeg Size: 1305 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Mon Oct 1 23:13:50 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 01 Oct 2012 16:13:50 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <5069F041.6060904@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> <506453CE.7000608@gmail.com> <5069F041.6060904@netmusician.org> Message-ID: <5069F97E.5020900@netmusician.org> Are performance issues a possible symptom of this problem? It was mentioned that this happens after disconnects, but does this break IDLE? > > Birta Levente > September 27, 2012 9:25 AM > > Hi Timo > > I just want to ask you: this issue is still in your task list? > If you need more debug information please tell me how can I give you. > > Thanks, > Levi > > > Timo Sirainen > September 24, 2012 10:58 AM > > Show your doveconf -n output at least. As for debugging information, > that would depend on how you installed Dovecot? From some RPM or sources? > > > Birta Levente > September 24, 2012 10:55 AM > > > I have the same problem, but on centos 6.3 64bit. How can I give you > the debug information? > > Levi > > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------ From dovecot-in at keystealth.org Mon Oct 1 23:36:25 2012 From: dovecot-in at keystealth.org (Scott Neville) Date: Mon, 1 Oct 2012 13:36:25 -0700 (PDT) Subject: [Dovecot] Logging IP address for failed login Message-ID: Hi, I am trying to use the logs to show the IP that brute force activity comes from, but Im not succeeding. I have read the archives and seen the advice others have had. I can see logs for repeated bad logins, but I need the IP address from the attempts. dovecot 2.0.12 / CentOS 5.4 / imaps only (993) I have tried a bunch of different combinations of 10-logging.conf settings. This is what I have currently (that does not work the way I want): auth_verbose = yes #auth_verbose_passwords = no #auth_debug = yes #auth_debug_passwords = no #mail_debug = no I *dont* want to see the passwords, either failed or successful. I just want to see failed logins for whatever reason and the IP they came from. In /var/log/maillog I get lines like this: Oct 1 04:19:12 olive dovecot: auth: pam(marketing): unknown user Oct 1 04:19:17 olive dovecot: auth: pam(marketing): unknown user When i had debugging turned on, I would get lines like this: Sep 9 01:14:59 olive dovecot: auth: Debug: passwd(dbelan,62.128.300.94): lookup but only for successful logins. The brute force attempts dont log like that: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): lookup service=dovecot Sep 16 00:02:57 olive dovecot: auth: Debug: pam(backup): #1/1 style=1 msg=Password: Sep 16 00:02:58 olive dovecot: auth: pam(backup): unknown user No IP anywhere in that. fail2ban seems to rely on the pop-login or imap-login lines to pull the IP from. I get an imap-login for my real logins: Oct 1 12:38:56 olive dovecot: imap-login: Login: user=, method=PLAIN, rip=62.128.300.94, lip=204.152.189.165, mpid=20360, TLS but no similar line for the failed logins. So is this a dovecot logging configuration combination I need to find? Is it getting lost in pam? Is it specific to CentOS? Any help appreciated - happy to read up on it myself, but would need a pointer, since the docs so far either assume I get an imap-login line for failed logins which I dont, or they assume I just want to see the repeated attempts/passwords. Scott. From fabiodepin at gmail.com Mon Oct 1 23:51:39 2012 From: fabiodepin at gmail.com (Fabio Depin) Date: Mon, 1 Oct 2012 17:51:39 -0300 Subject: [Dovecot] BUG to compile Plugin - deleted-to-trash on dovecot 2.1+ Message-ID: Hello, Today I needed to compile the plugin deleted-to-trash for dovecot 1.2.10 (> 2.1) and had problems compiling. To solve the problem efetuei changes below: -------------------------------------------------- -------------------- deleted-to-trash-plugin.c 79.80 C79 - Mailbox_alloc box = (list, name, MAILBOX_FLAG_KEEP_RECENT | - MAILBOX_FLAG_NO_INDEX_FILES); --- + Box = mailbox_alloc (list, name, MAILBOX_FLAG_NO_INDEX_FILES); 136c135 - If (keywords! = NULL) mailbox_keywords_unref (trash_box, & keywords); --- + If (keywords! = NULL) mailbox_keywords_unref (& keywords); -------------------------------------------------- -------------------- I would like to verify that this is correct, or can generate a problem. Thank you for your attention. Fabio Depin From jbates at brightok.net Tue Oct 2 00:15:54 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 16:15:54 -0500 Subject: [Dovecot] Logging IP address for failed login In-Reply-To: References: Message-ID: <506A080A.60906@brightok.net> On 10/1/2012 3:36 PM, Scott Neville wrote: > > In /var/log/maillog I get lines like this: > Oct 1 04:19:12 olive dovecot: auth: pam(marketing): unknown user > Oct 1 04:19:17 olive dovecot: auth: pam(marketing): unknown user > I'm guessing you are using a centos package. This may be package version specific. Here is RHEL6's dovecot 2.0.9 default except for setting auth_verbose = yes. Sep 28 21:12:10 compiler dovecot: auth: pam(test,::1): unknown user Sep 28 21:12:24 compiler dovecot: auth: pam(validuser,::1): pam_authenticate() failed: Authentication failure (password mismatch?) 2.1.9/2.1.10 which I packaged shows similar. Since I connected localhost, the IP is IPv6, of course. Jack From jbates at brightok.net Tue Oct 2 04:42:36 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 01 Oct 2012 20:42:36 -0500 Subject: [Dovecot] BUG to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2 In-Reply-To: References: Message-ID: <506A468C.10505@brightok.net> It looks like this might be a bug in glibc 2.3.3 http://sourceware.org/bugzilla/show_bug.cgi?id=1392 Jack On 10/1/2012 2:20 PM, Fabio Depin wrote: > Hello, > > Today I needed to compile dovecot 2.1.10 on Debian 4.0, using gcc 4.1.2. > When running 'make' getting the following error: > -------------------------------------------------- -------- > db-checkpassword.c: In function 'sigchld_handler': > db-checkpassword.c: 426: error: assignment of read-only member '__in' > db-checkpassword.c: 429: error: assignment of read-only member '__in' > db-checkpassword.c: 431: error: assignment of read-only member '__in' > db-checkpassword.c: 432: error: assignment of read-only member '__in' > make [3]: ** [db-checkpassword.o] Error 1 > make [3]: ** Waiting for other processes to finish. > mv-f .deps / auth-worker-server.Tpo .deps / auth-worker-server.Po > make [3]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src/auth ' > make [2]: ** [all-recursive] Error 1 > make [2]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7/src ' > make [1]: ** [all-recursive] Error 1 > make [1]: Leaving directory `/ usr/src/dovecot/dovecot-2.1.7 ' > make: ** [all] Error 2 > -------------------------------------------------- -------- > > -To work did the following: > -------------------------------------------------- -------- > 417a418 > + Int stat = status-> status; > 426c427 > - If (WIFSIGNALED (status-> status)) { > --- > + If (WIFSIGNALED (stat)) { > 429c430 > - Dec2str (status-> pid), WTERMSIG (status-> status)); > --- > + Dec2str (status-> pid), WTERMSIG (stat)); > 431.432 c432, 433 > -} Else if (WIFEXITED (status-> status)) { > - Request-> exit_status WEXITSTATUS = (status-> status); > --- > +} Else if (WIFEXITED (stat)) { > + Request-> exit_status WEXITSTATUS = (stat); > -------------------------------------------------- -------- > > With this change worked perfectly ntanto in debian 4 with gcc 4.1.2, as in > debian 6 with gcc 4.4.5. > -I wonder if I made the change may affect any function of dovecot, or if it is > correct. > > Thank you for your attention. > Fabio Depin > From amateo at um.es Tue Oct 2 11:41:51 2012 From: amateo at um.es (Angel L. Mateo) Date: Tue, 02 Oct 2012 10:41:51 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> Message-ID: <506AA8CF.8090605@um.es> Hello, I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: root at myotis30:/etc/dovecot/conf.d# doveadm user angel.luis at um.es userdb: angel.luis at um.es mail : mdbox:/home/alumnos/46/113246/mdbox:INDEX=/var/indexes/mdbox/angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=10G root at myotis30:/etc/dovecot/conf.d# doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=10G I guess I'm using different keys depending the user database used. I have configured three user databases, one for master-password, one for a ldap server and the other with pam (I need it because my webmail users authenticate in my SSO system through PAM). This is my config: passdb { driver = passwd-file master = yes args = /etc/dovecot/master-users # Unless you're using PAM, you probably still want the destination user to # be looked up from passdb that it really exists. pass=yes does that. pass = yes } passdb { driver = pam # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=] # [cache_key=] [] #args = dovecot args = session=yes cache_key=%n dovecot } passdb { driver = ldap # Path for LDAP configuration file, see example-config/dovecot-ldap.conf.ext args = /etc/dovecot/dovecot-ldap.conf.ext } # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # userdb { driver = prefetch } userdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf.ext # Default fields can be used to specify defaults that LDAP may override #default_fields = home=/home/virtual/%u } In my ldap configuration, I have a filter that looks for the uid of the user or the hole email: user_filter = (&()(|(uid=%u)(mail=%u))) I need this, because I have users that authenticate with just his/her login, not the complete email address. How can I unify those entries, so they use always just the login as key? El 18/09/12 18:31, Timo Sirainen escribi?: > On 18.9.2012, at 9.59, Angel L. Mateo wrote: > >>>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >>> >>> Works in my tests. >>> >> Is this cache the same than the user information cache? > > Yes. > >> The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: >> >> doveadm auth cache flush > > What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: > > doveadm auth cache flush foo # make sure it isn't there > doveadm user foo > doveadm auth cache flush foo > > Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. > > You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. > -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.4.0-030400-generic x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1000 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_exceeded_message = El mensaje no se ha entregado porque el destinatario del mismo tiene el buz?n lleno. quota_rule = *:storage=20G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 2003 } service auth { client_limit = 3000 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 2 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 2 } ssl = no ssl_cert = } From davide.marchi at mail.cgilfe.it Tue Oct 2 11:57:33 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Tue, 02 Oct 2012 10:57:33 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) In-Reply-To: (sfid-20121001_143220_258050_A12B6D1E) References: <50641AE7.6040201@mail.cgilfe.it> <50695B9E.3090502@mail.cgilfe.it> <50696839.7090901@mail.cgilfe.it> <5069770A.5030007@mail.cgilfe.it> <5069830F.4060401@mail.cgilfe.it> (sfid-20121001_143220_258050_A12B6D1E) Message-ID: <506AAC7D.8060408@mail.cgilfe.it> I'm unable to output nothing to syslog nor in other dovecot's log files about problem of plugin. If i use the test server the output is this: Oct 2 10:38:34 debnew imap: antispam: mailbox_is_unsure(SPAM): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_trash(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_trash(SPAM): 0 Oct 2 10:38:34 debnew imap: antispam: mail copy: from trash: 0, to trash: 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_spam(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_spam(SPAM): 1 Oct 2 10:38:34 debnew imap: antispam: mailbox_is_unsure(INBOX): 0 Oct 2 10:38:34 debnew imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Oct 2 10:38:34 debnew imap: antispam: /usr/local/bin/piper_log.sh --spam -u /home/vpopmail/domains/mail.cgilfe.it/davide.marchi --fileprefix=/opt/crm114/ and this is correct but anyway thunderbird say [CANNOT] Failed to call reaver Il 01/10/2012 14:04, Eugene Paskevich ha scritto: > On Mon, 01 Oct 2012 14:48:31 +0300, Davide > wrote: > >> Can i attach compressed log in a post? > > If it's of the sane size :-) > But I'm actually interested in the output which was triggered by the > mail move itself. > >> I can produce output replacing crm binary with a script bash to ouput >> command,user etc etc > > Wait a second... So the script is ran correctly but reaver isn't? > That probably means that either reaver dislikes its arguments or its > stdin... > From tss at iki.fi Tue Oct 2 21:37:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:37:09 +0300 Subject: [Dovecot] LTMP Proxy failure fix/hack In-Reply-To: <50660897.6040008@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> <506604BC.6050503@brightok.net> <50660897.6040008@brightok.net> Message-ID: On 28.9.2012, at 23.29, Jack Bates wrote: > On 9/28/2012 3:12 PM, Jack Bates wrote: >> >> Code needs to be written to handle the special case of us not having any proxy callbacks as they are all bad. >> > > > Timo, please check and approve. This was diff'd on 2.1.10 on my test server (2.1.9 and 2.1.10 at least had this callback issue). > > *** lmtp-proxy.c-orig 2012-09-28 20:17:36.138916678 +0000 > --- lmtp-proxy.c 2012-09-28 20:18:12.241940780 +0000 > *************** > *** 300,303 **** > --- 300,304 ---- > lmtp_client_send(conn->client, conn->data_input); > lmtp_client_send_more(conn->client); > } > + lmtp_proxy_try_finish(proxy); > } Looks ok. Added: http://hg.dovecot.org/dovecot-2.1/rev/38727d3e90ec From tss at iki.fi Tue Oct 2 21:41:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:41:12 +0300 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924184157.GA75341@kyoko.org> References: <20120924134810.GA62723@kyoko.org> <20120924150440.GA85969@kyoko.org> <20120924184157.GA75341@kyoko.org> Message-ID: On 24.9.2012, at 21.41, Philippe Chevalier wrote: > As for the ldap message, it errors if there's no domain in the login. > > In the doc, it says that %d is empty if there's no domain part. So I > guess it's an enhancement request : a configuration option to have it > filled out with a default domain if there's no one supplied by the > client. Maybe this is enough? auth_bind_userdn = dc=%Du,ou=Domains,ou=Mail,dc=dspnet,dc=fr See %D in http://wiki2.dovecot.org/Variables From listas at adminlinux.com.br Tue Oct 2 21:45:39 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Tue, 02 Oct 2012 15:45:39 -0300 Subject: [Dovecot] About ManageSieve and TLS Message-ID: <506B3653.5020804@adminlinux.com.br> Hi, I have a "ubuntu10.04 + dovecot-2.0.13" configuration in my server. It works fine with ~50k accounts. Recently I enabled TLS: $ cat /etc/dovecot/dovecot.conf ... # Use SSL ? ssl = yes ... The goal was to provide only IMAPS and POP3S. But Managesieve says "STARTTLS": $ telnet _MY_IP_ 2000 Trying _MY_IP_... Connected to _MY_IP_. Escape character is '^]'. "IMPLEMENTATION" "K8 ManageSieve" "SIEVE" "comparator-i;ascii-numeric copy envelope fileinto imapflags include notify regex reject relational subaddress vacation" "SASL" "PLAIN LOGIN" "STARTTLS" "VERSION" "1.0" OK "K8 IMAP/POP3 server" doveconf -a shows: service managesieve-login { ... inet_listener sieve { address = _MY_IP_ port = 4190 ssl = no } inet_listener sieve_deprecated { address = _MY_IP_ port = 2000 ssl = no } ... } I think there is something wrong there but I don't know. I think Managesieve should not say "STARTTLS". Can someone help me? Thanks. -- Thiago Henrique adminlinux.com.br From tss at iki.fi Tue Oct 2 21:56:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 21:56:33 +0300 Subject: [Dovecot] Spurious " Renaming not supported across conflicting directory permissions" In-Reply-To: <505DA946.90409@yahoo.com> References: <505DA946.90409@yahoo.com> Message-ID: On 22.9.2012, at 15.04, tlhackque wrote: > Dovecot 2.1.10 > Client = Thunderbird. Local disks. mbox format. > > Attempted to rename a folder, failed with: > > CANNOT Renaming not supported across conflicting directory permissions Fixed: http://hg.dovecot.org/dovecot-2.1/rev/83695d6d41aa From tss at iki.fi Tue Oct 2 22:28:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:28:22 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C393.5050209@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: On 19.9.2012, at 16.07, Alessio Cecchi wrote: > #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( > backend=0x1944240) at acl-backend-vfile-acllist.c:297 This backtrace is rather weird. Could you also do (instead of bt full): fr 1 p *ns p *ns.user p *auser It crashes because auser->dict = NULL, but it should never be NULL. From stephan at rename-it.nl Tue Oct 2 22:31:05 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 02 Oct 2012 21:31:05 +0200 Subject: [Dovecot] About ManageSieve and TLS In-Reply-To: <506B3653.5020804@adminlinux.com.br> References: <506B3653.5020804@adminlinux.com.br> Message-ID: <506B40F9.1000905@rename-it.nl> On 10/2/2012 8:45 PM, 3.listas at adminlinux.com.br wrote: > Hi, > > I have a "ubuntu10.04 + dovecot-2.0.13" configuration in my server. It > works fine with ~50k accounts. > > Recently I enabled TLS: > $ cat /etc/dovecot/dovecot.conf > ... > # Use SSL ? > ssl = yes > ... > > The goal was to provide only IMAPS and POP3S. But Managesieve says > "STARTTLS": > $ telnet _MY_IP_ 2000 > Trying _MY_IP_... > Connected to _MY_IP_. > Escape character is '^]'. > "IMPLEMENTATION" "K8 ManageSieve" > "SIEVE" "comparator-i;ascii-numeric copy envelope fileinto imapflags > include notify regex reject relational subaddress vacation" > "SASL" "PLAIN LOGIN" > "STARTTLS" > "VERSION" "1.0" > OK "K8 IMAP/POP3 server" > > I think there is something wrong there but I don't know. I think > Managesieve should not say "STARTTLS". > Can someone help me? The STARTTLS capability means that ManageSieve is prepared to accept a STARTTLS command that invokes the TLS handshake. Basically, the protocol starts in plaintext and switches to a TLS/SSL secured channel once the STARTTLS command is issued. However, the client can also choose not to use it. Therefore, it really shouldn't influence whether ManageSieve works properly (unless the client messes up TLS somehow). If you really want to, you can disable TLS for ManageSieve specifically by putting a ssl=no inside the protocol sieve {} section. Regards, Stephan. From tss at iki.fi Tue Oct 2 22:38:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:38:58 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <5062DF3C.3050601@um.es> References: <5062DF3C.3050601@um.es> Message-ID: On 26.9.2012, at 13.55, Angel L. Mateo wrote: > doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do > doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; > done -S auth-userdb? You've named it completely wrong if that works :) > The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: > > e62e0d3834ed094e5c7900007efb8a67 66 > e62e0d3834ed094e5c7900007efb8a67 71 > e62e0d3834ed094e5c7900007efb8a67 74 > e62e0d3834ed094e5c7900007efb8a67 > 75 e62e0d3834ed094e5c7900007efb8a67 > 77 e62e0d3834ed094e5c7900007efb8a67 > 78 e62e0d3834ed094e5c7900007efb8a67 Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/94c7e875f9b9 From tss at iki.fi Tue Oct 2 22:50:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 22:50:08 +0300 Subject: [Dovecot] Problem with process_limit In-Reply-To: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: On 1.10.2012, at 12.15, FABIO FERRARI wrote: > Occasionally, it happens that the dovecot.log shows this line: > master: Warning: service(imap): process_limit reached, client connections > are being dropped .. > Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the line > process_limit = 1500 But did you set it inside service imap {}? All of the services have process_limit parameter. From tss at iki.fi Tue Oct 2 23:12:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:12:51 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <5060AE92.5040904@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> Message-ID: On 24.9.2012, at 22.03, Joe Auty wrote: > #2 0x00007ff30074633d in master_service_settings_cache_deinit (_cache=) > at master-service-settings-cache.c:86 Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 From tss at iki.fi Tue Oct 2 23:15:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:15:22 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <83B37619-1CE8-4C5D-8147-A3C0E1C99CDC@iki.fi> On 2.10.2012, at 22.28, Timo Sirainen wrote: > On 19.9.2012, at 16.07, Alessio Cecchi wrote: > >> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > > This backtrace is rather weird. Could you also do (instead of bt full): Also, can you reproduce the crash always by running "doveadm quota recalc -u user at domain"? From tss at iki.fi Tue Oct 2 23:18:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 2 Oct 2012 23:18:48 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <506AA8CF.8090605@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> Message-ID: <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> On 2.10.2012, at 11.41, Angel L. Mateo wrote: > I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). You could flush the whole cache also. > I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: Flush both the user and user at domain entries? From florob at babelmonkeys.de Wed Oct 3 00:05:56 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 02 Oct 2012 23:05:56 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme Message-ID: <506B5734.30906@babelmonkeys.de> Hello, attached is an hg export on top of the current dovecot-2.2 branch, which adds support for a SCRAM-SHA-1 password scheme. Ideally I'd want doveadm pw's rounds flag to apply to this, but that's currently specific to the crypt password scheme, so I left it out for now. Regards, Florian Zeitz -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1348017219 -7200 # Node ID 21a0d1b4daa7bb924f1666f0bb7c7e697a19c950 # Parent 8802322d72573ee17c52ce5e972e77e6f8ad69d1 auth: Add and use SCRAM-SHA-1 password scheme diff --git a/src/auth/Makefile.am b/src/auth/Makefile.am --- a/src/auth/Makefile.am +++ b/src/auth/Makefile.am @@ -44,6 +44,7 @@ password-scheme.c \ password-scheme-crypt.c \ password-scheme-md5crypt.c \ + password-scheme-scram.c \ password-scheme-otp.c \ password-scheme-rpa.c diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -1,11 +1,13 @@ /* * SCRAM-SHA-1 SASL authentication, see RFC-5802 * - * Copyright (c) 2011 Florian Zeitz + * Copyright (c) 2011-2012 Florian Zeitz * * This software is released under the MIT license. */ +#include + #include "auth-common.h" #include "base64.h" #include "buffer.h" @@ -29,45 +31,22 @@ /* sent: */ const char *server_first_message; - unsigned char salt[16]; - unsigned char salted_password[SHA1_RESULTLEN]; + const char *snonce; /* received: */ const char *gs2_cbind_flag; const char *cnonce; - const char *snonce; const char *client_first_message_bare; const char *client_final_message_without_proof; buffer_t *proof; + + /* stored */ + buffer_t *stored_key; + buffer_t *server_key; }; -static void Hi(const unsigned char *str, size_t str_size, - const unsigned char *salt, size_t salt_size, unsigned int i, - unsigned char result[SHA1_RESULTLEN]) -{ - struct hmac_context ctx; - unsigned char U[SHA1_RESULTLEN]; - unsigned int j, k; - - /* Calculate U1 */ - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, salt, salt_size); - hmac_update(&ctx, "\0\0\0\1", 4); - hmac_final(&ctx, U); - - memcpy(result, U, SHA1_RESULTLEN); - - /* Calculate U2 to Ui and Hi */ - for (j = 2; j <= i; j++) { - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, U, sizeof(U)); - hmac_final(&ctx, U); - for (k = 0; k < SHA1_RESULTLEN; k++) - result[k] ^= U[k]; - } -} - -static const char *get_scram_server_first(struct scram_auth_request *request) +static const char *get_scram_server_first(struct scram_auth_request *request, + int iter, const char *salt) { unsigned char snonce[SCRAM_SERVER_NONCE_LEN+1]; string_t *str; @@ -84,12 +63,9 @@ snonce[sizeof(snonce)-1] = '\0'; request->snonce = p_strndup(request->pool, snonce, sizeof(snonce)); - random_fill(request->salt, sizeof(request->salt)); - - str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(request->salt))); - str_printfa(str, "r=%s%s,s=", request->cnonce, request->snonce); - base64_encode(request->salt, sizeof(request->salt), str); - str_printfa(str, ",i=%d", SCRAM_ITERATE_COUNT); + str = t_str_new(sizeof(snonce)); + str_printfa(str, "r=%s%s,s=%s,i=%d", request->cnonce, request->snonce, + salt, iter); return str_c(str); } @@ -105,15 +81,8 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Server Key", 10); - hmac_final(&ctx, server_key); - - safe_memset(request->salted_password, 0, - sizeof(request->salted_password)); - - hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_init(&ctx, request->server_key->data, request->server_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, server_signature); @@ -211,8 +180,7 @@ return TRUE; } -static bool verify_credentials(struct scram_auth_request *request, - const unsigned char *credentials, size_t size) +static bool verify_credentials(struct scram_auth_request *request) { struct hmac_context ctx; const char *auth_message; @@ -221,54 +189,76 @@ unsigned char stored_key[SHA1_RESULTLEN]; size_t i; - /* FIXME: credentials should be SASLprepped UTF8 data here */ - Hi(credentials, size, request->salt, sizeof(request->salt), - SCRAM_ITERATE_COUNT, request->salted_password); - - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Client Key", 10); - hmac_final(&ctx, client_key); - - sha1_get_digest(client_key, sizeof(client_key), stored_key); - auth_message = t_strconcat(request->client_first_message_bare, ",", request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_init(&ctx, request->stored_key->data, request->stored_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) - client_signature[i] ^= client_key[i]; + client_key[i] = + ((char*)request->proof->data)[i] ^ client_signature[i]; + + sha1_get_digest(client_key, sizeof(client_key), stored_key); safe_memset(client_key, 0, sizeof(client_key)); - safe_memset(stored_key, 0, sizeof(stored_key)); + safe_memset(client_signature, 0, sizeof(client_signature)); - return memcmp(client_signature, request->proof->data, - request->proof->used) == 0; + return memcmp(stored_key, request->stored_key->data, + request->stored_key->used) == 0; } static void credentials_callback(enum passdb_result result, const unsigned char *credentials, size_t size, struct auth_request *auth_request) { + const char *const *fields; + size_t len; + int iter; + const char *salt; struct scram_auth_request *request = (struct scram_auth_request *)auth_request; - const char *server_final_message; switch (result) { case PASSDB_RESULT_OK: - if (!verify_credentials(request, credentials, size)) { + fields = t_strsplit(t_strndup(credentials, size), ","); + + iter = atoi(fields[0]); + salt = fields[1]; + + len = strlen(fields[2]); + request->stored_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[2], len, NULL, + request->stored_key) < 0) { auth_request_log_info(auth_request, "scram-sha-1", - "password mismatch"); + "Invalid base64 encoding" + "of StoredKey in passdb"); auth_request_fail(auth_request); - } else { - server_final_message = get_scram_server_final(request); - auth_request_success(auth_request, server_final_message, - strlen(server_final_message)); + break; } + + len = strlen(fields[3]); + request->server_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[3], len, NULL, + request->server_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of ServerKey in passdb"); + auth_request_fail(auth_request); + break; + } + + request->server_first_message = p_strdup(request->pool, + get_scram_server_first(request, iter, salt)); + + auth_request_handler_reply_continue(auth_request, + request->server_first_message, + strlen(request->server_first_message)); break; case PASSDB_RESULT_INTERNAL_FAILURE: auth_request_internal_failure(auth_request); @@ -333,8 +323,6 @@ request->client_final_message_without_proof = p_strdup(request->pool, t_strarray_join(fields, ",")); - auth_request_lookup_credentials(&request->auth_request, "PLAIN", - credentials_callback); return TRUE; } @@ -345,22 +333,35 @@ struct scram_auth_request *request = (struct scram_auth_request *)auth_request; const char *error = NULL; + const char *server_final_message; + int len; if (!request->client_first_message_bare) { /* Received client-first-message */ if (parse_scram_client_first(request, data, data_size, &error)) { - request->server_first_message = p_strdup(request->pool, - get_scram_server_first(request)); - auth_request_handler_reply_continue(auth_request, - request->server_first_message, - strlen(request->server_first_message)); + auth_request_lookup_credentials(&request->auth_request, + "SCRAM-SHA1", + credentials_callback); return; } } else { /* Received client-final-message */ - if (parse_scram_client_final(request, data, data_size, &error)) - return; + if (parse_scram_client_final(request, data, data_size, + &error)) { + if (!verify_credentials(request)) { + auth_request_log_info(auth_request, + "scram-sha-1", + "password mismatch"); + } else { + server_final_message = + get_scram_server_final(request); + len = strlen(server_final_message); + auth_request_success(auth_request, + server_final_message, len); + return; + } + } } if (error != NULL) diff --git a/src/auth/password-scheme-scram.c b/src/auth/password-scheme-scram.c new file mode 100644 --- /dev/null +++ b/src/auth/password-scheme-scram.c @@ -0,0 +1,139 @@ +/* + * SCRAM-SHA-1 SASL authentication, see RFC-5802 + * + * Copyright (c) 2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include + +#include "lib.h" +#include "safe-memset.h" +#include "base64.h" +#include "buffer.h" +#include "hmac.h" +#include "randgen.h" +#include "sha1.h" +#include "str.h" +#include "password-scheme.h" + +/* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ +#define SCRAM_ITERATE_COUNT 4096 + +static void Hi(const unsigned char *str, size_t str_size, + const unsigned char *salt, size_t salt_size, unsigned int i, + unsigned char result[SHA1_RESULTLEN]) +{ + struct hmac_context ctx; + unsigned char U[SHA1_RESULTLEN]; + unsigned int j, k; + + /* Calculate U1 */ + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); + + memcpy(result, U, SHA1_RESULTLEN); + + /* Calculate U2 to Ui and Hi */ + for (j = 2; j <= i; j++) { + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); + for (k = 0; k < SHA1_RESULTLEN; k++) + result[k] ^= U[k]; + } +} + +/* password string format: iter,salt,stored_key,server_key */ + +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED) +{ + struct hmac_context ctx; + string_t *str; + const char *const *fields; + int iter; + const unsigned char *salt; + size_t salt_len; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + fields = t_strsplit(t_strndup(raw_password, size), ","); + iter = atoi(fields[0]); + salt = buffer_get_data(t_base64_decode_str(fields[1]), &salt_len); + str = t_str_new(strlen(fields[2])); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, salt_len, + iter, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + base64_encode(stored_key, sizeof(stored_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + return strcmp(fields[2], str_c(str)) == 0 ? 1 : 0; +} + +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r) +{ + string_t *str; + struct hmac_context ctx; + unsigned char salt[16]; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char server_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + random_fill(salt, sizeof(salt)); + + str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(salt))); + str_printfa(str, "%i,", SCRAM_ITERATE_COUNT); + base64_encode(salt, sizeof(salt), str); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, + sizeof(salt), SCRAM_ITERATE_COUNT, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + str_append_c(str, ','); + base64_encode(stored_key, sizeof(stored_key), str); + + /* Calculate ServerKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); + str_append_c(str, ','); + base64_encode(server_key, sizeof(server_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(server_key, 0, sizeof(server_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + *raw_password_r = (const unsigned char *)str_c(str); + *size_r = str_len(str); +} diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -822,6 +822,8 @@ { "PLAIN-TRUNC", PW_ENCODING_NONE, 0, plain_trunc_verify, plain_generate }, { "CRAM-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, + { "SCRAM-SHA1", PW_ENCODING_NONE, 0, scram_sha1_verify, + scram_sha1_generate}, { "HMAC-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, { "DIGEST-MD5", PW_ENCODING_HEX, MD5_RESULTLEN, diff --git a/src/auth/password-scheme.h b/src/auth/password-scheme.h --- a/src/auth/password-scheme.h +++ b/src/auth/password-scheme.h @@ -85,6 +85,12 @@ const unsigned char *raw_password, size_t size, const char **error_r); +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED); +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r); + /* check wich of the algorithms Blowfisch, SHA-256 and SHA-512 are supported by the used libc's/glibc's crypt() */ void password_scheme_register_crypt(void); From joe at netmusician.org Wed Oct 3 00:09:19 2012 From: joe at netmusician.org (Joe Auty) Date: Tue, 02 Oct 2012 17:09:19 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> Message-ID: <506B57FF.40809@netmusician.org> > > Timo Sirainen > October 2, 2012 4:12 PM > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 Awesome! Will this fix make it into 2.1.11, or should I toy with incorporating your change and compiling a new copy of 2.1.10 by hand? Also, will the seg fault have caused performance issues in breaking IDLE connections? Just wondering what sort of impact this fix might have... Thanks for working on this, it is most appreciated, I'm a big fan of Dovecot! > > > Joe Auty > September 24, 2012 3:03 PM >> >> Timo Sirainen >> September 24, 2012 10:32 AM >> >> Well, the good news is that it crashes only after it has already >> disconnected the client anyway. But I thought I fixed this bug in >> v2.1.10 and I'm not able to reproduce it myself.. Having debugging >> information available might show something useful. Try installing >> dovecot-dbg package and getting the bt full again? > > Thanks Timo, I have done so. Here is the results of my debugging info > now: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...Reading symbols > from /usr/lib/debug/usr/lib/dovecot/imap-login...done. > (no debugging symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...Reading > symbols from > /usr/lib/debug/usr/lib/dovecot/libdovecot-login.so.0.0.0...done. > (no debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...Reading > symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot.so.0.0.0...done. > (no debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 hash_table_destroy (_table=0x28) at hash.c:106 > 106 hash.c: No such file or directory. > in hash.c > (gdb) bt full > #0 hash_table_destroy (_table=0x28) at hash.c:106 > table = > #1 0x00007ff300721054 in settings_parser_deinit (_ctx= optimized out>) at settings-parser.c:237 > ctx = 0x0 > #2 0x00007ff30074633d in master_service_settings_cache_deinit > (_cache=) > at master-service-settings-cache.c:86 > cache = 0x9f9a60 > entry = 0xa016e0 > next = 0x0 > __FUNCTION__ = "master_service_settings_cache_deinit" > #3 0x00007ff3009a5018 in main_deinit (binary=, > argc=2, argv=0x9f8370) at main.c:355 > No locals. > #4 login_binary_run (binary=, argc=2, > argv=0x9f8370) at main.c:407 > set_pool = 0x9f8a30 > allow_core_dumps = > login_socket = > c = > #5 0x00007ff3003c0c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #6 0x0000000000402459 in _start () > No symbol table info available. > (gdb) > > >> >> >> Joe Auty >> September 23, 2012 7:05 AM >>> >>> Timo Sirainen >>> September 23, 2012 5:58 AM >>> >>> >>> You should have a similar log line about the crash in mail.log (or >>> wherever "doveadm log find" says that errors get logged). Find those >>> lines, then configure login processes to dump core files. This >>> probably should work: >>> >>> service imap-login { >>> executable = imap-login -D >>> } >>> >>> Next time it crashes hopefully you'll have >>> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >>> send it: >>> >>> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >>> bt full >> >> I hope I'm doing this correctly! >> >> # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> GNU gdb (GDB) 7.0.1-debian >> Copyright (C) 2009 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later >> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show >> copying" >> and "show warranty" for details. >> This GDB was configured as "x86_64-linux-gnu". >> For bug reporting instructions, please see: >> ... >> Reading symbols from /usr/lib/dovecot/imap-login...(no debugging >> symbols found)...done. >> >> warning: Can't read pathname for load map: Input/output error. >> Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no >> debugging symbols found)...done. >> Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 >> Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging >> symbols found)...done. >> Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 >> Reading symbols from /lib/libc.so.6...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libc.so.6 >> Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols >> found)...done. >> Loaded symbols for /usr/lib/libssl.so.0.9.8 >> Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging >> symbols found)...done. >> Loaded symbols for /usr/lib/libcrypto.so.0.9.8 >> Reading symbols from /lib/librt.so.1...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/librt.so.1 >> Reading symbols from /lib/libdl.so.2...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libdl.so.2 >> Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging >> symbols found)...done. >> Loaded symbols for /lib64/ld-linux-x86-64.so.2 >> Reading symbols from /usr/lib/libz.so.1...(no debugging symbols >> found)...done. >> Loaded symbols for /usr/lib/libz.so.1 >> Reading symbols from /lib/libpthread.so.0...(no debugging symbols >> found)...done. >> Loaded symbols for /lib/libpthread.so.0 >> Core was generated by `dovecot/imap-login ?'. >> Program terminated with signal 11, Segmentation fault. >> #0 0x00007f789cd08e14 in hash_table_destroy () from >> /usr/lib/dovecot/libdovecot.so.0 >> (gdb) bt full >> #0 0x00007f789cd08e14 in hash_table_destroy () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00007f789ccda054 in settings_parser_deinit () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () >> from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #3 0x00007f789cf5e018 in login_binary_run () from >> /usr/lib/dovecot/libdovecot-login.so.0 >> No symbol table info available. >> #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 >> No symbol table info available. >> #5 0x0000000000402459 in ?? () >> No symbol table info available. >> #6 0x00007fff8a9c65f8 in ?? () >> No symbol table info available. >> #7 0x000000000000001c in ?? () >> No symbol table info available. >> #8 0x0000000000000002 in ?? () >> No symbol table info available. >> #9 0x00007fff8a9c7e6a in ?? () >> No symbol table info available. >> #10 0x00007fff8a9c7e7d in ?? () >> No symbol table info available. >> #11 0x0000000000000000 in ?? () >> No symbol table info available. >> >> >>> >>> >>> Joe Auty >>> September 22, 2012 3:57 PM >>> Hello, >>> >>> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >>> I suspect this might be causing performance issues. Any suggestions >>> what I can try to fix this? >>> >>> >>> I'm using the 2.1.10 packages obtained with the following in my >>> sources.list: >>> >>> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >>> >>> >>> I need to use Dovecot 2.1.x because I need to support handling >>> multiple SSL certs. >>> ------------------------------------------------------------------------ >>> >> >> >> >> >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full > > ------------------------------------------------------------------------ From tss at iki.fi Wed Oct 3 00:13:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:13:10 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <506B57FF.40809@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <5060AE92.5040904@netmusician.org> <506B57FF.40809@netmusician.org> Message-ID: <9AC6F07E-8751-47C0-AF6C-82722D1BAE5D@iki.fi> On 3.10.2012, at 0.09, Joe Auty wrote: >> Timo Sirainen >> October 2, 2012 4:12 PM >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e29b627219b3 > > Awesome! > > Will this fix make it into 2.1.11, or should I toy with incorporating your change and compiling a new copy of 2.1.10 by hand? All changes added to dovecot-2.1 hg go to the next 2.1 release. But I don't know when v2.1.11 will be released, probably a few weeks at least. > Also, will the seg fault have caused performance issues in breaking IDLE connections? Just wondering what sort of impact this fix might have... It shouldn't have caused any user-visible problems. From tss at iki.fi Wed Oct 3 00:27:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:27:13 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B5734.30906@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> Message-ID: On 3.10.2012, at 0.05, Florian Zeitz wrote: > attached is an hg export on top of the current dovecot-2.2 branch, which > adds support for a SCRAM-SHA-1 password scheme. > > Ideally I'd want doveadm pw's rounds flag to apply to this, but that's > currently specific to the crypt password scheme, so I left it out for now. Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. From cor at xs4all.nl Wed Oct 3 00:39:39 2012 From: cor at xs4all.nl (Cor Bosman) Date: Tue, 2 Oct 2012 23:39:39 +0200 Subject: [Dovecot] possible nfs issue Message-ID: Hi all, we've started receiving complaints from users that seemingly use more quota than they actually have. We noticed that these users have (in some cases many) .nfs files in their mailspool. Some of our admins checked their own dirs, and noticed them there as well. This could of course be unrelated to dovecot (kernel issue, netapp issue) but maybe somehow has an idea about if dovecot could cause this. This has been going on for at least a year, not really enough to notice before now. root at userimap1# find . -type f|grep -i .nfs ./cur/.nfs00000000003967ad003c0603 ./cur/.nfs000000000757b44b003be609 ./cur/.nfs00000000035e89bd003be60b ./cur/.nfs000000000796251c003be60c ./cur/.nfs000000000796251f003be60e ./cur/.nfs000000000262f9a1003be33a ./cur/.nfs00000000096513f3003be524 ./cur/.nfs0000000007962525003be60f ./cur/.nfs0000000003e7d8ab003be62b ./cur/.nfs00000000026f4fad003be50d ./cur/.nfs0000000000bdaeab003c0611 ./cur/.nfs0000000005da42c7003be525 ./cur/.nfs0000000003d74729003be526 ./cur/.nfs000000000229769e003be535 ./cur/.nfs000000000440969e003be516 With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. Anyone else with NFS mailspools seeing this? Cor From tss at iki.fi Wed Oct 3 00:45:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 00:45:35 +0300 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: On 3.10.2012, at 0.39, Cor Bosman wrote: > With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) From florob at babelmonkeys.de Wed Oct 3 01:12:29 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 00:12:29 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: References: <506B5734.30906@babelmonkeys.de> Message-ID: <506B66CD.6010302@babelmonkeys.de> Am 02.10.2012 23:27, schrieb Timo Sirainen: > On 3.10.2012, at 0.05, Florian Zeitz wrote: > >> attached is an hg export on top of the current dovecot-2.2 branch, which >> adds support for a SCRAM-SHA-1 password scheme. >> >> Ideally I'd want doveadm pw's rounds flag to apply to this, but that's >> currently specific to the crypt password scheme, so I left it out for now. > > Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. > I shall. For the iteration count the endless loop should be fixed by restricting the largest value to UINT_MAX-1, right? I'm not too fond of stopping people from wasting their CPU time on Hi calculation beyond this. I can try to guestimate a "sane" upper limit, but given time I have an icky feeling that it will end up being too low. Thoughts? From tss at iki.fi Wed Oct 3 01:31:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 01:31:51 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B66CD.6010302@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <506B66CD.6010302@babelmonkeys.de> Message-ID: On 3.10.2012, at 1.12, Florian Zeitz wrote: > Am 02.10.2012 23:27, schrieb Timo Sirainen: >> On 3.10.2012, at 0.05, Florian Zeitz wrote: >> >>> attached is an hg export on top of the current dovecot-2.2 branch, which >>> adds support for a SCRAM-SHA-1 password scheme. >>> >>> Ideally I'd want doveadm pw's rounds flag to apply to this, but that's >>> currently specific to the crypt password scheme, so I left it out for now. >> >> Looks pretty good. But you could improve the error handling a bit. Instead of atoi() use str_to_uint() and verify the error value. Also verify that t_strsplit() returns the correct number of values. And there should be some sanity check for the iter count also.. I'm not sure what, but currently it's possible for Hi() to go to infinite loop. >> > I shall. For the iteration count the endless loop should be fixed by > restricting the largest value to UINT_MAX-1, right? Yeah. > I'm not too fond of > stopping people from wasting their CPU time on Hi calculation beyond > this. I can try to guestimate a "sane" upper limit, but given time I > have an icky feeling that it will end up being too low. Thoughts? Looks like RFC 5802 doesn't give any kind of a limit. But since it gets sent to various client implementations, INT_MAX is probably a good limit? Also 0 isn't a valid iteration count. From tss at iki.fi Wed Oct 3 01:35:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 01:35:20 +0300 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> On 3.10.2012, at 0.45, Timo Sirainen wrote: > On 3.10.2012, at 0.39, Cor Bosman wrote: > >> With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. > > How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) You could see if this old test program leaves .nfs files lying around: http://dovecot.org/tmp/readdir.c Just comment out the line: close(fd); From jtam.home at gmail.com Wed Oct 3 01:42:23 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 2 Oct 2012 15:42:23 -0700 (PDT) Subject: [Dovecot] Logging IP address for failed login In-Reply-To: References: Message-ID: Scott Neville writes: > I am trying to use the logs to show the IP that brute force activity > comes from, but Im not succeeding. I have read the archives and seen > the advice others have had. I can see logs for repeated bad logins, > but I need the IP address from the attempts. > > ... > but only for successful logins. The brute force attempts dont log like that: > > Sep 16 00:02:58 olive dovecot: auth: pam(backup): unknown user This was similar to another complaint several months ago. I conjectured that these attempts are SMTP AUTH, not IMAP, brute forcing. Are you using the dovecot's SASL feature to authenticate outgoing Email (i.e. via Postfix?). Maybe you verify this hypothesis by checking the Postfix logs. Joseph Tam From cor at xs4all.nl Wed Oct 3 01:53:15 2012 From: cor at xs4all.nl (Cor Bosman) Date: Wed, 3 Oct 2012 00:53:15 +0200 Subject: [Dovecot] possible nfs issue In-Reply-To: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> References: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> Message-ID: On Oct 3, 2012, at 12:35 AM, Timo Sirainen wrote: > On 3.10.2012, at 0.45, Timo Sirainen wrote: > >> On 3.10.2012, at 0.39, Cor Bosman wrote: >> >>> With NFS these files are created when a file gets unlinked, but another process still has it open. It disappears as soon as the other process closes it. For some reason they dont disappear. As far as I can tell we've had no server crashes that could explain this. One possible theory is that a rename happens after an unlink. In that case the file remains. This could possibly be a dovecot issue. >> >> How can a rename happen after unlink? The rename should fail. (Unless doing rename(.nfs1234, something), but Dovecot definitely isn't doing that.) > > You could see if this old test program leaves .nfs files lying around: > > http://dovecot.org/tmp/readdir.c > > Just comment out the line: > > close(fd); > I meant the .nfs1234 indeed, but it seemed very unlikely. Thanks for clarifying. The readdir program leaves no .nfs files. We'll have to explore other possibilities. Cor From tss at iki.fi Wed Oct 3 02:42:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 02:42:17 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B5734.30906@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> Message-ID: <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> On 3.10.2012, at 0.05, Florian Zeitz wrote: > attached is an hg export on top of the current dovecot-2.2 branch, which > adds support for a SCRAM-SHA-1 password scheme. Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. From florob at babelmonkeys.de Wed Oct 3 02:54:21 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 01:54:21 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> Message-ID: <506B7EAD.1080108@babelmonkeys.de> Am 03.10.2012 01:42, schrieb Timo Sirainen: > On 3.10.2012, at 0.05, Florian Zeitz wrote: > >> attached is an hg export on top of the current dovecot-2.2 branch, which >> adds support for a SCRAM-SHA-1 password scheme. > > Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. > Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the RFC, and SHA-1 is the hash name registered with IANA [1]. I did call the password scheme SCRAM-SHA1 to be consistent with other current password schemes. I'm not 100% sure which one to use, or whether a mix might even be the way to go ("correct" messages, but minimum user confusion for password schemes). [1] https://www.iana.org/assignments/hash-function-text-names/hash-function-text-names.xml From tss at iki.fi Wed Oct 3 02:58:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 02:58:25 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B7EAD.1080108@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> Message-ID: On 3.10.2012, at 2.54, Florian Zeitz wrote: > Am 03.10.2012 01:42, schrieb Timo Sirainen: >> On 3.10.2012, at 0.05, Florian Zeitz wrote: >> >>> attached is an hg export on top of the current dovecot-2.2 branch, which >>> adds support for a SCRAM-SHA-1 password scheme. >> >> Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. >> > Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the > RFC, and SHA-1 is the hash name registered with IANA [1]. > I did call the password scheme SCRAM-SHA1 to be consistent with other > current password schemes. I'm not 100% sure which one to use, or whether > a mix might even be the way to go ("correct" messages, but minimum user > confusion for password schemes). Hmm. Probably not worth it to have both SCRAM-SHA1 and SCRAM-SHA-1. And now I see that the user-visible strings are about SCRAM-SHA-1 mechanism, not the hash. So yeah, I guess the best way to avoid confusion is to call it SCRAM-SHA-1 everywhere. From florob at babelmonkeys.de Wed Oct 3 03:10:41 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 02:10:41 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> Message-ID: <506B8281.9060703@babelmonkeys.de> Am 03.10.2012 01:58, schrieb Timo Sirainen: > On 3.10.2012, at 2.54, Florian Zeitz wrote: > >> Am 03.10.2012 01:42, schrieb Timo Sirainen: >>> On 3.10.2012, at 0.05, Florian Zeitz wrote: >>> >>>> attached is an hg export on top of the current dovecot-2.2 branch, which >>>> adds support for a SCRAM-SHA-1 password scheme. >>> >>> Oh, and SCRAM-SHA1 or SCRAM-SHA-1? I'd think SCRAM-SHA1 as the scheme is now called, but elsewhere in the code (including user-visible strings) it says SCRAM-SHA-1. >>> >> Well, I usually prefer SCRAM-SHA-1, as that is how it is called in the >> RFC, and SHA-1 is the hash name registered with IANA [1]. >> I did call the password scheme SCRAM-SHA1 to be consistent with other >> current password schemes. I'm not 100% sure which one to use, or whether >> a mix might even be the way to go ("correct" messages, but minimum user >> confusion for password schemes). > > Hmm. Probably not worth it to have both SCRAM-SHA1 and SCRAM-SHA-1. And now I see that the user-visible strings are about SCRAM-SHA-1 mechanism, not the hash. So yeah, I guess the best way to avoid confusion is to call it SCRAM-SHA-1 everywhere. > Seems sensible. Attached is a new export incorporating your feedback. The iteration count is now limited to [4096, INT_MAX]. The lower bound is a recommendation of the RFC. -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1348017219 -7200 # Node ID a0b0eece12335905500631477ec1d6ab31014469 # Parent 99843f74422ac68bfde86e9cee6920164eae4d5d auth: Add and use SCRAM-SHA-1 password scheme diff --git a/src/auth/Makefile.am b/src/auth/Makefile.am --- a/src/auth/Makefile.am +++ b/src/auth/Makefile.am @@ -44,6 +44,7 @@ password-scheme.c \ password-scheme-crypt.c \ password-scheme-md5crypt.c \ + password-scheme-scram.c \ password-scheme-otp.c \ password-scheme-rpa.c diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -1,11 +1,14 @@ /* * SCRAM-SHA-1 SASL authentication, see RFC-5802 * - * Copyright (c) 2011 Florian Zeitz + * Copyright (c) 2011-2012 Florian Zeitz * * This software is released under the MIT license. */ +#include +#include + #include "auth-common.h" #include "base64.h" #include "buffer.h" @@ -15,6 +18,7 @@ #include "safe-memset.h" #include "str.h" #include "strfuncs.h" +#include "strnum.h" #include "mech.h" /* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ @@ -29,45 +33,22 @@ /* sent: */ const char *server_first_message; - unsigned char salt[16]; - unsigned char salted_password[SHA1_RESULTLEN]; + const char *snonce; /* received: */ const char *gs2_cbind_flag; const char *cnonce; - const char *snonce; const char *client_first_message_bare; const char *client_final_message_without_proof; buffer_t *proof; + + /* stored */ + buffer_t *stored_key; + buffer_t *server_key; }; -static void Hi(const unsigned char *str, size_t str_size, - const unsigned char *salt, size_t salt_size, unsigned int i, - unsigned char result[SHA1_RESULTLEN]) -{ - struct hmac_context ctx; - unsigned char U[SHA1_RESULTLEN]; - unsigned int j, k; - - /* Calculate U1 */ - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, salt, salt_size); - hmac_update(&ctx, "\0\0\0\1", 4); - hmac_final(&ctx, U); - - memcpy(result, U, SHA1_RESULTLEN); - - /* Calculate U2 to Ui and Hi */ - for (j = 2; j <= i; j++) { - hmac_init(&ctx, str, str_size, &hash_method_sha1); - hmac_update(&ctx, U, sizeof(U)); - hmac_final(&ctx, U); - for (k = 0; k < SHA1_RESULTLEN; k++) - result[k] ^= U[k]; - } -} - -static const char *get_scram_server_first(struct scram_auth_request *request) +static const char *get_scram_server_first(struct scram_auth_request *request, + int iter, const char *salt) { unsigned char snonce[SCRAM_SERVER_NONCE_LEN+1]; string_t *str; @@ -84,12 +65,9 @@ snonce[sizeof(snonce)-1] = '\0'; request->snonce = p_strndup(request->pool, snonce, sizeof(snonce)); - random_fill(request->salt, sizeof(request->salt)); - - str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(request->salt))); - str_printfa(str, "r=%s%s,s=", request->cnonce, request->snonce); - base64_encode(request->salt, sizeof(request->salt), str); - str_printfa(str, ",i=%d", SCRAM_ITERATE_COUNT); + str = t_str_new(sizeof(snonce)); + str_printfa(str, "r=%s%s,s=%s,i=%d", request->cnonce, request->snonce, + salt, iter); return str_c(str); } @@ -105,15 +83,8 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Server Key", 10); - hmac_final(&ctx, server_key); - - safe_memset(request->salted_password, 0, - sizeof(request->salted_password)); - - hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_init(&ctx, request->server_key->data, request->server_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, server_signature); @@ -211,8 +182,7 @@ return TRUE; } -static bool verify_credentials(struct scram_auth_request *request, - const unsigned char *credentials, size_t size) +static bool verify_credentials(struct scram_auth_request *request) { struct hmac_context ctx; const char *auth_message; @@ -221,54 +191,90 @@ unsigned char stored_key[SHA1_RESULTLEN]; size_t i; - /* FIXME: credentials should be SASLprepped UTF8 data here */ - Hi(credentials, size, request->salt, sizeof(request->salt), - SCRAM_ITERATE_COUNT, request->salted_password); - - hmac_init(&ctx, request->salted_password, - sizeof(request->salted_password), &hash_method_sha1); - hmac_update(&ctx, "Client Key", 10); - hmac_final(&ctx, client_key); - - sha1_get_digest(client_key, sizeof(client_key), stored_key); - auth_message = t_strconcat(request->client_first_message_bare, ",", request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_init(&ctx, request->stored_key->data, request->stored_key->used, + &hash_method_sha1); hmac_update(&ctx, auth_message, strlen(auth_message)); hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) - client_signature[i] ^= client_key[i]; + client_key[i] = + ((char*)request->proof->data)[i] ^ client_signature[i]; + + sha1_get_digest(client_key, sizeof(client_key), stored_key); safe_memset(client_key, 0, sizeof(client_key)); - safe_memset(stored_key, 0, sizeof(stored_key)); + safe_memset(client_signature, 0, sizeof(client_signature)); - return memcmp(client_signature, request->proof->data, - request->proof->used) == 0; + return memcmp(stored_key, request->stored_key->data, + request->stored_key->used) == 0; } static void credentials_callback(enum passdb_result result, const unsigned char *credentials, size_t size, struct auth_request *auth_request) { + const char *const *fields; + size_t len; + unsigned int iter; + const char *salt; struct scram_auth_request *request = (struct scram_auth_request *)auth_request; - const char *server_final_message; switch (result) { case PASSDB_RESULT_OK: - if (!verify_credentials(request, credentials, size)) { + fields = t_strsplit(t_strndup(credentials, size), ","); + + if (str_array_length(fields) != 4) { auth_request_log_info(auth_request, "scram-sha-1", - "password mismatch"); + "Invalid passdb entry"); auth_request_fail(auth_request); - } else { - server_final_message = get_scram_server_final(request); - auth_request_success(auth_request, server_final_message, - strlen(server_final_message)); + break; } + + if (str_to_uint(fields[0], &iter) || (iter < 4096) || + (iter > INT_MAX)) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid iteration count"); + auth_request_fail(auth_request); + break; + } + + salt = fields[1]; + + len = strlen(fields[2]); + request->stored_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[2], len, NULL, + request->stored_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of StoredKey in passdb"); + auth_request_fail(auth_request); + break; + } + + len = strlen(fields[3]); + request->server_key = buffer_create_dynamic(request->pool, + MAX_BASE64_DECODED_SIZE(len)); + if (base64_decode(fields[3], len, NULL, + request->server_key) < 0) { + auth_request_log_info(auth_request, "scram-sha-1", + "Invalid base64 encoding" + "of ServerKey in passdb"); + auth_request_fail(auth_request); + break; + } + + request->server_first_message = p_strdup(request->pool, + get_scram_server_first(request, iter, salt)); + + auth_request_handler_reply_continue(auth_request, + request->server_first_message, + strlen(request->server_first_message)); break; case PASSDB_RESULT_INTERNAL_FAILURE: auth_request_internal_failure(auth_request); @@ -333,8 +339,6 @@ request->client_final_message_without_proof = p_strdup(request->pool, t_strarray_join(fields, ",")); - auth_request_lookup_credentials(&request->auth_request, "PLAIN", - credentials_callback); return TRUE; } @@ -345,22 +349,35 @@ struct scram_auth_request *request = (struct scram_auth_request *)auth_request; const char *error = NULL; + const char *server_final_message; + int len; if (!request->client_first_message_bare) { /* Received client-first-message */ if (parse_scram_client_first(request, data, data_size, &error)) { - request->server_first_message = p_strdup(request->pool, - get_scram_server_first(request)); - auth_request_handler_reply_continue(auth_request, - request->server_first_message, - strlen(request->server_first_message)); + auth_request_lookup_credentials(&request->auth_request, + "SCRAM-SHA-1", + credentials_callback); return; } } else { /* Received client-final-message */ - if (parse_scram_client_final(request, data, data_size, &error)) - return; + if (parse_scram_client_final(request, data, data_size, + &error)) { + if (!verify_credentials(request)) { + auth_request_log_info(auth_request, + "scram-sha-1", + "password mismatch"); + } else { + server_final_message = + get_scram_server_final(request); + len = strlen(server_final_message); + auth_request_success(auth_request, + server_final_message, len); + return; + } + } } if (error != NULL) diff --git a/src/auth/password-scheme-scram.c b/src/auth/password-scheme-scram.c new file mode 100644 --- /dev/null +++ b/src/auth/password-scheme-scram.c @@ -0,0 +1,139 @@ +/* + * SCRAM-SHA-1 SASL authentication, see RFC-5802 + * + * Copyright (c) 2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include + +#include "lib.h" +#include "safe-memset.h" +#include "base64.h" +#include "buffer.h" +#include "hmac.h" +#include "randgen.h" +#include "sha1.h" +#include "str.h" +#include "password-scheme.h" + +/* SCRAM hash iteration count. RFC says it SHOULD be at least 4096 */ +#define SCRAM_ITERATE_COUNT 4096 + +static void Hi(const unsigned char *str, size_t str_size, + const unsigned char *salt, size_t salt_size, unsigned int i, + unsigned char result[SHA1_RESULTLEN]) +{ + struct hmac_context ctx; + unsigned char U[SHA1_RESULTLEN]; + unsigned int j, k; + + /* Calculate U1 */ + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); + + memcpy(result, U, SHA1_RESULTLEN); + + /* Calculate U2 to Ui and Hi */ + for (j = 2; j <= i; j++) { + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); + for (k = 0; k < SHA1_RESULTLEN; k++) + result[k] ^= U[k]; + } +} + +/* password string format: iter,salt,stored_key,server_key */ + +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED) +{ + struct hmac_context ctx; + string_t *str; + const char *const *fields; + int iter; + const unsigned char *salt; + size_t salt_len; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + fields = t_strsplit(t_strndup(raw_password, size), ","); + iter = atoi(fields[0]); + salt = buffer_get_data(t_base64_decode_str(fields[1]), &salt_len); + str = t_str_new(strlen(fields[2])); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, salt_len, + iter, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + base64_encode(stored_key, sizeof(stored_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + return strcmp(fields[2], str_c(str)) == 0 ? 1 : 0; +} + +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r) +{ + string_t *str; + struct hmac_context ctx; + unsigned char salt[16]; + unsigned char salted_password[SHA1_RESULTLEN]; + unsigned char client_key[SHA1_RESULTLEN]; + unsigned char server_key[SHA1_RESULTLEN]; + unsigned char stored_key[SHA1_RESULTLEN]; + + random_fill(salt, sizeof(salt)); + + str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(salt))); + str_printfa(str, "%i,", SCRAM_ITERATE_COUNT); + base64_encode(salt, sizeof(salt), str); + + /* FIXME: credentials should be SASLprepped UTF8 data here */ + Hi((const unsigned char *)plaintext, strlen(plaintext), salt, + sizeof(salt), SCRAM_ITERATE_COUNT, salted_password); + + /* Calculate ClientKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); + + /* Calculate StoredKey */ + sha1_get_digest(client_key, sizeof(client_key), stored_key); + str_append_c(str, ','); + base64_encode(stored_key, sizeof(stored_key), str); + + /* Calculate ServerKey */ + hmac_init(&ctx, salted_password, sizeof(salted_password), + &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); + str_append_c(str, ','); + base64_encode(server_key, sizeof(server_key), str); + + safe_memset(salted_password, 0, sizeof(salted_password)); + safe_memset(client_key, 0, sizeof(client_key)); + safe_memset(server_key, 0, sizeof(server_key)); + safe_memset(stored_key, 0, sizeof(stored_key)); + + *raw_password_r = (const unsigned char *)str_c(str); + *size_r = str_len(str); +} diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -822,6 +822,8 @@ { "PLAIN-TRUNC", PW_ENCODING_NONE, 0, plain_trunc_verify, plain_generate }, { "CRAM-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, + { "SCRAM-SHA-1", PW_ENCODING_NONE, 0, scram_sha1_verify, + scram_sha1_generate}, { "HMAC-MD5", PW_ENCODING_HEX, CRAM_MD5_CONTEXTLEN, NULL, cram_md5_generate }, { "DIGEST-MD5", PW_ENCODING_HEX, MD5_RESULTLEN, diff --git a/src/auth/password-scheme.h b/src/auth/password-scheme.h --- a/src/auth/password-scheme.h +++ b/src/auth/password-scheme.h @@ -85,6 +85,12 @@ const unsigned char *raw_password, size_t size, const char **error_r); +int scram_sha1_verify(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char *raw_password, size_t size, + const char **error_r ATTR_UNUSED); +void scram_sha1_generate(const char *plaintext, const char *user ATTR_UNUSED, + const unsigned char **raw_password_r, size_t *size_r); + /* check wich of the algorithms Blowfisch, SHA-256 and SHA-512 are supported by the used libc's/glibc's crypt() */ void password_scheme_register_crypt(void); From tss at iki.fi Wed Oct 3 03:50:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 03 Oct 2012 03:50:00 +0300 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <506B8281.9060703@babelmonkeys.de> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> <506B8281.9060703@babelmonkeys.de> Message-ID: <1349225400.18782.54.camel@innu> On Wed, 2012-10-03 at 02:10 +0200, Florian Zeitz wrote: > Attached is a new export incorporating your feedback. Committed. Also what do you think about the attached patch? (Compiles, untested.) -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 9633 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 3 05:23:57 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 02 Oct 2012 21:23:57 -0500 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <506BA1BD.1070903@brightok.net> On 10/2/2012 4:39 PM, Cor Bosman wrote: > > Anyone else with NFS mailspools seeing this? > > Cor > > I haven't seen them yet, however, to help troubleshoot, see this link and follow it's links for more details on .nfs files http://wordpress.org/support/topic/how-can-i-prevent-unwanted-nfs-files-from-being-created Jack From amateo at um.es Wed Oct 3 09:20:00 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 03 Oct 2012 08:20:00 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: References: <5062DF3C.3050601@um.es> Message-ID: <506BD910.90200@um.es> El 02/10/12 21:38, Timo Sirainen escribi?: > On 26.9.2012, at 13.55, Angel L. Mateo wrote: > >> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >> done > > -S auth-userdb? You've named it completely wrong if that works :) > auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? >> The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: >> >> e62e0d3834ed094e5c7900007efb8a67 66 >> e62e0d3834ed094e5c7900007efb8a67 71 >> e62e0d3834ed094e5c7900007efb8a67 74 >> e62e0d3834ed094e5c7900007efb8a67 >> 75 e62e0d3834ed094e5c7900007efb8a67 >> 77 e62e0d3834ed094e5c7900007efb8a67 >> 78 e62e0d3834ed094e5c7900007efb8a67 > > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/94c7e875f9b9 > Thanks, I'll check as soon as I can. From amateo at um.es Wed Oct 3 09:25:38 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 03 Oct 2012 08:25:38 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> Message-ID: <506BDA62.2010604@um.es> El 02/10/12 22:18, Timo Sirainen escribi?: > On 2.10.2012, at 11.41, Angel L. Mateo wrote: > >> I've been doing some more tests with this problem I have (I need to solve it because I'm planning to migrate mailboxes from maildir to mdbox and I need to change mail_location for my users without rebooting the server). > > You could flush the whole cache also. > Oh... I was so obfuscated trying to expire just the user that I forgot I could flush the whole cache :-( >> I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: > > Flush both the user and user at domain entries? > Yes, I could do this, but why there are entries with user and user at domain?, because I have three user databases: * master password: it is not normally used * pam: I have the cache_key=%n on it * ldap: I don't know to configure cache_key (I tried args = cache_key=%n /etc/dovecot/dovecot-ldap.conf.ext but it didn't work) From tss at iki.fi Wed Oct 3 16:26:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 16:26:34 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <506BDA62.2010604@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <506AA8CF.8090605@um.es> <88CDF1F8-678D-410B-8642-EC29917C0D50@iki.fi> <506BDA62.2010604@um.es> Message-ID: <5B43FF27-C875-48D7-91DA-FA86848B02B9@iki.fi> On 3.10.2012, at 9.25, Angel L. Mateo wrote: >>> I think I have found the source of the problem, although I don't know how to fix it. The problem is that I have different results if I ask for user information with just the login or with the whole email: >> >> Flush both the user and user at domain entries? >> > Yes, I could do this, but why there are entries with user and user at domain?, because I have three user databases: > > * master password: it is not normally used > * pam: I have the cache_key=%n on it > * ldap: I don't know to configure cache_key (I tried args = cache_key=%n /etc/dovecot/dovecot-ldap.conf.ext but it didn't work) For LDAP the cache_key is figured out automatically based on the used %variables. You can't override the cache key. The only way to make it work would be to change the LDAP query to use only %n and no %u/%d (which I guess would be possible by checking for %n@* ?) From tss at iki.fi Wed Oct 3 16:27:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 16:27:32 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <506BD910.90200@um.es> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> Message-ID: <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> On 3.10.2012, at 9.20, Angel L. Mateo wrote: > El 02/10/12 21:38, Timo Sirainen escribi?: >> On 26.9.2012, at 13.55, Angel L. Mateo wrote: >> >>> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >>> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >>> done >> >> -S auth-userdb? You've named it completely wrong if that works :) >> > auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? From florob at babelmonkeys.de Wed Oct 3 16:33:32 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 03 Oct 2012 15:33:32 +0200 Subject: [Dovecot] [PATCH] Add SCRAM-SHA-1 password scheme In-Reply-To: <1349225400.18782.54.camel@innu> References: <506B5734.30906@babelmonkeys.de> <7D0F3C66-CCE1-4EF6-B577-547221934B78@iki.fi> <506B7EAD.1080108@babelmonkeys.de> <506B8281.9060703@babelmonkeys.de> <1349225400.18782.54.camel@innu> Message-ID: <506C3EAC.6000504@babelmonkeys.de> Am 03.10.2012 02:50, schrieb Timo Sirainen: > On Wed, 2012-10-03 at 02:10 +0200, Florian Zeitz wrote: >> Attached is a new export incorporating your feedback. > > Committed. Also what do you think about the attached patch? (Compiles, > untested.) > Moving the passdb parsing into a separate function seems like a nice idea to me. Style changes and removing an unused variable is obviously fine (I'm a bit surprised I got no compiler warning about the latter, but oh well). I did a quick test. Login and error checking seem to still work fine with this patch in place. Wouldn't have seen anything in the code to suggest otherwise either. From list at airstreamcomm.net Wed Oct 3 17:34:51 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 03 Oct 2012 09:34:51 -0500 Subject: [Dovecot] LDA vs LMTP index files Message-ID: <506C4D0B.8030709@airstreamcomm.net> In the docs it states that LDA " ...takes mail from anMTAand delivers it to a user's mailbox, while keeping Dovecot index files up to date." I am wondering if LMTP also interacts with the Dovecot index files and keeps them up to date? From jbates at brightok.net Wed Oct 3 18:06:08 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 03 Oct 2012 10:06:08 -0500 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C4D0B.8030709@airstreamcomm.net> References: <506C4D0B.8030709@airstreamcomm.net> Message-ID: <506C5460.205@brightok.net> On 10/3/2012 9:34 AM, list at airstreamcomm.net wrote: > In the docs it states that LDA " ...takes mail from anMTAand delivers > it to a user's mailbox, while keeping Dovecot index files up to > date." I am wondering if LMTP also interacts with the Dovecot index > files and keeps them up to date? > Brand new account created with LMTP. So I'd say yes. -rw-------. 1 vmail vmail 16384 Oct 2 20:21 dovecot.index.cache -rw-------. 1 vmail vmail 560 Oct 2 20:21 dovecot.index.log -rw-------. 1 vmail vmail 152 Oct 2 20:21 dovecot-uidlist -rw-------. 1 vmail vmail 8 Oct 2 18:58 dovecot-uidvalidity -r--r--r--. 1 vmail vmail 0 Oct 2 18:58 dovecot-uidvalidity.506b393c -rw-------. 1 vmail vmail 28 Oct 2 20:21 maildirsize Jack From CMarcus at Media-Brokers.com Wed Oct 3 18:03:18 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 03 Oct 2012 11:03:18 -0400 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C4D0B.8030709@airstreamcomm.net> References: <506C4D0B.8030709@airstreamcomm.net> Message-ID: <506C53B6.6070706@Media-Brokers.com> On 2012-10-03 10:34 AM, list at airstreamcomm.net wrote: > In the docs it states that LDA " ...takes mail from anMTAand delivers > it to a user's mailbox, while keeping Dovecot index files up to > date." I am wondering if LMTP also interacts with the Dovecot index > files and keeps them up to date? Yes... although it doesn't seem to state that explicitly in the wiki... http://wiki2.dovecot.org/LMTP -- Best regards, Charles From jbates at brightok.net Wed Oct 3 18:13:40 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 03 Oct 2012 10:13:40 -0500 Subject: [Dovecot] LDA vs LMTP index files In-Reply-To: <506C53B6.6070706@Media-Brokers.com> References: <506C4D0B.8030709@airstreamcomm.net> <506C53B6.6070706@Media-Brokers.com> Message-ID: <506C5624.8090609@brightok.net> On 10/3/2012 10:03 AM, Charles Marcus wrote: > On 2012-10-03 10:34 AM, list at airstreamcomm.net > wrote: >> In the docs it states that LDA " ...takes mail from anMTAand delivers >> it to a user's mailbox, while keeping Dovecot index files up to >> date." I am wondering if LMTP also interacts with the Dovecot index >> files and keeps them up to date? > > Yes... although it doesn't seem to state that explicitly in the wiki... > > http://wiki2.dovecot.org/LMTP > "The main difference is that the LDA is a short-running process, started as a binary from command line, while LMTP is a long-running process started by Dovecot's master process." I think they are trying to reduce duplication of information. Jack From patrickdk at patrickdk.com Wed Oct 3 19:02:20 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 03 Oct 2012 12:02:20 -0400 Subject: [Dovecot] possible nfs issue In-Reply-To: References: <00D20478-828B-44E7-A7A8-F7446A53C102@iki.fi> Message-ID: <20121003120220.Horde.5oXwJJLnE6FQbGGMsr6Tr4A@mail.patrickdk.com> Maybe it's a cross program issue? We used to randomly have this happen a long time ago, when using postfix and dovecot. Since switching to using the dovecot lda/lmtp instead of postfix for mailbox delievery, I haven't seen this happen at all anymore. I'm not saying that postfix is at fault for this, but could be a timing/race issue between postfix/dovecot accesses to the mailbox. Quoting Cor Bosman : > On Oct 3, 2012, at 12:35 AM, Timo Sirainen wrote: > >> On 3.10.2012, at 0.45, Timo Sirainen wrote: >> >>> On 3.10.2012, at 0.39, Cor Bosman wrote: >>> >>>> With NFS these files are created when a file gets unlinked, but >>>> another process still has it open. It disappears as soon as the >>>> other process closes it. For some reason they dont disappear. As >>>> far as I can tell we've had no server crashes that could explain >>>> this. One possible theory is that a rename happens after an >>>> unlink. In that case the file remains. This could possibly be a >>>> dovecot issue. >>> >>> How can a rename happen after unlink? The rename should fail. >>> (Unless doing rename(.nfs1234, something), but Dovecot definitely >>> isn't doing that.) >> >> You could see if this old test program leaves .nfs files lying around: >> >> http://dovecot.org/tmp/readdir.c >> >> Just comment out the line: >> >> close(fd); >> > > I meant the .nfs1234 indeed, but it seemed very unlikely. Thanks for > clarifying. The readdir program leaves no .nfs files. We'll have to > explore other possibilities. > > Cor From list at airstreamcomm.net Wed Oct 3 22:26:20 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 03 Oct 2012 14:26:20 -0500 Subject: [Dovecot] LMTP userdb lookup Message-ID: <506C915C.2070709@airstreamcomm.net> Is it possible to have separate userdb lookups for LMTP and POP/IMAP? From marc at perkel.com Wed Oct 3 22:48:21 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 12:48:21 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release Message-ID: <506C9685.8070906@perkel.com> Hi, I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first. I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives. I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend. However What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders? Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains? One thing I can do is deliver the spam to 3 different places so it's visible on all levels. I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system. From ecasarero at gmail.com Wed Oct 3 22:58:35 2012 From: ecasarero at gmail.com (Eduardo Casarero) Date: Wed, 3 Oct 2012 16:58:35 -0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: Hi Marc, i solved this using an automated report for users quarantine. In front of dovecot i have 2 mailscanner boxes that stores spam emails in quarantine and logs them to a database, periodically there is a script that sends an html report to users that recieved spam in the last interval (1h, 4h, 24hs depending on the user preferences) showing a list of time-from-subject of all new items in quarantine. There is also a link to release the email from quarantine and the users recieves it on his inbox. So our users can release emails without bothering anyone. (There is also an admin view where the admin can see all the trafic for the domain). my 2cents. regards, eduardo. 2012/10/3 Marc Perkel > Hi, > > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. (Junk > Email Filter) I want to add a system where I store a copy of spam on a > server and make it available to the customer to review and maybe resent on > false positives. > > I know I could do something simple where I deliver all spam to a domain > account and make it available to an administrator. Then if it's a false > positive they would drag the message to a "resend" folder. I'll have > something the checks the folder one a minute to pick up and resend. > > However > > What would be very cool is delivering the spam to individual accounts. So > a user who logs in individually can see their own spam. But the admin for > the domain would be able to see all users. Maybe the users would appear as > folders? > > Then a master account (me) would be able to log in and see all the domains > as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And I'll > need an authentication system. > > From campbell at cnpapers.com Wed Oct 3 22:59:09 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Wed, 03 Oct 2012 15:59:09 -0400 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506C990D.50800@cnpapers.com> If you ever figure out how to do this, I've got an excellent name for it: MailWatch http://sourceforge.net/projects/mailwatch/ steve On 10/3/2012 3:48 PM, Marc Perkel wrote: > Hi, > > I'm looking for some advice to do a really advanced trick with > Dovecot. I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of > spam on a server and make it available to the customer to review and > maybe resent on false positives. > > I know I could do something simple where I deliver all spam to a > domain account and make it available to an administrator. Then if it's > a false positive they would drag the message to a "resend" folder. > I'll have something the checks the folder one a minute to pick up and > resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the > admin for the domain would be able to see all users. Maybe the users > would appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > From tss at iki.fi Wed Oct 3 23:04:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 03 Oct 2012 23:04:39 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: Maildir, layout=fs /var/vmail/domain/user/ Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Marc Perkel wrote: Hi, I'm looking for some advice to do a really advanced trick with Dovecot. I'm not sure if this can be done. I need to describe first. I have a spam filtering company that does front end spam filtering. (Junk Email Filter) I want to add a system where I store a copy of spam on a server and make it available to the customer to review and maybe resent on false positives. I know I could do something simple where I deliver all spam to a domain account and make it available to an administrator. Then if it's a false positive they would drag the message to a "resend" folder. I'll have something the checks the folder one a minute to pick up and resend. However What would be very cool is delivering the spam to individual accounts. So a user who logs in individually can see their own spam. But the admin for the domain would be able to see all users. Maybe the users would appear as folders? Then a master account (me) would be able to log in and see all the domains as folders and the users as folders inside the domains? One thing I can do is deliver the spam to 3 different places so it's visible on all levels. I'm just wondering if anyone out there has any ideas about that. And I'll need an authentication system. From hakon at alstadheim.priv.no Wed Oct 3 23:14:37 2012 From: hakon at alstadheim.priv.no (=?ISO-8859-1?Q?H=E5kon_Alstadheim?=) Date: Wed, 03 Oct 2012 22:14:37 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506C9CAD.5080004@alstadheim.priv.no> On 03. okt. 2012 21:48, Marc Perkel wrote: > Hi, > > I'm looking for some advice to do a really advanced trick with > Dovecot. I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of > spam on a server and make it available to the customer to review and > maybe resent on false positives. > > I know I could do something simple where I deliver all spam to a > domain account and make it available to an administrator. Then if it's > a false positive they would drag the message to a "resend" folder. > I'll have something the checks the folder one a minute to pick up and > resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the > admin for the domain would be able to see all users. Maybe the users > would appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > > Check out the dovecot sieve plugin. I use the following default pre-filter for all users: --- require ["regex", "fileinto", "imap4flags"]; # Catch mail tagged as Spam, except Spam retrained and delivered to the mailbox if allof (header :regex "X-DSPAM-Result" "^(Spam|Virus|Bl[ao]cklisted)$", not header :contains "X-DSPAM-Reclassified" "Innocent", not header :contains "Received-SPF" "pass .securityfocus.com") { # Mark as read #setflag "\\Seen"; addflag "$junk"; # Move into the Junk folder fileinto "INBOX.Junk"; # Stop processing here stop; } ----- Together with the dovecot antispam plugin this makes the beginnings of a very intuitive system. I just click to remove the junk flag on any false positive, and it gets re-delivered to me. The dovecot lda also supports a switch to deliver to a specific folder I believe. This would be an alternative if you get the spam delivered through a separate channel anyway. ------- The other part of your requirements could be met by using dovecot public folders, which I have never used myself. Maybe set up so admins can subscribe to the junk-folder of any user they want ? Refiling false positives might get messy for an admin though. Regards, H?kon. From tss at iki.fi Wed Oct 3 23:58:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 3 Oct 2012 23:58:21 +0300 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <506C915C.2070709@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> Message-ID: On 3.10.2012, at 22.26, list at airstreamcomm.net wrote: > Is it possible to have separate userdb lookups for LMTP and POP/IMAP? protocol lmtp { userdb { .. } } protocol !lmtp { userdb { .. } } From marc at perkel.com Thu Oct 4 02:42:52 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 16:42:52 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> Message-ID: <506CCD7C.6070507@perkel.com> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? On 10/3/2012 1:04 PM, Timo Sirainen wrote: > Maildir, layout=fs > /var/vmail/domain/user/ > > Spams get delivered there. User has access there. Domain admin has > mail_location=/var/vmail/domain, you have mail_location=/var/vmail > -- > Sent from my Android phone with K-9 Mail. Please excuse my brevity. > > Marc Perkel wrote: > > Hi, > > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of spam > on a server and make it available to the customer to review and maybe > resent on false positives. > > I know I could do something simple where I deliver all spam to a domain > account and make it available to an administrator. Then if it's a false > positive they would drag the message to a "resend" folder. I'll have > something the checks the folder one a minute to pick up and resend. > > However > > What would be very cool is delivering the spam to individual accounts. > So a user who logs in individually can see their own spam. But the admin > > for the domain would be able to see all users. Maybe the users would > appear as folders? > > Then a master account (me) would be able to log in and see all the > domains as folders and the users as folders inside the domains? > > One thing I can do is deliver the spam to 3 different places so it's > visible on all levels. > > I'm just wondering if anyone out there has any ideas about that. And > I'll need an authentication system. > From tss at iki.fi Thu Oct 4 02:46:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 02:46:34 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506CCD7C.6070507@perkel.com> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: On 4.10.2012, at 2.42, Marc Perkel wrote: > On 10/3/2012 1:04 PM, Timo Sirainen wrote: >> Maildir, layout=fs >> /var/vmail/domain/user/ >> >> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail > I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? users: mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs domain admins: mail_location = maildir:/var/vmail/%d:LAYOUT=fs full admins: mail_location = maildir:/var/vmail:LAYOUT=fs The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. From marc at perkel.com Thu Oct 4 03:10:23 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 17:10:23 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: <506CD3EF.6080704@perkel.com> On 10/3/2012 4:46 PM, Timo Sirainen wrote: > On 4.10.2012, at 2.42, Marc Perkel wrote: > >> On 10/3/2012 1:04 PM, Timo Sirainen wrote: >>> Maildir, layout=fs >>> /var/vmail/domain/user/ >>> >>> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail >> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? > users: > mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs > > domain admins: > mail_location = maildir:/var/vmail/%d:LAYOUT=fs > > full admins: > mail_location = maildir:/var/vmail:LAYOUT=fs > > The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. > > I'm testing it now and the user level works. But the other levels I don't see anything. I am a little brain dead today though. I'll test more From kgc at corp.sonic.net Thu Oct 4 04:55:42 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 03 Oct 2012 18:55:42 -0700 Subject: [Dovecot] possible nfs issue In-Reply-To: References: Message-ID: <506CEC9E.9060105@corp.sonic.net> On 10/2/2012 2:39 PM, Cor Bosman wrote: > Anyone else with NFS mailspools seeing this? Yes, it is like 1999 all over again. I haven't had a chance to track them down or setup a cron job to rm them all. All of the ones I'm seeing are ex dovecot.index files but it looks like yours are ex messages? I figured this was a probably a regression in the RHEL6.3 (Sl6.3) (2.6.32-279.9.1.el6.x86_64) kernel. What are you running Cor? -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From marc at perkel.com Thu Oct 4 05:28:19 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 19:28:19 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> Message-ID: <506CF443.5080904@perkel.com> On 10/3/2012 4:46 PM, Timo Sirainen wrote: > On 4.10.2012, at 2.42, Marc Perkel wrote: > >> On 10/3/2012 1:04 PM, Timo Sirainen wrote: >>> Maildir, layout=fs >>> /var/vmail/domain/user/ >>> >>> Spams get delivered there. User has access there. Domain admin has mail_location=/var/vmail/domain, you have mail_location=/var/vmail >> I'm a little confused. What about the cur, new, and tmp directories? How does that fit in? > users: > mail_location = maildir:/var/vmail/%d/%n:LAYOUT=fs > > domain admins: > mail_location = maildir:/var/vmail/%d:LAYOUT=fs > > full admins: > mail_location = maildir:/var/vmail:LAYOUT=fs > > The cur/new/tmp directories are under the /var/vmail/domain/username/. So the spams get delivered to /var/vmail/domain/username/new/ directory. This is visible to the users as their INBOX. Domain admins see the mails under the "username" mailbox. Full admins see the mails under "domain/username" mailbox. > > Hi Timo, Thanks for your help. I think I'm close. This works: mail_location = maildir:/email/%d/%n:LAYOUT=fs This doesn't: mail_location = maildir:/email/%d:LAYOUT=fs The email client doesn't see the directories as folders and nothing is visible. I must be missing something. From marc at perkel.com Thu Oct 4 07:40:37 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 21:40:37 -0700 Subject: [Dovecot] Getting rid of the subscription file Message-ID: <506D1345.9070406@perkel.com> I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? Thanks in advance. From daniel.parthey at informatik.tu-chemnitz.de Thu Oct 4 07:58:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 4 Oct 2012 06:58:18 +0200 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1345.9070406@perkel.com> References: <506D1345.9070406@perkel.com> Message-ID: <20121004045818.GA15696@daniel.localdomain> Marc Perkel wrote: > I'd like to eliminate the subscription file and have it just list > the folders that are there. How do I do that? Dovecot allows to automatically subscribe folders when mails are delivered: lda_mailbox_autosubscribe = yes RFC 3501 tells that the server side MUST NOT unilaterally remove an existing mailbox name from the subscription list even if a mailbox by that name no longer exists, see SUBSCRIBE in section 6.3.6: http://tools.ietf.org/html/rfc3501#section-6.3.6 Regards Daniel -- https://plus.google.com/103021802792276734820 From marc at perkel.com Thu Oct 4 08:06:28 2012 From: marc at perkel.com (Marc Perkel) Date: Wed, 03 Oct 2012 22:06:28 -0700 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <20121004045818.GA15696@daniel.localdomain> References: <506D1345.9070406@perkel.com> <20121004045818.GA15696@daniel.localdomain> Message-ID: <506D1954.1000204@perkel.com> On 10/3/2012 9:58 PM, Daniel Parthey wrote: > Marc Perkel wrote: >> I'd like to eliminate the subscription file and have it just list >> the folders that are there. How do I do that? > Dovecot allows to automatically subscribe folders when mails are delivered: > lda_mailbox_autosubscribe = yes > > RFC 3501 tells that the server side MUST NOT unilaterally remove an > existing mailbox name from the subscription list even if a mailbox > by that name no longer exists, see SUBSCRIBE in section 6.3.6: > http://tools.ietf.org/html/rfc3501#section-6.3.6 > > Regards > Daniel In my case I don't care what the standard is. I want to get rid of the subscription concept completely. From amateo at um.es Thu Oct 4 09:14:29 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 04 Oct 2012 08:14:29 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> Message-ID: <506D2945.9040208@um.es> El 03/10/12 15:27, Timo Sirainen escribi?: > On 3.10.2012, at 9.20, Angel L. Mateo wrote: > >> El 02/10/12 21:38, Timo Sirainen escribi?: >>> On 26.9.2012, at 13.55, Angel L. Mateo wrote: >>> >>>> doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do >>>> doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; >>>> done >>> >>> -S auth-userdb? You've named it completely wrong if that works :) >>> >> auth-userdb is the socket for the auth system. I has always worked for me (I don't know why). What socket shoud I use? director-userdb? > > -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? > Yes, I have tried and it works without using -S. So, what is the reason for this option? Because I'm sure I'm using because I've read it in examples (not with auth-userdb obviously, this is my mistake) From robert at schetterer.org Thu Oct 4 09:36:39 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 04 Oct 2012 08:36:39 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506C9685.8070906@perkel.com> References: <506C9685.8070906@perkel.com> Message-ID: <506D2E77.5030303@schetterer.org> Am 03.10.2012 21:48, schrieb Marc Perkel: > I'm looking for some advice to do a really advanced trick with Dovecot. > I'm not sure if this can be done. I need to describe first. > > I have a spam filtering company that does front end spam filtering. > (Junk Email Filter) I want to add a system where I store a copy of spam > on a server and make it available to the customer to review and maybe > resent on false positives. this is the job of your filter comapny first, anyway , dont use them anymore and use i.e amavis with quarantaine i dont think other cases make sense in real by getting very complicated -- Best Regards MfG Robert Schetterer From cor at xs4all.nl Thu Oct 4 11:11:56 2012 From: cor at xs4all.nl (Cor Bosman) Date: Thu, 4 Oct 2012 10:11:56 +0200 Subject: [Dovecot] possible nfs issue In-Reply-To: <506CEC9E.9060105@corp.sonic.net> References: <506CEC9E.9060105@corp.sonic.net> Message-ID: <5A995EF4-6EAA-41FF-926B-912FFD59EC07@xs4all.nl> On Oct 4, 2012, at 3:55 AM, Kelsey Cummings wrote: > On 10/2/2012 2:39 PM, Cor Bosman wrote: >> Anyone else with NFS mailspools seeing this? > > Yes, it is like 1999 all over again. I haven't had a chance to track them down or setup a cron job to rm them all. All of the ones I'm seeing are ex dovecot.index files but it looks like yours are ex messages? > > I figured this was a probably a regression in the RHEL6.3 (Sl6.3) (2.6.32-279.9.1.el6.x86_64) kernel. What are you running Cor? We're running debian with a 3.2.2 kernel. Just yesterday one of my colleagues had a few new ones in his mailspool. Definitely no server crash or anything. Something is creating these outside the 'normal' parameters for .nfs files. My colleague said these were emails he deleted that day. We've set up a cleaning run, and are probably going to ignore it for now. These things are near impossible to track down without a lot of debugging. Cor From marc at perkel.com Thu Oct 4 15:54:35 2012 From: marc at perkel.com (Marc Perkel) Date: Thu, 04 Oct 2012 05:54:35 -0700 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506D2E77.5030303@schetterer.org> References: <506C9685.8070906@perkel.com> <506D2E77.5030303@schetterer.org> Message-ID: <506D870B.5020001@perkel.com> On 10/3/2012 11:36 PM, Robert Schetterer wrote: > Am 03.10.2012 21:48, schrieb Marc Perkel: >> I'm looking for some advice to do a really advanced trick with Dovecot. >> I'm not sure if this can be done. I need to describe first. >> >> I have a spam filtering company that does front end spam filtering. >> (Junk Email Filter) I want to add a system where I store a copy of spam >> on a server and make it available to the customer to review and maybe >> resent on false positives. > this is the job of your filter comapny first, > anyway , dont use them anymore and use i.e amavis with quarantaine > i dont think other cases make sense in real by getting very complicated I am the spam filtering company. :) From benedetto.vassallo at unipa.it Thu Oct 4 16:00:03 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Thu, 04 Oct 2012 15:00:03 +0200 Subject: [Dovecot] Maildir hardlinks Message-ID: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Hello list, Excuse me for my poor english. I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. All works fine, but with the new version it seems that dovecot don't do hardlinks when deliver a message to multiple users. I have checked my config and the only rule I can see aboout that is maildir_copy_with_hardlinks = yes in /etc/dovecot/conf.d/10-mail.conf My mail location config is mail_location = maildir:~/MailDir:LAYOUT=fs I tryed using lmtp directly issuing 'telnet localhost 24' and sending a test message to 3 recipients. Then issuing a 'ls -il' in the "new" directory of that users, I saw the inode was not the same. I rechecked my config and take a look in the wiki and in the list for someone who had the same problems, but found nothing. Maybe I did something wrong, but I can't understand what. Can you help me? Thank you -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From tss at iki.fi Thu Oct 4 17:20:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:20:03 +0300 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1345.9070406@perkel.com> References: <506D1345.9070406@perkel.com> Message-ID: On 4.10.2012, at 7.40, Marc Perkel wrote: > I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? Write a plugin. From tss at iki.fi Thu Oct 4 17:21:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:21:02 +0300 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <506D2945.9040208@um.es> References: <5062DF3C.3050601@um.es> <506BD910.90200@um.es> <285073A5-D19C-4F77-8D83-63A01A22B780@iki.fi> <506D2945.9040208@um.es> Message-ID: <2E23C2EC-6324-4182-9828-1F063F63C86C@iki.fi> On 4.10.2012, at 9.14, Angel L. Mateo wrote: >> -S points to doveadm-server socket. Sounds like it's not being used at all, so you can probably just leave it out? >> > Yes, I have tried and it works without using -S. So, what is the reason for this option? Because I'm sure I'm using because I've read it in examples (not with auth-userdb obviously, this is my mistake) I'm guessing that it's used only when the user lookup isn't returning proxy=y From tss at iki.fi Thu Oct 4 17:29:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:29:02 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506CF443.5080904@perkel.com> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> Message-ID: <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> On 4.10.2012, at 5.28, Marc Perkel wrote: > Thanks for your help. I think I'm close. > > This works: > mail_location = maildir:/email/%d/%n:LAYOUT=fs > > This doesn't: > mail_location = maildir:/email/%d:LAYOUT=fs > > The email client doesn't see the directories as folders and nothing is visible. I must be missing something. Dunno. At least this method of testing works: create test mail: doveadm -O -o mail=maildir:/tmp/vmail/domain/user mailbox create INBOX touch /tmp/vmail/domain/user/cur/newmail test that user at domain works: ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs a select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS test that domain works: ./imap -O -o mail=maildir:/tmp/vmail/domain:LAYOUT=fs b list "" * * LIST (\HasNoChildren) "/" "user" * LIST (\HasNoChildren) "/" "INBOX" b OK List completed. c select user * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 0 RECENT test that superuser works: ./imap -O -o mail=maildir:/tmp/vmail:LAYOUT=fs d list "" * * LIST (\Noselect \HasChildren) "/" "domain" * LIST (\HasNoChildren) "/" "domain/user" * LIST (\HasNoChildren) "/" "INBOX" d OK List completed. e select domain/user * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS From list at airstreamcomm.net Thu Oct 4 17:41:39 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 04 Oct 2012 09:41:39 -0500 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: References: <506C915C.2070709@airstreamcomm.net> Message-ID: <506DA023.5030609@airstreamcomm.net> On 10/3/12 3:58 PM, Timo Sirainen wrote: > On 3.10.2012, at 22.26, list at airstreamcomm.net wrote: > >> Is it possible to have separate userdb lookups for LMTP and POP/IMAP? > protocol lmtp { > userdb { > .. > } > } > protocol !lmtp { > userdb { > .. > } > } > > Forgot to mention I am running 2.0.17. I separated all the userdb passdb lookups into their own protocol configuration like so: protocol imap { userdb { .. } passdb { .. } } protocol pop3 { userdb { .. } passdb { .. } } protocol lmtp { userdb { .. } } And I am getting the following error: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one From a previous post it appears that Dovecot cannot run without a global lookups specified: http://www.dovecot.org/list/dovecot/2012-March/064407.html Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? From tss at iki.fi Thu Oct 4 17:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 4 Oct 2012 17:58:53 +0300 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <506DA023.5030609@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> Message-ID: On 4.10.2012, at 17.41, list at airstreamcomm.net wrote: >> protocol lmtp { >> userdb { >> .. >> } >> } >> protocol !lmtp { >> userdb { >> .. >> } >> } >> >> > Forgot to mention I am running 2.0.17. The above works in v2.1. > And I am getting the following error: > > auth: Fatal: No passdbs specified in configuration file. PLAIN > mechanism needs one > > > From a previous post it appears that Dovecot cannot run without a global lookups specified: > > http://www.dovecot.org/list/dovecot/2012-March/064407.html > > Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? The !lmtp version avoids that fatal problem. So the solution is: upgrade. From CMarcus at Media-Brokers.com Thu Oct 4 18:03:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 04 Oct 2012 11:03:02 -0400 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Message-ID: <506DA526.4020606@Media-Brokers.com> On 2012-10-04 9:00 AM, Benedetto Vassallo wrote: > Hello list, > Excuse me for my poor english. > I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. If you are going to update, why ohy why update to an outdated version? Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... From benedetto.vassallo at unipa.it Thu Oct 4 19:13:41 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Thu, 04 Oct 2012 18:13:41 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <506DA526.4020606@Media-Brokers.com> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <506DA526.4020606@Media-Brokers.com> Message-ID: <20121004181341.14266g7w0m2ie75h@webmail.unipa.it> Def. Quota Charles Marcus : > On 2012-10-04 9:00 AM, Benedetto Vassallo > wrote: >> Hello list, >> Excuse me for my poor english. >> I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. > > If you are going to update, why ohy why update to an outdated version? > > Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... > I did it, but still don't work :-( -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From slusarz at curecanti.org Thu Oct 4 21:57:45 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Thu, 04 Oct 2012 12:57:45 -0600 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: <506D1954.1000204@perkel.com> References: <506D1345.9070406@perkel.com> <20121004045818.GA15696@daniel.localdomain> <506D1954.1000204@perkel.com> Message-ID: <20121004125745.Horde.Wz49XoF5lbhQbdwp53YwxRA@bigworm.curecanti.org> Quoting Marc Perkel : > On 10/3/2012 9:58 PM, Daniel Parthey wrote: >> Marc Perkel wrote: >>> I'd like to eliminate the subscription file and have it just list >>> the folders that are there. How do I do that? [snip] >> RFC 3501 tells that the server side MUST NOT unilaterally remove an >> existing mailbox name from the subscription list even if a mailbox >> by that name no longer exists, see SUBSCRIBE in section 6.3.6: >> http://tools.ietf.org/html/rfc3501#section-6.3.6 >> >> Regards >> Daniel > > In my case I don't care what the standard is. I want to get rid of > the subscription concept completely. Use an MUA that allows configuration to explicitly ignore the subscription concept. michael From micha at krausam.de Fri Oct 5 10:45:07 2012 From: micha at krausam.de (Micha Krause) Date: Fri, 05 Oct 2012 09:45:07 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> Message-ID: <506E9003.7030201@krausam.de> Hi, > ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs Wow, thats a really cool way to debug/test mailboxes, is this documented somewhere? What does -O do, any other interesting options? Micha Krause From stsiol at yahoo.co.uk Fri Oct 5 15:00:20 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Fri, 5 Oct 2012 13:00:20 +0100 (BST) Subject: [Dovecot] horde sync status ? Message-ID: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Hello all, I had a quick look at the horde site and noticed that horde is being advertised as, let's say, "smartphone friendly". Does anyone know if the newest horde version can "talk" to? smart phones in regards to e-mails ? In other words, can a user owning a smartphone get his/her e-mails on it apart from the webpage ? Thank you, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From h.reindl at thelounge.net Fri Oct 5 15:05:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 05 Oct 2012 14:05:03 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506ECCEF.1020904@thelounge.net> Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > Hello all, > > I had a quick look at the horde site and noticed that > horde is being advertised as, let's say, "smartphone friendly". > > Does anyone know if the newest horde version can "talk" to > smart phones in regards to e-mails ? > > In other words, can a user owning a smartphone get his/her > e-mails on it apart from the webpage? a little off-topic at all, but however horde/imp is a webmail and accessing the mailserver via IMAP smart-phone freindly means it can be used on smartphones without scrolling to dead horde is not for having a layer between mail-client on the smartphone and the server - this makes pretty no sense each smartphone these days has a mail-client like K9 on android and can access imap/exchange directly - why should horde be involved here als additional layer? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From patrickdk at patrickdk.com Fri Oct 5 17:17:44 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Fri, 05 Oct 2012 10:17:44 -0400 Subject: [Dovecot] horde sync status ? In-Reply-To: <506ECCEF.1020904@thelounge.net> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506ECCEF.1020904@thelounge.net> Message-ID: <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> Quoting Reindl Harald : > Am 05.10.2012 14:00, schrieb Spyros Tsiolis: >> Hello all, >> >> I had a quick look at the horde site and noticed that >> horde is being advertised as, let's say, "smartphone friendly". >> >> Does anyone know if the newest horde version can "talk" to >> smart phones in regards to e-mails ? >> >> In other words, can a user owning a smartphone get his/her >> e-mails on it apart from the webpage? > > a little off-topic at all, but however > > horde/imp is a webmail and accessing the mailserver via IMAP > smart-phone freindly means it can be used on smartphones > without scrolling to dead > > horde is not for having a layer between mail-client on the > smartphone and the server - this makes pretty no sense > > each smartphone these days has a mail-client like K9 on > android and can access imap/exchange directly - why should > horde be involved here als additional layer? Many reasons for this, I personally use it for contact and calender sync, and the new version of horde that is still in beta, can also be used for email sync too. This will simplify setup for many people, using autodiscovery feature of activesync. From tss at iki.fi Fri Oct 5 17:31:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 5 Oct 2012 17:31:18 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <506E9003.7030201@krausam.de> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> Message-ID: <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> On 5.10.2012, at 10.45, Micha Krause wrote: >> ./imap -O -o mail=maildir:/tmp/vmail/domain/user:LAYOUT=fs > > Wow, thats a really cool way to debug/test mailboxes, is this documented somewhere? No. The -O, -o, -k and some other options should be put into some new global.inc where it gets included to all doveadm/dovecot/doveconf man pages.. > What does -O do, any other interesting options? All the global settings are: -O ignores dovecot.conf and just uses the default settings. -o = can be used multiple times to override any setting -k preserves environment variables (which can also be used to override settings, e.g. MAIL=foo) -c changes dovecot.conf path -i changes to dovecot.conf used by the given instance name -L logs directly to destination specified by log_path/info_log_path/debug_log_path, bypassing log process (allowing logging to different location than normally, log process always logs only to one location) From h.reindl at thelounge.net Fri Oct 5 17:38:50 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 05 Oct 2012 16:38:50 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506ECCEF.1020904@thelounge.net> <20121005101744.Horde.Y1nzH5LnE6FQbuwIDldzhcA@mail.patrickdk.com> Message-ID: <506EF0FA.1010307@thelounge.net> Am 05.10.2012 16:17, schrieb Patrick Domack: >> each smartphone these days has a mail-client like K9 on >> android and can access imap/exchange directly - why should >> horde be involved here als additional layer? > > Many reasons for this, I personally use it for contact and calender sync, and the new version of horde that is > still in beta, can also be used for email sync too. This will simplify setup for many people, using autodiscovery > feature of activesync. why does one need this with IMAP as mail-backend? sounds more like "having solution and searching fro problem" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From robert at schetterer.org Fri Oct 5 17:56:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 05 Oct 2012 16:56:36 +0200 Subject: [Dovecot] horde sync status ? In-Reply-To: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506EF524.7060604@schetterer.org> Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > Hello all, > > I had a quick look at the horde site and noticed that > horde is being advertised as, let's say, "smartphone friendly". > > Does anyone know if the newest horde version can "talk" to > smart phones in regards to e-mails ? yes since version 5 > > In other words, can a user owning a smartphone get his/her > e-mails on it apart from the webpage ? horde 5 acts as active-sync server for mail , calendar, adressbook ,tasks ,notes syncml with funambol app on the smartphone side for calendar, adressbook ,tasks ,notes roadmap 5.1 is planned as card/caldav server http://wiki.horde.org/ActiveSync > > Thank you, > > spyros > > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > this is off topic with the dovecot list -- Best Regards MfG Robert Schetterer From mikydevel at yahoo.fr Fri Oct 5 20:30:45 2012 From: mikydevel at yahoo.fr (Mik J) Date: Fri, 5 Oct 2012 18:30:45 +0100 (BST) Subject: [Dovecot] Dovecot configuration and question about IP trusted Message-ID: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> Hello list, I've just finished to install Dovecot and things seems to work so far. After some little efforts though. My version is 2.0.20 Question 1: I'm trying to tighten the security a little bit and added in dovecot.conf login_trusted_networks = 192.168.1.0/30 Then restarted Dovecot My client has the IP 192.168.1.20 and it's still able to retrieve emails. I expected it to be forbidden. Am I missing something ? # dovecot -n | grep trust login_trusted_networks = 192.168.1.0/30 Question 2: I feel that Dovecot is slow. I'm doing my test with my iphone as an imap client. Test 1: I retrieve a mail on a remote server provided by a hosting company, it takes 2 seconds Test 2: I retrieve a mail on my server which is on my LAN, the mail includes a few letters in the subject and a few letters in the body. The action takes about 8 seconds. It's quite subtule to measure so first I would like to know if Dovecot tries to do a dns reverse lookup or something like that. And it would explain the overhead. Thank you From bob at computerisms.ca Fri Oct 5 21:01:13 2012 From: bob at computerisms.ca (Bob Miller) Date: Fri, 05 Oct 2012 11:01:13 -0700 Subject: [Dovecot] Dovecot configuration and question about IP trusted In-Reply-To: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> Message-ID: <1349460073.4213.59.camel@worklian> Hi, > I > 'm trying to tighten the security a little bit and added in dovecot.conf > login_trusted_networks = 192.168.1.0/30 > Then restarted Dovecot > > > My client has the IP 192.168.1.20 and it's still able to retrieve emails. I expected it to be forbidden. Am I missing something ? My interpretation of the documentation indicates that the trusted network setting causes certain authentication and security checks to be bypassed if a computer is in the trusted network, and to not bypass those authentication and security checks if the computer is not in the trusted range. I see nothing indicating this setting will "forbid" anything... > I feel that Dovecot is slow. I'm doing my test with my iphone as an imap client. > Test 1: I retrieve a mail on a remote server provided by a hosting company, it takes 2 seconds > Test 2: I retrieve a mail on my server which is on my LAN, the mail includes a few letters in the subject and a few letters in the body. The action takes about 8 seconds. > It's quite subtule to measure so first I would like to know if Dovecot tries to do a dns reverse lookup or something like that. And it would explain the overhead. I don't know about the reverse lookup, but this sounds like a caching issue to me. http://wiki2.dovecot.org/IndexFiles > > Thank you -- Computerisms Bob Miller 867-334-7117 / 867 633 3760 http://computerisms.ca From lists at luigirosa.com Fri Oct 5 21:14:53 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Fri, 05 Oct 2012 20:14:53 +0200 Subject: [Dovecot] IPv6 & SSL Message-ID: <506F239D.6090007@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I have a dual stack server with Dovecot 2.1.10 listening on v4 and v6 Dovecot has a Comodo SSL certificate issued via NameCheap that works as expected with IPv4 in 10-ssl.conf I have enabled these configuraction directives: ssl = yes ssl_cert = < /path/to/file.crt ssl_key = < /path/to/file.key ssl_parameters_regenerate = 202 hours If I connect to Dovecot using the IPv6 address of the server with Thunderbird 15.0.1 uising CRAM-MD5 averything is ok. If I enable SSL _and_ IPv6 on Thunderbird I get this error: Oct 5 20:05:04 mail dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=2001:470:1f09:203:fdbf:508e:4a29:56c5, lip=2001:470:1f09:203::badd:ecaf, TLS: SSL_read() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, session= Ciao, luigi - -- / +--[Luigi Rosa]-- \ I will tell you a great secret, Captain. Perhaps the greatest of all time. The molecules of your body are the same molecules that make up this station and the nebula outside, that burn inside the stars themselves. We are star stuff, we are the universe made manifest, trying to figure itself out. As we have both learned, sometimes the universe requires a change of perspective." --Delenn, "Distant Star", Babylon 5 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBvI50ACgkQ3kWu7Tfl6ZRBSACfRkp4FYpWaEZUQhIh0t6Vfs/I JbcAoKGZ769yogYS7faCXKvPTuhQiHA8 =jxCB -----END PGP SIGNATURE----- From lists at wildgooses.com Fri Oct 5 22:37:37 2012 From: lists at wildgooses.com (Ed W) Date: Fri, 05 Oct 2012 20:37:37 +0100 Subject: [Dovecot] horde sync status ? In-Reply-To: <506EF524.7060604@schetterer.org> References: <1349438420.39014.YahooMailNeo@web132203.mail.ird.yahoo.com> <506EF524.7060604@schetterer.org> Message-ID: <506F3701.5050805@wildgooses.com> On 05/10/2012 15:56, Robert Schetterer wrote: > Am 05.10.2012 14:00, schrieb Spyros Tsiolis: > >> In other words, can a user owning a smartphone get his/her >> e-mails on it apart from the webpage ? > horde 5 acts as active-sync server > for mail , calendar, adressbook ,tasks ,notes > > syncml with funambol app on the smartphone side > for calendar, adressbook ,tasks ,notes > > roadmap > 5.1 is planned as card/caldav server > > http://wiki.horde.org/ActiveSync > Also see Sogo (and owncloud). Plus the Sogosync connector This is a developing area (at last) Ed W From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 5 22:48:47 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 5 Oct 2012 21:48:47 +0200 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> Message-ID: <20121005194847.GA15222@daniel.localdomain> Timo Sirainen wrote: > -i changes to dovecot.conf used by the given instance name This does not seem to work, at least not with version 2.1.10: mail01:~# doveadm instance list path name last used running /var/run/dovecot dovecot-mailbox 2012-10-05 19:19:33 yes /var/run/dovecot-director dovecot-director 2012-10-05 19:20:13 yes mail01:~# doveadm -c /etc/dovecot-director/dovecot-director.conf director status dparthey at example.org Current: 10.129.3.192 (expires 2012-10-07 20:10:25) Hashed: 10.129.3.192 Initial config: 10.129.3.192 mail01:~# doveadm -i dovecot-director director status dparthey at example.org doveadm(root): Fatal: read(/var/run/dovecot/director-admin) failed: Connection reset by peer Regards Daniel -- https://plus.google.com/103021802792276734820 From nick+dovecot at bunbun.be Fri Oct 5 23:47:53 2012 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Fri, 05 Oct 2012 22:47:53 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F239D.6090007@luigirosa.com> References: <506F239D.6090007@luigirosa.com> Message-ID: <506F4779.4040109@bunbun.be> Luigi Rosa wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > I have a dual stack server with Dovecot 2.1.10 listening on v4 and v6 > > Dovecot has a Comodo SSL certificate issued via NameCheap that works as > expected with IPv4 > > in 10-ssl.conf I have enabled these configuraction directives: > > ssl = yes > ssl_cert =< /path/to/file.crt > ssl_key =< /path/to/file.key > ssl_parameters_regenerate = 202 hours > > > If I connect to Dovecot using the IPv6 address of the server with Thunderbird > 15.0.1 uising CRAM-MD5 averything is ok. > If I enable SSL _and_ IPv6 on Thunderbird I get this error: How do you enable this in Thunderbird? If by "enabling IPv6" you mean you put in the IPv6 address in stead of the hostname, that's probably where you're wrong. The certificate contains your hostname, not the IP-address so the hostname verification check fails if you insert the IPv6 address (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your connection fails). I've verified this by changing the hostname to IPv6 in Thunderbird and got the same error as you do. You would get the same error if you configure the IPv4 address in TB. > Oct 5 20:05:04 mail dovecot: imap-login: Disconnected (no auth attempts in 1 > secs): user=<>, rip=2001:470:1f09:203:fdbf:508e:4a29:56c5, > lip=2001:470:1f09:203::badd:ecaf, TLS: SSL_read() failed: error:14094418:SSL > routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, > session= This is a valid connection when I use the hostname: 2012-10-04T18:07:51.614187+02:00 mail dovecot: imap-login: Login: user=, method=CRAM-MD5, rip=yyyy:yyyy:::yyyy, lip=xxxx:xxxx:::xxxx, mpid=58179, TLS, TLSv1 with cipher RC4-MD5 (128/128 bits) Configure your DNS so your hostname points to both the IPv6 and IPv4 address. Your client will take take whichever protocol is preferred (IPv4 or IPv6). Rgds, N. > > Ciao, > luigi > > - -- > / > +--[Luigi Rosa]-- > \ > > I will tell you a great secret, Captain. Perhaps the greatest of all > time. The molecules of your body are the same molecules that make up > this station and the nebula outside, that burn inside the stars > themselves. We are star stuff, we are the universe made manifest, > trying to figure itself out. As we have both learned, sometimes > the universe requires a change of perspective." > --Delenn, "Distant Star", Babylon 5 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Mozilla -http://www.enigmail.net/ > > iEYEARECAAYFAlBvI50ACgkQ3kWu7Tfl6ZRBSACfRkp4FYpWaEZUQhIh0t6Vfs/I > JbcAoKGZ769yogYS7faCXKvPTuhQiHA8 > =jxCB > -----END PGP SIGNATURE----- From lists at luigirosa.com Sat Oct 6 08:20:20 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 07:20:20 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F4779.4040109@bunbun.be> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> Message-ID: <506FBF94.30607@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nick Rosier said the following on 05/10/12 22:47: >> How do you enable this in Thunderbird? If by "enabling IPv6" you mean you >> put in the IPv6 address in stead of the hostname, that's probably where >> you're wrong. The certificate contains your hostname, not the IP-address >> so the hostname verification check fails if you insert the IPv6 address >> (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your >> connection fails). Good point. But does not explain why it works if I put the IPv4 address of the server (the local LAN IPv4, not the public IPv4). >> I've verified this by changing the hostname to IPv6 in Thunderbird and >> got the same error as you do. You would get the same error if you >> configure the IPv4 address in TB. The server I am referring to has 2 NICs one with a public IP and the other with a local IP address (10.0.0.254) If I put 10.0.0.254 instead of the IPv6 address I can successfully connect using TLS: Oct 6 07:13:44 mail dovecot: imap-login: Login: user=, method=CRAM-MD5, rip=10.0.0.155, lip=10.0.0.254, mpid=17812, TLS, session= >> Configure your DNS so your hostname points to both the IPv6 and IPv4 >> address. Your client will take take whichever protocol is preferred (IPv4 >> or IPv6). Thunderbird uses IPv4 as mail protocol, I wanted to test IPv6... Thank you for your help Ciao, luigi - -- / +--[Luigi Rosa]-- \ Success is 99% failure. --Soichiro Honda -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBvv4kACgkQ3kWu7Tfl6ZQp2wCgvXPgRGANlAIaVkMvXZHIThYE OiwAoIOqIMD+3mT1znMl6lCCbHanwBta =B/r2 -----END PGP SIGNATURE----- From kamath at moltingpenguin.com Sat Oct 6 08:44:56 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Fri, 5 Oct 2012 22:44:56 -0700 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506FBF94.30607@luigirosa.com> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> <506FBF94.30607@luigirosa.com> Message-ID: <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> On Oct 5, 2012, at 10:20 PM, Luigi Rosa wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Nick Rosier said the following on 05/10/12 22:47: > >>> How do you enable this in Thunderbird? If by "enabling IPv6" you mean you >>> put in the IPv6 address in stead of the hostname, that's probably where >>> you're wrong. The certificate contains your hostname, not the IP-address >>> so the hostname verification check fails if you insert the IPv6 address >>> (i.e. hostname.tld != 2001:470:1f09:203:fdbf:508e:4a29:56c5so your >>> connection fails). > > Good point. But does not explain why it works if I put the IPv4 address of the > server (the local LAN IPv4, not the public IPv4). > >>> I've verified this by changing the hostname to IPv6 in Thunderbird and >>> got the same error as you do. You would get the same error if you >>> configure the IPv4 address in TB. > > The server I am referring to has 2 NICs one with a public IP and the other > with a local IP address (10.0.0.254) > > If I put 10.0.0.254 instead of the IPv6 address I can successfully connect > using TLS: > > Oct 6 07:13:44 mail dovecot: imap-login: Login: user=, > method=CRAM-MD5, rip=10.0.0.155, lip=10.0.0.254, mpid=17812, TLS, > session= And do you have a PTR record for 10.0.0.254? Sean From lists at luigirosa.com Sat Oct 6 09:33:31 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 08:33:31 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> References: <506F239D.6090007@luigirosa.com> <506F4779.4040109@bunbun.be> <506FBF94.30607@luigirosa.com> <5447B3C9-5EB1-4ABE-B396-2A48B406FB38@moltingpenguin.com> Message-ID: <506FD0BB.2020000@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sean Kamath said the following on 06/10/12 07:44: >> Oct 6 07:13:44 mail dovecot: imap-login: Login: >> user=, method=CRAM-MD5, rip=10.0.0.155, >> lip=10.0.0.254, mpid=17812, TLS, session= > > And do you have a PTR record for 10.0.0.254? No, no PTR o other DNS entry for that address. No entry of that address in /etc/hosts on the Linux with Thunderbird or on the Linux with Dovecot. Ciao, luigi - -- / +--[Luigi Rosa]-- \ The past was erased, the erasure was forgotten, the lie became truth. --George Orwell, "1984" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBv0LsACgkQ3kWu7Tfl6ZRTUgCgh1epu40NUiZ6CPlBrcFZezt/ nMYAnjUS5IxodwJfW7o9pJHfKoVCc9xK =8O4T -----END PGP SIGNATURE----- From pw at wk-serv.de Sat Oct 6 10:29:05 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 06 Oct 2012 09:29:05 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506F239D.6090007@luigirosa.com> References: <506F239D.6090007@luigirosa.com> Message-ID: Can you provide the output of doveconf -n? Regards Patrick From lists at luigirosa.com Sat Oct 6 11:10:40 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 06 Oct 2012 10:10:40 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: References: <506F239D.6090007@luigirosa.com> Message-ID: <506FE780.9000900@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Patrick Westenberg said the following on 06/10/12 09:29: > Can you provide the output of doveconf -n? Sure, here it is: # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.1.1.el5.centos.plus x86_64 CentOS release 5.8 (Final) auth_cache_negative_ttl = 0 auth_cache_size = 100 k auth_cache_ttl = 8 hours auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes base_dir = /var/run/dovecot/ login_greeting = Ready. login_trusted_networks = 10.0.0.0/24 mail_plugins = " stats" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 10s stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service stats { fifo_listener stats-mail { mode = 0666 } } ssl_cert = References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> Message-ID: <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> > De?: Bob Miller > > Hi, >> I'm trying to tighten the security a little bit and added in dovecot.conf > >> login_trusted_networks = 192.168.1.0/30 >> Then restarted Dovecot >> >> >> My client has the IP 192.168.1.20 and it's still able to retrieve > emails. I expected it to be forbidden. Am I missing something ? > > My interpretation of the documentation indicates that the trusted > network setting causes certain authentication and security checks to be > bypassed if a computer is in the trusted network, and to not bypass > those authentication and security checks if the computer is not in the > trusted range.? I see nothing indicating this setting will "forbid" > anything... > >> I feel that Dovecot is slow. I'm doing my test with my iphone as an > imap client. >> Test 1: I retrieve a mail on a remote server provided by a hosting company, > it takes 2 seconds >> Test 2: I retrieve a mail on my server which is on my LAN, the mail > includes a few letters in the subject and a few letters in the body. The action > takes about 8 seconds. >> It's quite subtule to measure so first I would like to know if Dovecot > tries to do a dns reverse lookup or something like that. And it would explain > the overhead. > > I don't know about the reverse lookup, but this sounds like a caching > issue to me.? http://wiki2.dovecot.org/IndexFiles Hello Bob, Thank you for this clarification about the parameter login_trusted_networks. Regarding the indexfiles, I've read the page but I don't see at any moment, how to enable or disable the indexes. Also how do you understand this sentence "# Don't use mmap() at all. This is required if you store indexes to shared # filesystems (NFS or clustered filesystem) or for some operating systems # which use a separate cache for mmap, such as OpenBSD. mmap_disable = yes" I've read it 10 times, and I don't know if this should be set to yes or no (probably because my english is not perfect). My operating system is OpenBSD and I don't share NFS or cluster filesystems. Thank you From pw at wk-serv.de Sat Oct 6 13:02:30 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 06 Oct 2012 12:02:30 +0200 Subject: [Dovecot] IPv6 & SSL In-Reply-To: <506FE780.9000900@luigirosa.com> References: <506F239D.6090007@luigirosa.com> <506FE780.9000900@luigirosa.com> Message-ID: <507001B6.2000704@wk-serv.de> Hi Luigi, with regard to SSL my configuration is much more simple and it works fine with IPv4 and IPv6. But you have of course to use a hostname matching the certificates common name. # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain login director_mail_servers = 172.17.1.1 172.17.1.2 director_servers = 172.17.1.3 172.17.1.4 lmtp_proxy = yes log_path = /var/log/dovecot.log managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacati on subaddress comparator-i;ascii-numeric relational regex imap4flags copy includ e variables body enotify environment mailbox date ihave protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 172.17.1.3 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 172.17.1.3 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = References: <506F239D.6090007@luigirosa.com> <506FE780.9000900@luigirosa.com> <507001B6.2000704@wk-serv.de> Message-ID: <507044D3.3030309@puzzled.xs4all.nl> On 10/06/2012 12:02 PM, Patrick Westenberg wrote: > Hi Luigi, > > with regard to SSL my configuration is much more simple and it works > fine with IPv4 and IPv6. But you have of course to use a hostname > matching the certificates common name. You could add additional hostnames in the certificate by specifying them in SubjectAltName. I use that so my certificate works with both the public FQDN going over the Internet as well as the internal hostname when using a VPN or on the local LAN. Regards, Patrick From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 6 18:53:53 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 6 Oct 2012 17:53:53 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <98D34C84-B1F4-47B3-9145-06E262FC11D7@iki.fi> References: <53B237A0-3A44-47DC-B41A-82CB5D174254@iki.fi> <20120710224947.GA10641@daniel.localdomain> <20120801193209.GA9069@daniel.localdomain> <20120801202502.GA9951@daniel.localdomain> <98D34C84-B1F4-47B3-9145-06E262FC11D7@iki.fi> Message-ID: <20121006155353.GA11391@daniel.localdomain> Hi Timo and list members, Timo Sirainen wrote: > On 1.8.2012, at 23.25, Daniel Parthey wrote: > > >> The error is still the same "config permission denied" shown above? I found that also from my server and added a debug patch, but it hasn't crashed yet. Could you try the attached patch and getting a gdb backtrace from the resulting core file? (Or at least the raw backtrace - getting a core file might be tricky.) > > > > Running command on a four host setup with mailbox+director instance each: > > /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A > > > > dovecot: doveadm(username at example.org): Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(vmail) egid=123(vmail) missing +r perm: /var/run/dovecot/auth-userdb, we're not in group 122(dovecot), dir owned by 0:0 mode=0755) > > Ah, so the original patch helped! This is a different error. > > > srw-rw---- 1 dovecot dovecot 0 2012-07-11 18:35 auth-userdb > > Simplest solution now would be to make this world-rw, see the auth-userdb socket configuration in http://wiki2.dovecot.org/LDA#Virtual_users > > But I guess this should also be fixed by doveadm-server. Although I don't think this should be happening by default anyway. Maybe this is also solved by the http://hg.dovecot.org/dovecot-2.1/rev/476381017ec7 patch? I finally found time to update from 2.1.8 to 2.1.10 and change service auth-user db socket to default mode of 0666. Unfortunately, the issue is still not solved and I did not manage to get a gdb backtrace, since it does not crash or assert. Current configuration of both mailbox and director is attached. The error "Permission denied" from the mailbox logs is gone, but the director doveadm command: /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A still throws the error message: doveadm(username at example.org): Error: doveadm server failure doveadm: Error: Failed to iterate through some users Any idea what I could do in addition to making /var/run/dovecot/auth-userdb world-rw? Regards Daniel -- https://plus.google.com/103021802792276734820 -------------- next part -------------- # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-43-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = imapc_features = rfc822.size imapc_host = local-mailbox imapc_port = 18143 instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota stats mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota stats imap_quota imap_stats } protocol lmtp { mail_plugins = quota stats sieve } -------------- next part -------------- # 2.1.10: /etc/dovecot-director/dovecot-director.conf # OS: Linux 2.6.32-43-server x86_64 Ubuntu 10.04.4 LTS auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot-director deliver_log_format = director: deliver: msgid=%m from=%f: %$ director_doveadm_port = 20000 director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_user_expire = 2 days disable_plaintext_auth = no doveadm_password = doveadm_proxy_port = 19000 instance_name = dovecot-director lmtp_proxy = yes login_greeting = Mail Balancer login_log_format = director: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "director: mail: %s(%u): " mail_max_userip_connections = 20 mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /etc/dovecot-director/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { executable = doveadm-server director inet_listener doveadm-server { port = 20000 } } service imap-login { executable = imap-login director inet_listener imap { port = 20143 } inet_listener imaps { port = 20993 ssl = yes } } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { address = * port = 20024 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 20200 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 20110 } inet_listener pop3s { port = 20995 ssl = yes } } ssl_cert = References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> Message-ID: <1349543117.2086.10.camel@worklian> Hi Mik, > > Also how do you understand this sentence > "# Don't use mmap() at all. This is required if you store indexes to shared > # filesystems (NFS or clustered filesystem) or for some operating systems > # which use a separate cache for mmap, such as OpenBSD. > mmap_disable = yes" > I've read it 10 times, and I don't know if this should be set to yes or no (probably because my english is not perfect). > My operating system is OpenBSD and I don't share NFS or cluster filesystems. Well, your english is monumentally better than my second language (if you could even say I have one), so good on you... I interpret this sentence as an if statement: if [[ (using NFS||Cluster) == true || (using OS w separate cache for mmap, such as OpenBSD) == true ]]; then setting is required (set to yes/true) fi > > Thank you -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca From marc at perkel.com Sat Oct 6 22:08:12 2012 From: marc at perkel.com (Marc Perkel) Date: Sat, 06 Oct 2012 12:08:12 -0700 Subject: [Dovecot] Getting rid of the subscription file In-Reply-To: References: <506D1345.9070406@perkel.com> Message-ID: <5070819C.40104@perkel.com> On 10/4/2012 7:20 AM, Timo Sirainen wrote: > On 4.10.2012, at 7.40, Marc Perkel wrote: > >> I'd like to eliminate the subscription file and have it just list the folders that are there. How do I do that? > Write a plugin. > I have had some luck using an external script to generate the subscriptions files. From p.heinlein at heinlein-support.de Sun Oct 7 00:32:56 2012 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Sat, 06 Oct 2012 23:32:56 +0200 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files Message-ID: <5070A388.8070205@heinlein-support.de> Several times we already had the problems, that accounts with more the 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if vsize_limit of 750 MB is set. In this case, the lmtpd-process haven't been able to allocate more memory to read/write/update the index-files and crashed (and the index-files become corrupted at the end.) [Please -- don't discuss about the need of INBOXes with 1.7 million (unread) e-mails (don't discuss that with ME. Personally, I agree, that there's NO need for that...).] But: We also noticed accounts with ~ 300.000 e-Mails running out of memory in the same situations. This happends, if the subject is very large (subject or some other header attributes). And: We've been able to reproduce out-of-memory-Problems with just 13.000 e-mails with VERY long subjects (e.g.: network monitoring status informations), even with a vsize_limit of 750 MB (which is already very much). 13.000 e-mails isn't very much. And it's easy to inject several thousands of prepared e-mails. Having many mails for accounts with huge (and broken) index-files slows down the delivery rate VERY much and increases the need for memory and cpu resources and I/O very much. So: This could be used for a very easy to do denial-of-service attac against Dovecot-based mailservers. I don't have a clear solution for that, Dovecot needs the subject information in its index files. But it looks like, it isn't a good idea to put the whole subject into the index. Maybe it's better/necessary to use just the first 50-70 characters for that and to keep the rest away from the index? I think I would prefer that even if that means, that accessing those folders with "special" e-mails will become slower because Dovecot has to get those informations directly from the e-mail. This performance issue is just a problem for the user. But crashing lmtpd-processes and lowering the delivery rate is a *real* problem for the whole IMAP-cluster. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From slitt at troubleshooters.com Sun Oct 7 02:44:24 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Sat, 6 Oct 2012 19:44:24 -0400 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <5070A388.8070205@heinlein-support.de> References: <5070A388.8070205@heinlein-support.de> Message-ID: <20121006194424.47f7f80b@mydesk> On Sat, 06 Oct 2012 23:32:56 +0200, Peer Heinlein said: > > Several times we already had the problems, that accounts with more the > 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if > vsize_limit of 750 MB is set. > > In this case, the lmtpd-process haven't been able to allocate more > memory to read/write/update the index-files and crashed (and the > index-files become corrupted at the end.) > > [Please -- don't discuss about the need of INBOXes with 1.7 million > (unread) e-mails (don't discuss that with ME. Personally, I agree, > that there's NO need for that...).] > > But: We also noticed accounts with ~ 300.000 e-Mails running out of > memory in the same situations. This happends, if the subject is very > large (subject or some other header attributes). > > And: We've been able to reproduce out-of-memory-Problems with just > 13.000 e-mails with VERY long subjects (e.g.: network monitoring > status informations), even with a vsize_limit of 750 MB (which is > already very much). > > 13.000 e-mails isn't very much. And it's easy to inject several > thousands of prepared e-mails. > > Having many mails for accounts with huge (and broken) index-files > slows down the delivery rate VERY much and increases the need for > memory and cpu resources and I/O very much. > > So: This could be used for a very easy to do denial-of-service attac > against Dovecot-based mailservers. > > I don't have a clear solution for that, Dovecot needs the subject > information in its index files. But it looks like, it isn't a good > idea to put the whole subject into the index. Maybe it's > better/necessary to use just the first 50-70 characters for that and > to keep the rest away from the index? > > I think I would prefer that even if that means, that accessing those > folders with "special" e-mails will become slower because Dovecot has > to get those informations directly from the e-mail. > > This performance issue is just a problem for the user. > > But crashing lmtpd-processes and lowering the delivery rate is a > *real* problem for the whole IMAP-cluster. > > Peer While the real solution is being decided, can I avoid this possible DOS attack by using procmail to /dev/null anything with more than a 256 byte subject, before it ever gets to Dovecot IMAP? Thanks SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From mikydevel at yahoo.fr Sun Oct 7 12:36:59 2012 From: mikydevel at yahoo.fr (Mik J) Date: Sun, 7 Oct 2012 10:36:59 +0100 (BST) Subject: [Dovecot] Dovecot configuration and question about IP trusted In-Reply-To: <1349543117.2086.10.camel@worklian> References: <1349458245.86274.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349460073.4213.59.camel@worklian> <1349513093.40087.YahooMailNeo@web28803.mail.ir2.yahoo.com> <1349543117.2086.10.camel@worklian> Message-ID: <1349602619.92555.YahooMailNeo@web28801.mail.ir2.yahoo.com> > De?: Bob Miller > Hi Mik, > >> >> Also how do you understand this sentence >> "# Don't use mmap() at all. This is required if you store indexes > to shared >> # filesystems (NFS or clustered filesystem) or for some operating systems >> # which use a separate cache for mmap, such as OpenBSD. >> mmap_disable = yes" >> I've read it 10 times, and I don't know if this should be set to > yes or no (probably because my english is not perfect). >> My operating system is OpenBSD and I don't share NFS or cluster > filesystems. > > Well, your english is monumentally better than my second language (if > you could even say I have one), so good on you... > > I interpret this sentence as an if statement: > > if [[ (using NFS||Cluster) == true || (using OS w separate cache for > mmap, such as OpenBSD) == true ]]; then > ??? setting is required (set to yes/true) > fi Thank you for your answers. Have a nice week end From marc at perkel.com Sun Oct 7 22:47:44 2012 From: marc at perkel.com (Marc Perkel) Date: Sun, 07 Oct 2012 12:47:44 -0700 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? Message-ID: <5071DC60.8090108@perkel.com> Here's what I'm trying to do. I have a spam filtering operation as a front end for other servers. I've created a virtual server for spam storage where the user will be able to log in using squirrelmail/dovecot to review and release their spam. The email is stored in maildir format. Piecing it together I can use squirrelmail to pipe the email into something so that if a use finds a false positive they can hit the "release" button and the message is sent on to the recipient. Squirrelmail sends the message as you would receive it as stored in maildir format. I need to take this format and translate it to send it on to the user. Wondering what is the best way to do that. Something that translated it into bsmtp format would be great. I'm sure thare must be something out there. I just haven't found it yet. Thanks in advance. From robert at schetterer.org Sun Oct 7 23:48:10 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 07 Oct 2012 22:48:10 +0200 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? In-Reply-To: <5071DC60.8090108@perkel.com> References: <5071DC60.8090108@perkel.com> Message-ID: <5071EA8A.3060605@schetterer.org> Am 07.10.2012 21:47, schrieb Marc Perkel: > Here's what I'm trying to do. I have a spam filtering operation as a > front end for other servers. I've created a virtual server for spam > storage where the user will be able to log in using squirrelmail/dovecot > to review and release their spam. The email is stored in maildir format. > > Piecing it together I can use squirrelmail to pipe the email into > something so that if a use finds a false positive they can hit the > "release" button and the message is sent on to the recipient. > Squirrelmail sends the message as you would receive it as stored in > maildir format. > > I need to take this format and translate it to send it on to the user. > > Wondering what is the best way to do that. Something that translated it > into bsmtp format would be great. I'm sure thare must be something out > there. I just haven't found it yet. > > Thanks in advance. > sorry my hard words ,thats enorm complicated the whole idea sounds broken somehow why not reject spam in smtp income level i.e with clamav-milter and sanesecurity antispam signatures and spamass-milter reject all mail tagged over i.e level 15 for the rest ( which will be quite low ) use i.e some quarantaine feature amavis etc if users should manage it ie http://www.maiamailguard.com/maia/wiki/AboutMaia or equal may a good idea and i am quite sure there are some other well done projects out there which doing equal stuff -- Best Regards MfG Robert Schetterer From tss at iki.fi Mon Oct 8 03:11:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 8 Oct 2012 03:11:15 +0300 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <5070A388.8070205@heinlein-support.de> References: <5070A388.8070205@heinlein-support.de> Message-ID: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> On 7.10.2012, at 0.32, Peer Heinlein wrote: > Several times we already had the problems, that accounts with more the > 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if > vsize_limit of 750 MB is set. > > In this case, the lmtpd-process haven't been able to allocate more > memory to read/write/update the index-files and crashed (and the > index-files become corrupted at the end.) I don't think dovecot.index file is much of a problem. With 1M mails it usually only takes something like 8-32 MB of memory depending on what mailbox format is used. dovecot.index.log file doesn't depend on the mailbox size at all. The main problem is dovecot.index.cache file. I've thought about the cache file problems earlier also, but it's a bit difficult to figure out the best solution for it. And since nobody had actually complained about it, I hadn't really done anything about it. Also I hadn't previously thought of LMTP/LDA processes crashing because of it, that's a bigger problem than IMAP process crashing. Although I think you're getting a lot more of "mmap(dovecot.index.cache) failed: Out of memory" errors than crashes for large mailboxes? So, subproblems related to this: 1. Filling out dovecot.index.cache too easily. A rather simple possibility that would catch all the possible ways would be to limit the max. size of a single message's cache entry to X kilobytes (64?). If it becomes larger, it's simply not written to the cache file. 2. Filling out memory too easily. If a long header is wanted to be cached or used for other purposes (e.g. Message-ID), it's still fully read into memory. Add some reasonable limit to max. length of a single header. Can't be too small, because some headers are legitimately pretty long (DKIM and such). Maybe something like 10kB would be safe enough for everyone? 3. If existing dovecot.index.cache is larger than X MB, shrink it first below X. Shrinking could begin with trying to do it the nice way of removing only unneeded data, but if that fails it could forcibly just remove some old messages. The X would have to be related to the process's VSZ limit. 4. Dovecot currently doesn't close index files immediately when mailbox is closed, because it's thinking that IMAP clients might reopen the index soon anyway. Max 3 indexes can be kept open, so 3x already different very large indexes can be too much. I'm not sure if this is actually useful at all. Maybe I should disable it for LMTP, or maybe just remove it completely. The 3. part is what I like changing the least. An alternative solution would be to just not map the entire cache file into memory all at once. The code was actually originally designed to do just that, but munmap()ing + mmap()ing again wasn't very efficient. But for LMTP there's really no need to map the whole file. All it really wants is to read a couple of header records and then append to the file. Maybe it could use an alternative code path that would simply do that instead of mmap()ing anything. It wouldn't solve it for IMAP though. > I don't have a clear solution for that, Dovecot needs the subject > information in its index files. But it looks like, it isn't a good idea > to put the whole subject into the index. Maybe it's better/necessary to > use just the first 50-70 characters for that and to keep the rest away > from the index? 50-70 is way too little. The cached subject gets sent to the IMAP client. I think 200 bytes would be minimum and 1000 would be something I could probably even hardcode. But anyway, subject isn't the only way to trigger this and 1000 bytes is too low for some headers. From tss at iki.fi Mon Oct 8 04:45:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 08 Oct 2012 04:45:14 +0300 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> References: <5070A388.8070205@heinlein-support.de> <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> Message-ID: <1349660714.13571.75.camel@hurina> On Mon, 2012-10-08 at 03:11 +0300, Timo Sirainen wrote: > The 3. part is what I like changing the least. An alternative solution > would be to just not map the entire cache file into memory all at > once. The code was actually originally designed to do just that, but > munmap()ing + mmap()ing again wasn't very efficient. But for LMTP > there's really no need to map the whole file. All it really wants is > to read a couple of header records and then append to the file. Maybe > it could use an alternative code path that would simply do that > instead of mmap()ing anything. It wouldn't solve it for IMAP though. Attached patch changes LMTP/LDA to not mmap() the target mailbox's cache file. I did a few quick tests and it seems to work. I'll probably commit it to 2.1 hg after some more tests. -------------- next part -------------- A non-text attachment was scrubbed... Name: index-saveonly.diff Type: text/x-patch Size: 17995 bytes Desc: not available URL: From stan at hardwarefreak.com Mon Oct 8 09:11:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 08 Oct 2012 01:11:50 -0500 Subject: [Dovecot] [OT] How do I convert maildir to bsmtp format? In-Reply-To: <5071DC60.8090108@perkel.com> References: <5071DC60.8090108@perkel.com> Message-ID: <50726EA6.4030702@hardwarefreak.com> This request for assistance is a train wreck, with cars strewn everywhere, chaos ensuing, the carnage preventing everyone from being able to see what's actually going on... On 10/7/2012 2:47 PM, Marc Perkel wrote: > Here's what I'm trying to do. I have a spam filtering operation as a > front end for other servers. I've created a virtual server for spam > storage where the user will be able to log in using squirrelmail/dovecot > to review and release their spam. The email is stored in maildir format. So you're trying to somewhat duplicate the functionality of a Barracuda or other AS gateway appliance, with vanilla SM and Dovecot with very little modification. Good luck with this. You're going to need to write a pretty complex shell or perl script to do most of the work, and call it from the SM "release button" routine. > Piecing it together I can use squirrelmail to pipe the email into > something so that if a use finds a false positive they can hit the > "release" button and the message is sent on to the recipient. > Squirrelmail sends the message as you would receive it as stored in > maildir format. SM only sends messages via SMTP submitted to the SMTP relay host specified in the config file, or via dropping to the local MTA via stdin/out. Maildir is a mail storage directory and file format protocol, not a transmission protocol. It's physically impossible to "send" a msg in maildir format, or mbox, or dbox, etc. The format of the SMTP headers and message body is plain text, possibly with MIME encoding. So what you really mean is you need a way to read a maildir message file, scrape the recipient address, strip all of the AS headers you've inserted, drop this 'new' message to the MTA, which sends the message to the recipient. When it arrives it appears never to have been molested by your AS software, with only the proper headers and original body. Then your script needs to check the log for successful delivery (250), then send the proper commands to dovecot to log into the account as the user and delete the message. I can't begin to estimate the amount of coding and testing required here, but it will be high. > I need to take this format and translate it to send it on to the user. Translate it? I've never used maildir, but I can't imagine the on disk message file contents need "translating". See above. > Wondering what is the best way to do that. Something that translated it > into bsmtp format would be great. I'm sure thare must be something out > there. I just haven't found it yet. BSMTP isn't a file format. It's a simple Mail User Agent with some unique capabilities. Given you already have an MTA on the host, why would you need BSMTP to submit or deliver the msg? You simply need to learn the proper commands to submit mail to your local MTA. With Postfix you'd use postdrop or the sendmail compatibility command which use stdin to read the message. My no BS assessment and recommendation are that you're likely in way over your head here (and don't realize it yet), and you need to focus your efforts on locating an integrated FOSS solution, or even a commercial solution, to fit your needs. I seem to understand better than you what is required to pull this off, and I wouldn't touch this project with a 50 ft pole. I simply don't have the coding skills for this. -- Stan From stan at hardwarefreak.com Mon Oct 8 10:44:57 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 08 Oct 2012 02:44:57 -0500 Subject: [Dovecot] Large subjects increase memory-usage and enlarge index-files In-Reply-To: <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> References: <5070A388.8070205@heinlein-support.de> <6D4D2F75-E58D-497F-90D2-1A3EF5FBC225@iki.fi> Message-ID: <50728479.6070003@hardwarefreak.com> On 10/7/2012 7:11 PM, Timo Sirainen wrote: > I don't think dovecot.index file is much of a problem. With 1M mails it usually only takes something like 8-32 MB of memory depending on what mailbox format is used. dovecot.index.log file doesn't depend on the mailbox size at all. The main problem is dovecot.index.cache file. > > I've thought about the cache file problems earlier also, but it's a bit difficult to figure out the best solution for it. And since nobody had actually complained about it, I hadn't really done anything about it. Also I hadn't previously thought of LMTP/LDA processes crashing because of it, that's a bigger problem than IMAP process crashing. Although I think you're getting a lot more of "mmap(dovecot.index.cache) failed: Out of memory" errors than crashes for large mailboxes? > > So, subproblems related to this: > > 1. Filling out dovecot.index.cache too easily. A rather simple possibility that would catch all the possible ways would be to limit the max. size of a single message's cache entry to X kilobytes (64?). If it becomes larger, it's simply not written to the cache file. > > 2. Filling out memory too easily. If a long header is wanted to be cached or used for other purposes (e.g. Message-ID), it's still fully read into memory. Add some reasonable limit to max. length of a single header. Can't be too small, because some headers are legitimately pretty long (DKIM and such). Maybe something like 10kB would be safe enough for everyone? > > 3. If existing dovecot.index.cache is larger than X MB, shrink it first below X. Shrinking could begin with trying to do it the nice way of removing only unneeded data, but if that fails it could forcibly just remove some old messages. The X would have to be related to the process's VSZ limit. > > 4. Dovecot currently doesn't close index files immediately when mailbox is closed, because it's thinking that IMAP clients might reopen the index soon anyway. Max 3 indexes can be kept open, so 3x already different very large indexes can be too much. I'm not sure if this is actually useful at all. Maybe I should disable it for LMTP, or maybe just remove it completely. > > The 3. part is what I like changing the least. An alternative solution would be to just not map the entire cache file into memory all at once. The code was actually originally designed to do just that, but munmap()ing + mmap()ing again wasn't very efficient. But for LMTP there's really no need to map the whole file. All it really wants is to read a couple of header records and then append to the file. Maybe it could use an alternative code path that would simply do that instead of mmap()ing anything. It wouldn't solve it for IMAP though. > 50-70 is way too little. The cached subject gets sent to the IMAP client. I think 200 bytes would be minimum and 1000 would be something I could probably even hardcode. But anyway, subject isn't the only way to trigger this and 1000 bytes is too low for some headers. Nearly all mail servers have two resources of interest here in great excess: CPU cycles, and cache/RAM b/w, due to multicore CPUs and 2-4 memory channels per socket. The two bottlenecks are IO bandwidth/latency, and, for many, RAM capacity. So let's take advantage of both the strengths and weaknesses of our hardware to possibly address the above issue. What happens if we insert a subroutine to compress/decompress each field in the cache array files individually, in real time? You should still be able to mmap the files. The individual array fields and total cache file sizes would be much smaller on disk and in memory. Any cache file contents mapped to memory, that aren't currently being used, are stored compressed in memory, directly addressing the problem in this thread. When a field is needed we decompress it on the fly after reading it from memory. This should be very fast as the fields are relatively small. When it's written out we compress on the fly. With each field stored compressed on disk, not only is file size decreased, but more importantly, each read/write moves more data per physical IO. So not only are increasing storage capacity, we're also decreasing IOPS. It would be preferable to do this de/compression in kernel rather than user space, but I don't think that's a real option. However, libz and libbz2 are pretty fast and small, and the code easily fits in CPU cache. Combined with the massive L1/L2/L3 and RAM b/w of modern systems, execution in user space should still be very fast, and not noticeably degrade performance. I'm not a programmer, so I have no idea if this is even plausible, or possible. But if it is, it seems worth exploring, as it would seem to benefit Dovecot performance in multiples areas, and possibly solve this, and other current/future memory capacity and/or performance related problems. -- Stan From benedetto.vassallo at unipa.it Mon Oct 8 15:44:05 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Mon, 08 Oct 2012 14:44:05 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <506DA526.4020606@Media-Brokers.com> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <506DA526.4020606@Media-Brokers.com> Message-ID: <20121008144405.21350fs5aq2vwl91@webmail.unipa.it> Def. Quota Charles Marcus : > On 2012-10-04 9:00 AM, Benedetto Vassallo > wrote: >> Hello list, >> Excuse me for my poor english. >> I have updated on a test server dovecot 2.0.13 to dovecot 2.1.1. > > If you are going to update, why ohy why update to an outdated version? > > Current version is 2.1.10... LOTS of bug fixes for the 2.1.x line... > Hello, I upgraded to 2.1.10 but still dont't have hardlinks working. May I have something else to do to make them working? Thanks. -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From tibby at tibby.hu Mon Oct 8 16:49:33 2012 From: tibby at tibby.hu (Tibby) Date: Mon, 8 Oct 2012 15:49:33 +0200 (CEST) Subject: [Dovecot] another mysql quota problem Message-ID: <1182860249.2592.1349704173895.JavaMail.root@tibby.hu> Hello! I am using dovecot 1.2.15 on Debian 6 I have Postfix configured with mysql, and also dovecot is auth-ing form mysql users table. The users table has username password quota stored. When I set qouta to whatever number, it doesnt even show in dovecot. telnet localhost 143 a login username at domain.tld password a getquotaroot inbox * QUOTAROOT "INBOX" a OK Getquotaroot completed. Basicly it doesn't show any kind of quota. in the /etc/dovecot/dovecot-sql.conf: driver = mysql connect = host=127.0.0.1 dbname=mail user=USERNAME password=PASSWORD default_pass_scheme = CRYPT password_query = SELECT email as user, password FROM users WHERE email='%u'; user_query = SELECT CONCAT(('/home/vmail/'), SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1),'/') AS home, 5000 AS uid, 5000 AS gid, CONCAT('maildir:storage=', floor(quota/1024)) AS quota FROM users WHERE email='%u'; Why it doesn't get the quota out from the mail DB's users table? From roundcube222 at alaadin.org Mon Oct 8 18:12:58 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Mon, 08 Oct 2012 18:12:58 +0300 Subject: [Dovecot] Dovecot Hangs while mutile users download mail for same account using pop3 Message-ID: <793760c2702e89acc526a66c0b543293@Coptics.org> Hello, I have a weird problem in dovecot, Dovecot Hangs while multiple users download mail for same account using pop3 Three persons use 1 same email , and three of them use outlook express to check That specific mail .. Also some times one of the three users check the mail for this Account using imap (squirrel mail) And here comes the problem. suddenly all the three users are unable to check the mail And when the outlook express connect to the pop3 server, they are unable to disconnect And the outlook express keep asking "the server is not responding for 60 mins would you like to wait ? i checked /var/log/maillog, i can see the following errors dovecot: pop3(sales): Disconnected: Storage error during logout. to=, orig_to=, relay=local, delay=357, delays=338/0.01/0/19, dsn=4.2.0, status=deferred (cannot update mailbox /var/mail/sales for user sales. unable to lock for exclusive access: Resource temporarily unavailable) When i used lsof | grep sales pop3 4278 sales 10u REG 8,3 22897673 12615705 /var/spool/mail/sales i went to /home/sales/mail/.imap/INBOX , but i didnot find any lock files -rw------- 1 sales sales 944 Oct 7 13:35 dovecot.index -rw------- 1 sales sales 49152 Oct 7 16:57 dovecot.index.cache -rw------- 1 sales sales 14044 Oct 7 16:57 dovecot.index.log 1- So what is exactly the problem? is the problem that three users are checking the same exact mail ? 2- is the problem that 2 users checking mail using pop3 and the third one checking it using imap insame time ? 3- is dovecot locking the inbox file while one of the users getting the mail ? 4- What is the cause of the problem ? Please help Regards Robert JR From marc at perkel.com Mon Oct 8 19:23:27 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 09:23:27 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? Message-ID: <5072FDFF.8050402@perkel.com> I'm trying to grasp the namespace and prefix stuff and looking for a good tutorial page that explains what it is and how to use it. Thanks in advance From robert at schetterer.org Mon Oct 8 20:59:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 19:59:34 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5072FDFF.8050402@perkel.com> References: <5072FDFF.8050402@perkel.com> Message-ID: <50731486.4080003@schetterer.org> Am 08.10.2012 18:23, schrieb Marc Perkel: > I'm trying to grasp the namespace and prefix stuff and looking for a > good tutorial page that explains what it is and how to use it. > > Thanks in advance > look http://wiki2.dovecot.org/Namespaces -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:09:00 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:09:00 -0700 Subject: [Dovecot] Master User Question Message-ID: <507316BC.1050008@perkel.com> OK - I'm trying to do weird stuff so rather not answer why I'm doing this. Trying to log in using a master user: user at example.com*master at master.com When debugging the master authenticates - but then it tries to authenticate user at example.com and it's not found. And - it is true that the user doesn't actually exist. What I want to do is allow it to log in without checking if the user exists, just on the credentials of the master. I'm not quite understanding what the login attempt for user at example.com is trying to do. Is it looking for more necessary information? Hope I asked this clearly enough. Thanks in advance. From marc at perkel.com Mon Oct 8 21:10:21 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:10:21 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <50731486.4080003@schetterer.org> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> Message-ID: <5073170D.2040508@perkel.com> On 10/8/2012 10:59 AM, Robert Schetterer wrote: > Am 08.10.2012 18:23, schrieb Marc Perkel: >> I'm trying to grasp the namespace and prefix stuff and looking for a >> good tutorial page that explains what it is and how to use it. >> >> Thanks in advance >> > look > > http://wiki2.dovecot.org/Namespaces > I've read that but it doesn't tell me what a prefix is or what a namespace is. I'm having a hard time grasping the overall concept. From robert at schetterer.org Mon Oct 8 21:26:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:26:14 +0200 Subject: [Dovecot] Master User Question In-Reply-To: <507316BC.1050008@perkel.com> References: <507316BC.1050008@perkel.com> Message-ID: <50731AC6.4060707@schetterer.org> Am 08.10.2012 20:09, schrieb Marc Perkel: > OK - I'm trying to do weird stuff so rather not answer why I'm doing this. > > Trying to log in using a master user: > > user at example.com*master at master.com > > When debugging the master authenticates - but then it tries to > authenticate user at example.com and it's not found. And - it is true that > the user doesn't actually exist. > > What I want to do is allow it to log in without checking if the user > exists, just on the credentials of the master. i dont understand why login should work with masteruser to user , when user does not exist, would you like to get the user created on the fly by masteruser login etc ? > > I'm not quite understanding what the login attempt for user at example.com > is trying to do. Is it looking for more necessary information? > > Hope I asked this clearly enough. Thanks in advance. > > you may manipulate your master query in some magic way http://wiki2.dovecot.org/Authentication/MasterUsers -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Oct 8 21:31:52 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:31:52 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5073170D.2040508@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> Message-ID: <50731C18.5060009@schetterer.org> Am 08.10.2012 20:10, schrieb Marc Perkel: > > On 10/8/2012 10:59 AM, Robert Schetterer wrote: >> Am 08.10.2012 18:23, schrieb Marc Perkel: >>> I'm trying to grasp the namespace and prefix stuff and looking for a >>> good tutorial page that explains what it is and how to use it. >>> >>> Thanks in advance >>> >> look >> >> http://wiki2.dovecot.org/Namespaces >> > > I've read that but it doesn't tell me what a prefix is or what a > namespace is. I'm having a hard time grasping the overall concept. > lookink to this http://www.ietf.org/rfc/rfc2342.txt ? -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:32:51 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:32:51 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731AC6.4060707@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> Message-ID: <50731C53.2050309@perkel.com> On 10/8/2012 11:26 AM, Robert Schetterer wrote: > Am 08.10.2012 20:09, schrieb Marc Perkel: >> OK - I'm trying to do weird stuff so rather not answer why I'm doing this. >> >> Trying to log in using a master user: >> >> user at example.com*master at master.com >> >> When debugging the master authenticates - but then it tries to >> authenticate user at example.com and it's not found. And - it is true that >> the user doesn't actually exist. >> >> What I want to do is allow it to log in without checking if the user >> exists, just on the credentials of the master. > i dont understand why login should work with masteruser to user , when > user does not exist, would you like to get the user created on the fly > by masteruser login etc ? The directory structure for user at example.com exists. Just not in the regular mysql database. If I create a fake passdb system it works: passdb { driver = static args = nopassword=y } However - it leaves the system wide open for anyone. Basically - only master users are going to log in and manage stored spam for regular users. The master user will be able to spoof being the regular user to review stored spam. From slusarz at curecanti.org Mon Oct 8 21:28:18 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 08 Oct 2012 12:28:18 -0600 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <5073170D.2040508@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> Message-ID: <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> Quoting Marc Perkel : > On 10/8/2012 10:59 AM, Robert Schetterer wrote: >> Am 08.10.2012 18:23, schrieb Marc Perkel: >>> I'm trying to grasp the namespace and prefix stuff and looking for a >>> good tutorial page that explains what it is and how to use it. >>> >>> Thanks in advance >>> >> look >> >> http://wiki2.dovecot.org/Namespaces >> > > I've read that but it doesn't tell me what a prefix is or what a > namespace is. I'm having a hard time grasping the overall concept. Try reading the defining RFC document itself: http://tools.ietf.org/html/rfc2342 michael From marc at perkel.com Mon Oct 8 21:42:02 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:42:02 -0700 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> Message-ID: <50731E7A.9020001@perkel.com> On 10/8/2012 11:28 AM, Michael M Slusarz wrote: > Quoting Marc Perkel : > >> On 10/8/2012 10:59 AM, Robert Schetterer wrote: >>> Am 08.10.2012 18:23, schrieb Marc Perkel: >>>> I'm trying to grasp the namespace and prefix stuff and looking for a >>>> good tutorial page that explains what it is and how to use it. >>>> >>>> Thanks in advance >>>> >>> look >>> >>> http://wiki2.dovecot.org/Namespaces >>> >> >> I've read that but it doesn't tell me what a prefix is or what a >> namespace is. I'm having a hard time grasping the overall concept. > > Try reading the defining RFC document itself: > > http://tools.ietf.org/html/rfc2342 > > michael > I went there and maybe I'm just stupid. I don't understand what a prefix is. I'm trying to grasp the whole process. From robert at schetterer.org Mon Oct 8 21:43:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:43:20 +0200 Subject: [Dovecot] Master User Question In-Reply-To: <50731C53.2050309@perkel.com> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> Message-ID: <50731EC8.6010101@schetterer.org> Am 08.10.2012 20:32, schrieb Marc Perkel: > > On 10/8/2012 11:26 AM, Robert Schetterer wrote: >> Am 08.10.2012 20:09, schrieb Marc Perkel: >>> OK - I'm trying to do weird stuff so rather not answer why I'm doing >>> this. >>> >>> Trying to log in using a master user: >>> >>> user at example.com*master at master.com >>> >>> When debugging the master authenticates - but then it tries to >>> authenticate user at example.com and it's not found. And - it is true that >>> the user doesn't actually exist. >>> >>> What I want to do is allow it to log in without checking if the user >>> exists, just on the credentials of the master. >> i dont understand why login should work with masteruser to user , when >> user does not exist, would you like to get the user created on the fly >> by masteruser login etc ? > > The directory structure for user at example.com exists. Just not in the > regular mysql database. If I create a fake passdb system it works: > > passdb { > driver = static > args = nopassword=y > } > > However - it leaves the system wide open for anyone. Basically - only > master users are going to log in and manage stored spam for regular > users. The master user will be able to spoof being the regular user to > review stored spam. > > whats the problem in simply create the user mailbox? i.e lda etc may do it on the fly be recieving a mail http://wiki2.dovecot.org/LDA -m : Destination mailbox (default is INBOX). If the mailbox doesn't exist, it will not be created (unless the lda_mailbox_autocreate setting is set to yes). If message couldn't be saved to the mailbox for any reason, it's delivered to INBOX instead. -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Oct 8 21:49:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 08 Oct 2012 20:49:33 +0200 Subject: [Dovecot] Namespace Prefix Tutorial? In-Reply-To: <50731E7A.9020001@perkel.com> References: <5072FDFF.8050402@perkel.com> <50731486.4080003@schetterer.org> <5073170D.2040508@perkel.com> <20121008122818.Horde.N31-J4F5lbhQcxtCfziA2eA@bigworm.curecanti.org> <50731E7A.9020001@perkel.com> Message-ID: <5073203D.7060801@schetterer.org> Am 08.10.2012 20:42, schrieb Marc Perkel: > > On 10/8/2012 11:28 AM, Michael M Slusarz wrote: >> Quoting Marc Perkel : >> >>> On 10/8/2012 10:59 AM, Robert Schetterer wrote: >>>> Am 08.10.2012 18:23, schrieb Marc Perkel: >>>>> I'm trying to grasp the namespace and prefix stuff and looking for a >>>>> good tutorial page that explains what it is and how to use it. >>>>> >>>>> Thanks in advance >>>>> >>>> look >>>> >>>> http://wiki2.dovecot.org/Namespaces >>>> >>> >>> I've read that but it doesn't tell me what a prefix is or what a >>> namespace is. I'm having a hard time grasping the overall concept. >> >> Try reading the defining RFC document itself: >> >> http://tools.ietf.org/html/rfc2342 >> >> michael >> > > I went there and maybe I'm just stupid. I don't understand what a prefix > is. I'm trying to grasp the whole process. > > try http://wiki.dovecot.org/Design/Storage/MailNamespace -- Best Regards MfG Robert Schetterer From marc at perkel.com Mon Oct 8 21:52:45 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 11:52:45 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731EC8.6010101@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> <50731EC8.6010101@schetterer.org> Message-ID: <507320FD.2000201@perkel.com> On 10/8/2012 11:43 AM, Robert Schetterer wrote: > Am 08.10.2012 20:32, schrieb Marc Perkel: >> On 10/8/2012 11:26 AM, Robert Schetterer wrote: >>> Am 08.10.2012 20:09, schrieb Marc Perkel: >>>> OK - I'm trying to do weird stuff so rather not answer why I'm doing >>>> this. >>>> >>>> Trying to log in using a master user: >>>> >>>> user at example.com*master at master.com >>>> >>>> When debugging the master authenticates - but then it tries to >>>> authenticate user at example.com and it's not found. And - it is true that >>>> the user doesn't actually exist. >>>> >>>> What I want to do is allow it to log in without checking if the user >>>> exists, just on the credentials of the master. >>> i dont understand why login should work with masteruser to user , when >>> user does not exist, would you like to get the user created on the fly >>> by masteruser login etc ? >> The directory structure for user at example.com exists. Just not in the >> regular mysql database. If I create a fake passdb system it works: >> >> passdb { >> driver = static >> args = nopassword=y >> } >> >> However - it leaves the system wide open for anyone. Basically - only >> master users are going to log in and manage stored spam for regular >> users. The master user will be able to spoof being the regular user to >> review stored spam. >> >> > whats the problem in simply create the user mailbox? > i.e lda etc may do it on the fly be recieving a mail > > http://wiki2.dovecot.org/LDA > -m : Destination mailbox (default is INBOX). If the mailbox > doesn't exist, it will not be created (unless the lda_mailbox_autocreate > setting is set to yes). If message couldn't be saved to the mailbox for > any reason, it's delivered to INBOX instead. Maybe there's some way to have more than one mail_location? I have this - and it's confusing: maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs the /fakedir directory is externally built and it has symlinks pointing onto the /email directory where the email is stored. If I could set up a passdb and specify a different mail_location that might solve my problem, at least one of them. So close - yet so far away. From marc at perkel.com Mon Oct 8 22:07:28 2012 From: marc at perkel.com (Marc Perkel) Date: Mon, 08 Oct 2012 12:07:28 -0700 Subject: [Dovecot] Master User Question In-Reply-To: <50731EC8.6010101@schetterer.org> References: <507316BC.1050008@perkel.com> <50731AC6.4060707@schetterer.org> <50731C53.2050309@perkel.com> <50731EC8.6010101@schetterer.org> Message-ID: <50732470.300@perkel.com> Making some progress = this seems to work. passdb { driver = static args = mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } From calestyo at scientia.net Tue Oct 9 00:37:19 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 08 Oct 2012 23:37:19 +0200 Subject: [Dovecot] some questions on dovecot or rather a mail system setup Message-ID: <1349732239.3344.62.camel@fermat.scientia.net> Hi folks. Perhaps you find some time to look into this,... if you think I should better direct this do some dovecot mailing list,.. just tell :) I'm trying to plan my mail system and would have some questions. The overall idea is about the following: - There is a (internet) server, which is the MTA (which will be postfix) and imap and/or pop3 server (which shall be, guess, dovecot!). - Any spam filtering, virus-scanning, mail filtering happens on that server. - I have not yet decided on whether to use maildrop for this or Sieve. Maildrop is nice, but has one major deficiency which I don't know how to work around. - A few clients (that means I won't serve 100 of users) connect to that server via imap and should see all mails, etc. already in some fancy sorted hierarchy (that means filtered into different directories). So far nothing complicated. But now... - Apart from spam, I never delete mail; and because I'm subscribed to many lists, I get a lot of mail. - Storage on my server is limited and it's located somewhere at my ISP, so I generally do not trust it with respect to safety... For both reasons, I want the canonical archive of all mail to be at home at some local server. - The local server should also be an imap server, so that I can access the archive from may computers at home. - The local server won't be available from the internet. - The local archive should have the same folder hierarchy as the internet server (I'd prefer not to filter twice). Mail Flow: - I generally want to have _all_ mail (which is not sorted out because of being spam) to be archived at the local server. - But(!) I want to selectively keep (in addition) mail at the internet server. For example I may want to select the folder that contains all mail form some friend to be kept online completely. But I may want to decide that mailinglists keep only the last 10 days and/or 1000 messages of mail. - The idea is, that the local server regularly (when it is online/running) catches new mail from the internet server... and stores it in the archive. - So apart from new mail that has not yet been read, that local archive contains always all mails that are also on the internet server... the later may contain (for specific directories) the same, or just parts of. - The MUAs will then have two imap accounts, one to the internet server and one to the local archive,... each one being usable, depending on where I am. 1) This is where my first problem arises: How can I implement that mail flow, especially: - How do I secure that all mail is read from the internet server (i.e. that nothing is "forgotten"? - How do I make sure that no mails are retrieved twice (or more)? A problem which I often had with pop, when the mail client crashed during sync? - Further it must be secured, that when I delete something on the internet server, it is NOT deleted on the local server (on the next mail-fetching).... this is why I don't use the word "sync". a) One stupid solution would be, that I duplicate all mail on the online server,... one part is for staying online, one part is for being fetched to the local archive. As soon as it was fetched... that copy gets removed (always). That solution would give a clean and secured separation of both? b) I don't think offlineimap or any other caching-like solution is the right thing... especially as one must always fear that such a cache may be accidentally wiped. Are there better solutions than (a)? 2) Problem would be already a refinement of a working solution for (1) (but obviously not when using (1).(a) ). When e.g. reply to or forward a mail using the online server,... and that mail had already been fetched,... can I make the flag synced? 3) Is dovecot suitable for the local server? - Ideally of course, I would use dovecot there, too... because that would mean one piece of software less to understand. - I couldn't sue maildir locally, because I loose just to much space to the block fragmentation. - I'd prefer not to use dbox (the thing that the indices are crucial scares me a bit off). a) When using mbox... is dovecot able to manage a really big folder hierarchy that basically ever keeps growing... with easily several 100k mails per folder... and that is in total already over 100GB? - I would prefer to have fast full text search. Does dovecot provide this? I was looking into database backed mail systems (again,... just for the local archive)... namely dbmail and archiveopteryx (are there other open source solutions?)... Not sure which of the two... or whether it's a good idea at all. I remember some dovecot wiki page that showed a comparison which said that both do not perfectly implement imap. Any suggestions with respect to that? Or is there even some SQL backend planned for dovecot? Thanks a lot, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From robert at schetterer.org Tue Oct 9 09:51:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 09 Oct 2012 08:51:18 +0200 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <1349732239.3344.62.camel@fermat.scientia.net> References: <1349732239.3344.62.camel@fermat.scientia.net> Message-ID: <5073C966.6030609@schetterer.org> Am 08.10.2012 23:37, schrieb Christoph Anton Mitterer: > > > - Apart from spam, I never delete mail; and because I'm subscribed to > many lists, I get a lot of mail. > > - Storage on my server is limited and it's located somewhere at my ISP, > so I generally do not trust it with respect to safety... > For both reasons, I want the canonical archive of all mail to be at home > at some local server. sorry you questioned very complex, try to ask more simple there are many tools which may help you bcc_copy with postfix imapsync rsync dsync getmail you may use filters too like sieve, maildrop , procmail etc at the end that should solve nearly all what you might goal its not that much a dovecot question, it more depends if you find that general layout which fits best to your ideas however there is no magical imap/pop3 server more flexibel to configure then dovecot, if your ideas dont work with it, your ideas are broken -- Best Regards MfG Robert Schetterer From stan at hardwarefreak.com Tue Oct 9 10:57:43 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 09 Oct 2012 02:57:43 -0500 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <1349732239.3344.62.camel@fermat.scientia.net> References: <1349732239.3344.62.camel@fermat.scientia.net> Message-ID: <5073D8F7.1060802@hardwarefreak.com> On 10/8/2012 4:37 PM, Christoph Anton Mitterer wrote: The proper way to accomplish your goals, or at least the big ones. > - I generally want to have _all_ mail (which is not sorted out because > of being spam) to be archived at the local server. http://www.postfix.org/postconf.5.html#always_bcc > - But(!) I want to selectively keep (in addition) mail at the internet > server. > For example I may want to select the folder that contains all mail form > some friend to be kept online completely. See above. > But I may want to decide that mailinglists keep only the last 10 days > and/or 1000 messages of mail. http://wiki2.dovecot.org/Plugins/Expire Does age based deletion, but not folder message count based. You must use your MUA, TBird, for the latter. It's far easier to configure this in TBird than in Dovecot config files. You seem like the type who wants flexibility so you can change things often, so use TBird to be happy here. > - The idea is, that the local server regularly (when it is > online/running) catches new mail from the internet server... and stores > it in the archive. This is not an option. The system must be up and connected to the internet 24x7x365. It must have an MX record associated and a valid domain, or a VPN tunnel and entries in both systems hosts files, along with a Postfix transport table, and other tweaks. http://www.postfix.org/transport.5.html If you refuse to run this "local server" 24x7x365 then you will have to use a fetchmail based solution, which will not work well, and whose configuration will prompt you to kill yourself. I cannot help you with any of that. > - So apart from new mail that has not yet been read, that local archive > contains always all mails that are also on the internet server... the > later may contain (for specific directories) the same, or just parts of. No. Mail arriving at the colo/VPS host is immediately sent to the always_bcc address, an address and mailbox on your home server. You will create a duplicate IMAP folder structure on the home server by hand in your MUA. Once this is completed you will write individual user sieve scripts that sort the mail into folder just as it is sorted on the colo/VPS server. Basically, home server Dovecot IMAP config is identical in structure to colo/VPS setup, only the mailbox account names differ. Folder tree, folders, sieve scripts identical, retention policy different. > - The MUAs will then have two imap accounts, one to the internet server > and one to the local archive,... each one being usable, depending on > where I am. Yep. > > > 1) This is where my first problem arises: > How can I implement that mail flow, especially: > - How do I secure that all mail is read from the internet server (i.e. > that nothing is "forgotten"? Done: always_bcc > - How do I make sure that no mails are retrieved twice (or more)? A > problem which I often had with pop, when the mail client crashed during > sync? Done: always_bcc > - Further it must be secured, that when I delete something on the > internet server, it is NOT deleted on the local server (on the next > mail-fetching).... this is why I don't use the word "sync". Done: always_bcc > a) One stupid solution would be, that I duplicate all mail on the online > server,... one part is for staying online, one part is for being fetched > to the local archive. Done: always_bcc And yes that is stupid. > As soon as it was fetched... that copy gets removed (always). > That solution would give a clean and secured separation of both? > b) I don't think offlineimap or any other caching-like solution is the > right thing... especially as one must always fear that such a cache may > be accidentally wiped. > > Are there better solutions than (a)? Yes. Already done: always_bcc > 2) Problem would be already a refinement of a working solution for (1) > (but obviously not when using (1).(a) ). > When e.g. reply to or forward a mail using the online server,... and > that mail had already been fetched,... can I make the flag synced? No. Your stated goal is that the local server is a mail archive put into service due to limited space on your colo/VPS server. An archive is an archive, not a secondary online server. It should only be accessed, read only, when you want to search and read an old message. And in fact, since this is an archive, you should implement the zlib plugin with dbox so all this archived mail is compressed in real time. Make up your mind. You can't have it both ways. I hear the iPhone5 can do anything automatically, no setup. Get one of those, problem solved. ;) > 3) Is dovecot suitable for the local server? Yes. Probably more than any other IMAP server. > - I couldn't sue maildir locally, because I loose just to much space to > the block fragmentation. Maildir causes the least filesystem fragmentation. You must be thinking of mbox, which causes heavy fragmentation due to constant appends past EOF. As I said you need dbox. One email per file, similar to maildir, but better integration and performance with Dovecot. > - I'd prefer not to use dbox (the thing that the indices are crucial > scares me a bit off). Are you designing/building this home server to be unreliable? Does it crash often? If so fix that problem and dbox is fine. If can't make it reliable use maildir which has expendable indexes. > a) When using mbox... is dovecot able to manage a really big folder > hierarchy that basically ever keeps growing... with easily several 100k > mails per folder... and that is in total already over 100GB? You have 100K emails in a single Dovecot mbox file? Or are you talking about an IMAP folder in TB that has no email in it, but many more IMAP folders whose combined email total is 100K? If you're worried about dbox index corruption, then you should be far more worried about mbox file corruption. With mbox files that large I'm surprised you've not hit it already. This would suggest that system is pretty stable. > - I would prefer to have fast full text search. Does dovecot provide > this? Yes. The problem with speed is two fold: 1. You must FTS often to keep the search indexes up to date. Wait a week between searches, after many new emails have been added to the IMAP folder, and your search crawls, as the file contents must be reindexed before the search starts. So you need to have a cron'd script that searches daily to keep the indexes up to date. 2. The mailbox file formats that best avoid fragmentation also have the slowest FTS times as the OS much open every file, 100K of them. If you use mbox or mdbox, you have far fewer files to open. mbox has the fastest FTS times of any format when indexes aren't fully up to date. It's also the fastest when updating the indexes. Your home server probably has a single SATA disk. mbox wins hands down for FTS due to very low IOPS load on the disk. The downside here is lack of good compression support--once you compress an mbox file you can't add new mail to it. This is where mdbox with compression comes in handy. With you 100K emails declaration, I think you're best served by mdbox with zlib compression. > I was looking into database backed mail systems (again,... just for the > local archive)... namely dbmail and archiveopteryx (are there other open > source solutions?)... > Not sure which of the two... or whether it's a good idea at all. > I remember some dovecot wiki page that showed a comparison which said > that both do not perfectly implement imap. > > Any suggestions with respect to that? If you're worried about fragmentation, or performance, I'd steer clear of a database driven mail store. Please, please, do not reply to each of my points here, and do not make this thread 100 replies. I'm not here to hold your hand. I don't have the time (nor patience) to engage in these lengthy emails. I gave you the architectural overview to build the correct solution to your problem. It's up to you to choose to use it or not, and if so, to do your own homework and self education, asking here only if something is unclear to you. In closing, you need real time bcc delivery which solves a ton of your mentioned problems. I'm not open to debating the merits of this. If you're not willing to meet the requirements for always_bcc, and you're determined to power the home server down most of the time, then you need assistance from someone else, as I simply have never used fetchmail, period, and have no idea if it can meet your needs. My guess is no, simply because, AFAIK, it doesn't work with LDA, which means you can't use sieve scripts and Dovecot's automatic sorting and indexing. Good luck. -- Stan From stan at hardwarefreak.com Tue Oct 9 11:02:46 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 09 Oct 2012 03:02:46 -0500 Subject: [Dovecot] some questions on dovecot or rather a mail system setup In-Reply-To: <5073D8F7.1060802@hardwarefreak.com> References: <1349732239.3344.62.camel@fermat.scientia.net> <5073D8F7.1060802@hardwarefreak.com> Message-ID: <5073DA26.5060809@hardwarefreak.com> On 10/9/2012 2:57 AM, Stan Hoeppner wrote: > http://www.postfix.org/postconf.5.html#always_bcc Correction. In your case you'll need to use: http://www.postfix.org/postconf.5.html#recipient_bcc_maps Because you said you only want to archive email for some users, not simply all mail received by the colo/VPS server. -- Stan From wamp at promax.media.pl Tue Oct 9 14:36:00 2012 From: wamp at promax.media.pl (wamp at promax.media.pl) Date: Tue, 9 Oct 2012 13:36:00 +0200 Subject: [Dovecot] Quota - usage counting. Message-ID: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> Hello, I use dovecot 1.2 version with postfix virtual users and mysql. All information about quota for every user is in mysql table. How dovecot compare if quota in database is over quota in /var/vmail/exampleuser directory ? It uses something like du command ? regards, Wamp From tibby at tibby.hu Tue Oct 9 14:57:45 2012 From: tibby at tibby.hu (Tibby) Date: Tue, 9 Oct 2012 13:57:45 +0200 (CEST) Subject: [Dovecot] Quota - usage counting. In-Reply-To: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> References: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> Message-ID: <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> Do you actually have a Guide? How did you set up quota form MySql? I'm having an issu getting it working. Can you share your dovecot.conf and dovecot-sql.conf? What's the user_query in your dovecot-sql.conf ? Thank you! ----- Original Message ----- From: wamp at promax.media.pl To: dovecot at dovecot.org Subject: [Dovecot] Quota - usage counting. Hello, I use dovecot 1.2 version with postfix virtual users and mysql. All information about quota for every user is in mysql table. How dovecot compare if quota in database is over quota in /var/vmail/exampleuser directory ? It uses something like du command ? regards, Wamp From wamp at promax.media.pl Tue Oct 9 16:12:41 2012 From: wamp at promax.media.pl (wamp at promax.media.pl) Date: Tue, 9 Oct 2012 15:12:41 +0200 Subject: [Dovecot] Quota - usage counting. In-Reply-To: <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> References: <4bcbc7b2aacfcef03ce82e35fb1f91df.squirrel@poczta.promax.media.pl> <2077186011.3397.1349783865262.JavaMail.root@tibby.hu> Message-ID: <7066b6528ee587362d87aa76ba75e7f4.squirrel@poczta.promax.media.pl> Hi > Do you actually have a Guide? No, I read some docs like http://www.serverubuntu.it/postfix-dovecot-guide > How did you set up quota form MySql? > I'm having an issu getting it working. > Can you share your dovecot.conf and dovecot-sql.conf? > > What's the user_query in your dovecot-sql.conf ? I can't make it working - so need information about general idea how this values should be compared. Where is info about actual size of maildir. regards, > > Thank you! > > ----- Original Message ----- > From: wamp at promax.media.pl > To: dovecot at dovecot.org > Subject: [Dovecot] Quota - usage counting. > > Hello, > > I use dovecot 1.2 version with postfix virtual users and mysql. All > information about quota for every user is in mysql table. > How dovecot compare if quota in database is over quota in > /var/vmail/exampleuser directory ? > > It uses something like du command ? > > > regards, > Wamp > > > > From fxmulder at gmail.com Tue Oct 9 21:41:08 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 12:41:08 -0600 Subject: [Dovecot] LDAP encryption Message-ID: We have an LDAP server that contains AES encrypted passwords. So far I've been able to use this by adding a passdb module that encrypts the user's password prior to ldap comparison. Now I am looking at supporting client-side encrypted passwords. To do this I need to decrypt the password returned by LDAP. Is there a way to insert a module to do this decryption between ldap returning and the auth mechanism? From fxmulder at gmail.com Tue Oct 9 22:16:15 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 13:16:15 -0600 Subject: [Dovecot] LDAP encryption In-Reply-To: <5074750D.4070302@bitrate.net> References: <5074750D.4070302@bitrate.net> Message-ID: I don't think I understand. Right now the problem is the password retrieved from LDAP cannot be hashed to compare against what the user sent because it is encrypted. I have to perform my AES decryption before it can be hashed and compared. On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: > On 2012.10.09 14.41, James Devine wrote: > >> We have an LDAP server that contains AES encrypted passwords. So far I've >> been able to use this by adding a passdb module that encrypts the user's >> password prior to ldap comparison. Now I am looking at supporting >> client-side encrypted passwords. To do this I need to decrypt the >> password >> returned by LDAP. Is there a way to insert a module to do this decryption >> between ldap returning and the auth mechanism? >> >> that would be unwise, generally speaking. as a rule of thumb, in terms > of security fundamentals, only the rootdn [or equiv] should be able to read > the values in an ldap entry's password attribute. certainly the service > account used by dovecot should not. > > in the context of ldap, authentication should be accomplished by binding > as the user, not by retrieving attribute values and performing string > comparisons. among other things, this decouples the two components and > allows applications [e.g. dovecot] to be unconcerned with whatever password > hashing scheme the directory server might be using. > > -ben > From fxmulder at gmail.com Tue Oct 9 22:24:21 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 9 Oct 2012 13:24:21 -0600 Subject: [Dovecot] LDAP encryption In-Reply-To: References: <5074750D.4070302@bitrate.net> Message-ID: Here is an example of the problem: Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: password(user at domain.tld,192.168.160.49): Generating NTLM from user 'user at domain.tld@', password 'IfBG6G3jykirE5r5vienC4w==' Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: password(user at domain.tld,192.168.160.49): Credentials: f124dc24328ed3d90db035f0d5284636 The listed password is a base64 representation of its encrypted form which I need to somehow decrypt between the time LDAP returns it and these credentials are generated. On Tue, Oct 9, 2012 at 1:16 PM, James Devine wrote: > I don't think I understand. Right now the problem is the password > retrieved from LDAP cannot be hashed to compare against what the user sent > because it is encrypted. I have to perform my AES decryption before it can > be hashed and compared. > > > On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: > >> On 2012.10.09 14.41, James Devine wrote: >> >>> We have an LDAP server that contains AES encrypted passwords. So far >>> I've >>> been able to use this by adding a passdb module that encrypts the user's >>> password prior to ldap comparison. Now I am looking at supporting >>> client-side encrypted passwords. To do this I need to decrypt the >>> password >>> returned by LDAP. Is there a way to insert a module to do this >>> decryption >>> between ldap returning and the auth mechanism? >>> >>> that would be unwise, generally speaking. as a rule of thumb, in terms >> of security fundamentals, only the rootdn [or equiv] should be able to read >> the values in an ldap entry's password attribute. certainly the service >> account used by dovecot should not. >> >> in the context of ldap, authentication should be accomplished by binding >> as the user, not by retrieving attribute values and performing string >> comparisons. among other things, this decouples the two components and >> allows applications [e.g. dovecot] to be unconcerned with whatever password >> hashing scheme the directory server might be using. >> >> -ben >> > > From tss at iki.fi Tue Oct 9 22:30:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 9 Oct 2012 22:30:19 +0300 Subject: [Dovecot] LDAP encryption In-Reply-To: References: <5074750D.4070302@bitrate.net> Message-ID: <97988640-DCC0-4068-A88F-D0201DBBE8EF@iki.fi> I don't think you can do that with a plugin without core Dovecot modifications. Unless you replace the whole passdb ldap. For example you could use passdb checkpassword if performance isn't a big issue. On 9.10.2012, at 22.24, James Devine wrote: > Here is an example of the problem: > > Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: > password(user at domain.tld,192.168.160.49): Generating NTLM from user > 'user at domain.tld@', password 'IfBG6G3jykirE5r5vienC4w==' > Oct 9 13:19:53 smtp-outgoing2 dovecot: auth: Debug: > password(user at domain.tld,192.168.160.49): Credentials: > f124dc24328ed3d90db035f0d5284636 > > The listed password is a base64 representation of its encrypted form which > I need to somehow decrypt between the time LDAP returns it and these > credentials are generated. > > > > On Tue, Oct 9, 2012 at 1:16 PM, James Devine wrote: > >> I don't think I understand. Right now the problem is the password >> retrieved from LDAP cannot be hashed to compare against what the user sent >> because it is encrypted. I have to perform my AES decryption before it can >> be hashed and compared. >> >> >> On Tue, Oct 9, 2012 at 1:03 PM, btb wrote: >> >>> On 2012.10.09 14.41, James Devine wrote: >>> >>>> We have an LDAP server that contains AES encrypted passwords. So far >>>> I've >>>> been able to use this by adding a passdb module that encrypts the user's >>>> password prior to ldap comparison. Now I am looking at supporting >>>> client-side encrypted passwords. To do this I need to decrypt the >>>> password >>>> returned by LDAP. Is there a way to insert a module to do this >>>> decryption >>>> between ldap returning and the auth mechanism? >>>> >>>> that would be unwise, generally speaking. as a rule of thumb, in terms >>> of security fundamentals, only the rootdn [or equiv] should be able to read >>> the values in an ldap entry's password attribute. certainly the service >>> account used by dovecot should not. >>> >>> in the context of ldap, authentication should be accomplished by binding >>> as the user, not by retrieving attribute values and performing string >>> comparisons. among other things, this decouples the two components and >>> allows applications [e.g. dovecot] to be unconcerned with whatever password >>> hashing scheme the directory server might be using. >>> >>> -ben >>> >> >> From marc at perkel.com Wed Oct 10 01:16:39 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 15:16:39 -0700 Subject: [Dovecot] Multiple Maildir? Message-ID: <5074A247.6080307@perkel.com> OK - Getting close to everything working in my weird configuration. Here's a problem I'm still having. I'm authenticating with this: userdb { driver = sql args = /etc/dovecot/master-combined-sql.conf } Default mail location: mail_location = maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs However - what I'd like to do is if the mail location doesn't exist then I want to try a second mail location: mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs How do I do that? Thanks in advance From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 10 01:42:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 10 Oct 2012 00:42:18 +0200 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074A247.6080307@perkel.com> References: <5074A247.6080307@perkel.com> Message-ID: <20121009224218.GA11401@daniel.localdomain> Marc Perkel wrote: > if the mail location doesn't exist > then I want to try a second mail location: > mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs You might do this with a script which exports the MAIL environment variable and then executes the service binary: http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection Regards Daniel -- https://plus.google.com/103021802792276734820 From kgc at corp.sonic.net Wed Oct 10 03:12:47 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 09 Oct 2012 17:12:47 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <20121009224218.GA11401@daniel.localdomain> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> Message-ID: <5074BD7F.50206@corp.sonic.net> On 10/09/12 15:42, Daniel Parthey wrote: > Marc Perkel wrote: >> if the mail location doesn't exist >> then I want to try a second mail location: >> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > > You might do this with a script which exports the MAIL environment > variable and then executes the service binary: It will work, we do this to set the maildir location to a custom hashed directory and muck around with the namespaces a bit. The script is in perl - the relevant parts look like this. #set user's maildir location for dovecot $ENV{'MAIL'} = 'maildir:' . getmaildir($ENV{'USER'}); $ENV{'USERDB_KEYS'} .= 'MAIL'; #pass along to dovecot's next process exec { $ARGV[0] } @ARGV; -K From marc at perkel.com Wed Oct 10 04:09:22 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:09:22 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <20121009224218.GA11401@daniel.localdomain> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> Message-ID: <5074CAC2.2030507@perkel.com> On 10/9/2012 3:42 PM, Daniel Parthey wrote: > Marc Perkel wrote: >> if the mail location doesn't exist >> then I want to try a second mail location: >> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > You might do this with a script which exports the MAIL environment > variable and then executes the service binary: > > http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection > > Regards > Daniel I see the idea. But how do I pick up the name and domain parameters to test the directory? From marc at perkel.com Wed Oct 10 04:12:11 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:12:11 -0700 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074BD7F.50206@corp.sonic.net> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> <5074BD7F.50206@corp.sonic.net> Message-ID: <5074CB6B.4090202@perkel.com> On 10/9/2012 5:12 PM, Kelsey Cummings wrote: > On 10/09/12 15:42, Daniel Parthey wrote: >> Marc Perkel wrote: >>> if the mail location doesn't exist >>> then I want to try a second mail location: >>> mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> >> You might do this with a script which exports the MAIL environment >> variable and then executes the service binary: > > It will work, we do this to set the maildir location to a custom > hashed directory and muck around with the namespaces a bit. > > The script is in perl - the relevant parts look like this. > > #set user's maildir location for dovecot > $ENV{'MAIL'} = 'maildir:' . getmaildir($ENV{'USER'}); > $ENV{'USERDB_KEYS'} .= 'MAIL'; > > #pass along to dovecot's next process > exec { $ARGV[0] } @ARGV; > > -K > > Namespaces is something I don't understand. Still wondering what environment variables I can pick up in this script. What I want to do is first try /fakedir/%d/%n and if that doesn't exist I want to go to /email/%d/%n From marc at perkel.com Wed Oct 10 04:34:08 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 18:34:08 -0700 Subject: [Dovecot] Feature Request Message-ID: <5074D090.3010909@perkel.com> It would be handy (for me) if there were a userdb where a directory structure defined the db. userdb stat { mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } userdb stat { mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs } The idea being that if the first directory doesn't exist then it will try the second one. From tss at iki.fi Wed Oct 10 05:29:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 05:29:05 +0300 Subject: [Dovecot] Feature Request In-Reply-To: <5074D090.3010909@perkel.com> References: <5074D090.3010909@perkel.com> Message-ID: <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> On 10.10.2012, at 4.34, Marc Perkel wrote: > It would be handy (for me) if there were a userdb where a directory structure defined the db. > > userdb stat { > mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > } > > userdb stat { > mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > } > > The idea being that if the first directory doesn't exist then it will try the second one. You could already implement this as userdb checkpassword script. From marc at perkel.com Wed Oct 10 05:40:27 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 09 Oct 2012 19:40:27 -0700 Subject: [Dovecot] Feature Request In-Reply-To: <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> Message-ID: <5074E01B.8030001@perkel.com> On 10/9/2012 7:29 PM, Timo Sirainen wrote: > On 10.10.2012, at 4.34, Marc Perkel wrote: > >> It would be handy (for me) if there were a userdb where a directory structure defined the db. >> >> userdb stat { >> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> } >> >> userdb stat { >> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >> } >> >> The idea being that if the first directory doesn't exist then it will try the second one. > You could already implement this as userdb checkpassword script. Can you give me an example? From tss at iki.fi Wed Oct 10 05:50:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 05:50:15 +0300 Subject: [Dovecot] Feature Request In-Reply-To: <5074E01B.8030001@perkel.com> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> <5074E01B.8030001@perkel.com> Message-ID: <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> On 10.10.2012, at 5.40, Marc Perkel wrote: >>> It would be handy (for me) if there were a userdb where a directory structure defined the db. >>> >>> userdb stat { >>> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >>> } >>> >>> userdb stat { >>> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs >>> } >>> >>> The idea being that if the first directory doesn't exist then it will try the second one. >> You could already implement this as userdb checkpassword script. > > Can you give me an example? Something like this: userdb { driver = checkpasword args = /usr/local/bin/userdb.sh } /usr/local/bin/userdb.sh: #!/bin/sh path=/fakedir/$AUTH_DOMAIN/$AUTH_USERNAME if [ -d $path ]; then MAIL=maildir:$path:INBOX=/email/$AUTH_DOMAIN/$AUTH_USERNAME:LAYOUT=fs EXTRA=mail AUTHORIZED=2 exec "$@" fi exit 1 (I'm not sure if the MAIL/mail should be USERDB_MAIL/userdb_mail instead. Probably not.) From raabe at froglogic.com Wed Oct 10 10:29:00 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 09:29:00 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes Message-ID: <507523BC.9050004@froglogic.com> Hi, I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for providing access to a mailing list archive. The archive is implemented as a public read-only mailbox with per-user index files (i.e. the \Seen flags are per-user). I recently enbled the Squat plugin to accelerate searches in the message bodies and noticed that every user (I'm using a virtual user setup) gets his own dovecot.index.search and dovecot.index.search.uids copies. Is it possible to share those files among all users of the system? The squat plugin appears to store the search indices among the other index files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no matter what; I considered storing a central copy of the index files somewhere and then creating symlinks for all users. It should be ok as far as file-permissions go since all mail is owned by a single vmail system user, but I wonder whether the indices are really the same (I noticed their md5 checksums differ) and whether there may be file locking issues in case two users search message bodies simultaneously. Can anybody shed some light? -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From robert at schetterer.org Wed Oct 10 10:49:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 09:49:33 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <507523BC.9050004@froglogic.com> References: <507523BC.9050004@froglogic.com> Message-ID: <5075288D.9080304@schetterer.org> Am 10.10.2012 09:29, schrieb Frerich Raabe: > Hi, > > I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for > providing access to a mailing list archive. The archive is implemented > as a public read-only mailbox with per-user index files (i.e. the \Seen > flags are per-user). i guess better upgrade to 2.1.x first > > I recently enbled the Squat plugin to accelerate searches in the message > bodies and noticed that every user (I'm using a virtual user setup) gets > his own dovecot.index.search and dovecot.index.search.uids copies. > > Is it possible to share those files among all users of the system? The > squat plugin appears to store the search indices among the other index > files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no > matter what; I considered storing a central copy of the index files > somewhere and then creating symlinks for all users. It should be ok as > far as file-permissions go since all mail is owned by a single vmail > system user, but I wonder whether the indices are really the same (I > noticed their md5 checksums differ) and whether there may be file > locking issues in case two users search message bodies simultaneously. > > Can anybody shed some light? > after upgrade http://wiki2.dovecot.org/Plugins/FTS/Lucene may be better choice this info might help http://wiki2.dovecot.org/MailLocation ---snip Index files Index files are by default stored under the same directory as mails. With maildir they are stored in the actual maildirs, with mbox they are stored under .imap/ directory. You may want to change the index file location if you're using NFS or if you're setting up shared mailboxes. You can change the index file location by adding :INDEX= to mail_location. For example: mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u --snip after upgrade come back, ask again, or meanwhile Timo gives better advice -- Best Regards MfG Robert Schetterer From raabe at froglogic.com Wed Oct 10 11:06:47 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 10:06:47 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <5075288D.9080304@schetterer.org> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> Message-ID: <50752C97.1010209@froglogic.com> Am 10.10.2012 09:49, schrieb Robert Schetterer: > Am 10.10.2012 09:29, schrieb Frerich Raabe: >> I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for >> providing access to a mailing list archive. The archive is implemented >> as a public read-only mailbox with per-user index files (i.e. the \Seen >> flags are per-user). > > i guess better upgrade to 2.1.x > first Given that Dovecot 1.2.17 works fine for me, I actually didn't see the need to upgrade yet. >> I recently enbled the Squat plugin to accelerate searches in the message >> bodies and noticed that every user (I'm using a virtual user setup) gets >> his own dovecot.index.search and dovecot.index.search.uids copies. >> >> Is it possible to share those files among all users of the system? The >> squat plugin appears to store the search indices among the other index >> files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no >> matter what; I considered storing a central copy of the index files >> somewhere and then creating symlinks for all users. It should be ok as >> far as file-permissions go since all mail is owned by a single vmail >> system user, but I wonder whether the indices are really the same (I >> noticed their md5 checksums differ) and whether there may be file >> locking issues in case two users search message bodies simultaneously. >> >> Can anybody shed some light? >> > > after upgrade > http://wiki2.dovecot.org/Plugins/FTS/Lucene > may be better choice Why? > this info might help > > http://wiki2.dovecot.org/MailLocation > > ---snip > Index files > > Index files are by default stored under the same directory as mails. > With maildir they are stored in the actual maildirs, with mbox they are > stored under .imap/ directory. You may want to change the index file > location if you're using NFS or if you're setting up shared mailboxes. > > You can change the index file location by adding :INDEX= to > mail_location. For example: > > mail_location = maildir:~/Maildir:INDEX=/var/indexes/%u > --snip I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: namespace public { separator = / prefix = Lists/ location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists subscriptions = no } Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) > after upgrade come back, ask again, or meanwhile Timo gives better advice Does this imply that questions regarding Dovecot 1.2.17 are considered offtopic on this list? If so, I apologize - I'll look for another forum then. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From robert at schetterer.org Wed Oct 10 11:24:49 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 10:24:49 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <50752C97.1010209@froglogic.com> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> Message-ID: <507530D1.8070202@schetterer.org> Am 10.10.2012 10:06, schrieb Frerich Raabe: > I already use this; as I mentioned, the index files of the public > readonly mailbox is stored per-user so that each user has his own set of > \Seen flags. Here's my public namespace: > > namespace public { > separator = / > prefix = Lists/ > location = > maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > > subscriptions = no > } > > Alas, this means that *all* index files (including the Squat index) is > stored per-user whereas I'd just to have just *some* of them per-user. :-) > >> after upgrade come back, ask again, or meanwhile Timo gives better advice > > Does this imply that questions regarding Dovecot 1.2.17 are considered > offtopic on this list? If so, I apologize - I'll look for another forum > then. no wait till others will reply, indexing questions might be more complex to answer, then i know recent in short time perhaps meanwhile this helps -snip http://wiki2.dovecot.org/SharedMailboxes/Public With Maildir a dovecot-shared file controls if the \Seen flags are shared or private. The file must be created separately inside each Maildir, although if the file already exists in the Maildir root it's automatically copied for newly created mailboxes. If dovecot-shared file doesn't exist in Maildir, the \Seen flags are shared. If it exists, the \Seen flag state is stored only in the user's index files. By making each user have their own private index files, you can make the \Seen flag private for the users. For example: namespace { type = public separator = / prefix = Public/ location = maildir:/var/mail/public:INDEX=~/Maildir/public subscriptions = no } Now when accessing e.g. "Public/lkml" mailbox, Dovecot keeps its index files in ~/Maildir/public/lkml/ directory. If it ever gets deleted, the \Seen flags are lost. If you want to change what flags are shared when dovecot-shared file exists, currently you'll have to modify the source code: src/lib-storage/index/maildir/maildir-storage.c maildir_open() has mbox->ibox.box.private_flags_mask = MAIL_SEEN; Change the MAIL_SEEN to any flag combination you want. See src/lib-mail/mail-types.h for list of valid flags. --snip however if there is a need for some new/debugged/better features relate to what you asked you have to upgrade to latest dovecot -- Best Regards MfG Robert Schetterer From sandro.tosi at dada.eu Wed Oct 10 12:23:17 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Wed, 10 Oct 2012 11:23:17 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups Message-ID: <50753E85.5060904@dada.eu> Hello, we're scouting if it's possible to use Pigeonhole (currently v0.3.1, as this will be provided with an upcoming Debian package) with MySQL dict lookups with the mail setup we're designing. Our (main) goals are: 1. store the filters on the database 2. allow each user to enable/disable any of the filters set we provide (it's a static set of some general filters, available to all the users; we're currently not providing the possibility to users to write their own filters) For point 1) we already see[1] that's possible, but it uses the map construct that might not fit with our current database structure: we have a domain table (storing the domain info) and a mailbox table (storing the mailbox info, but the username is composed by the local part, stored in this table, and the domain part is a FK to the domain table, using an id). Do you think it's possible to run a join query on domain+mailbox to retrieve the mailbox_id needed to query the table for the filters? Or do we have to create the filter table and store the local at domain.ext info there ("relaxing" the integrity relationships between tables)? How do we specify which filters are enabled for any given user? We originally thought of an "Enabled" field on the filter table, but in the example in the doc[1] I hadn't seen a way to do that: it seems like the filter list is specified in the proxy definition - am I wrong? How can we do that? [1] http://hg.rename-it.nl/dovecot-2.1-pigeonhole/file/e9ed5d5cef4b/doc/script-location-dict.txt I think it's enough for now, maybe additional questions will arise going deeper in details :) Thanks in advance, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From raabe at froglogic.com Wed Oct 10 12:49:27 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 10 Oct 2012 11:49:27 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <507530D1.8070202@schetterer.org> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <507530D1.8070202@schetterer.org> Message-ID: <507544A7.9050306@froglogic.com> Am 10.10.2012 10:24, schrieb Robert Schetterer: > Am 10.10.2012 10:06, schrieb Frerich Raabe: >> I already use this; as I mentioned, the index files of the public >> readonly mailbox is stored per-user so that each user has his own set of >> \Seen flags. Here's my public namespace: >> >> namespace public { >> separator = / >> prefix = Lists/ >> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >> subscriptions = no >> } [..] > perhaps meanwhile this helps > > -snip > > http://wiki2.dovecot.org/SharedMailboxes/Public [..] > namespace { > type = public > separator = / > prefix = Public/ > location = maildir:/var/mail/public:INDEX=~/Maildir/public > subscriptions = no > } Note how this is basically exactly the same as what I posted, except that it uses the Dovecot 2 configuration file format ('type = public') and that it calls the prefix/location "public" instead of "lists". > If you want to change what flags are shared when dovecot-shared file > exists, currently you'll have to modify the source code: > src/lib-storage/index/maildir/maildir-storage.c maildir_open() has > mbox->ibox.box.private_flags_mask = MAIL_SEEN; Change the MAIL_SEEN to > any flag combination you want. See src/lib-mail/mail-types.h for list of > valid flags. I don't think this is applicable to my case, and a check of the source code seems to confirm that: I'm not trying to change the set of flags stored for a given mail but rather the index file of the Squat plugin. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From natanael.copa at gmail.com Wed Oct 10 13:00:51 2012 From: natanael.copa at gmail.com (Natanael Copa) Date: Wed, 10 Oct 2012 12:00:51 +0200 Subject: [Dovecot] [PATCH] dovadm plugins underlinking Message-ID: Hi, Running doveadm on Alpine Linux will show various underlinking errors: /usr/bin/doveadm: symbol 'acl_user_module': can't resolve symbol in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. /usr/bin/doveadm: symbol 'acl_identifier_parse': can't resolve symbol in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. ... (complete list is found here: http://bugs.alpinelinux.org/issues/1274) I understand that those are "harmless" (at least as long as you dont use those plugins), and even if some dynamic linkers are forgiving, the doveadm plugin modules are technically underlinked. The lib10_doveadm_acl_plugin.so uses symbol acl_user_module: $ nm -D /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep acl_user_module U acl_user_module That symbol is defined in lib01_acl_plugin.so: $ nm -D /usr/lib/dovecot/lib01_acl_plugin.so | grep acl_user_module 000000000020fb80 D acl_user_module Which means that lib10_doveadm_acl_plugin.so should be directly linked to lib01_acl_plugin.so. But it is not: $ readelf -d /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so | grep NEEDED 0x0000000000000001 (NEEDED) Shared library: [librt.so.0.9.32] 0x0000000000000001 (NEEDED) Shared library: [libc.so.0.9.32] Below is a patch that should solve it. --- diff -r 4d268e810c15 src/plugins/acl/Makefile.am --- a/src/plugins/acl/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/acl/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -10,7 +10,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_acl_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_acl_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_acl_plugin_la_LIBADD = $(module_LTLIBRARIES) lib01_acl_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/expire/Makefile.am --- a/src/plugins/expire/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/expire/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -14,7 +14,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_expire_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_expire_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_expire_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_expire_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/fts/Makefile.am --- a/src/plugins/fts/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/fts/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib20_doveadm_fts_plugin_la_LDFLAGS = -module -avoid-version +lib20_doveadm_fts_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib20_doveadm_fts_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_fts_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/quota/Makefile.am --- a/src/plugins/quota/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/quota/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_quota_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_quota_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_quota_plugin_la_LIBADD = $(module_LTLIBRARIES) lib10_quota_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ diff -r 4d268e810c15 src/plugins/zlib/Makefile.am --- a/src/plugins/zlib/Makefile.am Mon Oct 08 08:53:54 2012 +0300 +++ b/src/plugins/zlib/Makefile.am Wed Oct 10 11:03:49 2012 +0200 @@ -11,7 +11,8 @@ -I$(top_srcdir)/src/doveadm NOPLUGIN_LDFLAGS = -lib10_doveadm_zlib_plugin_la_LDFLAGS = -module -avoid-version +lib10_doveadm_zlib_plugin_la_LDFLAGS = -module -avoid-version -Wl,-rpath,$(moduledir) +lib10_doveadm_zlib_plugin_la_LIBADD = $(module_LTLIBRARIES) lib20_zlib_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ From tss at iki.fi Wed Oct 10 13:22:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 13:22:53 +0300 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: References: Message-ID: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> On 10.10.2012, at 13.00, Natanael Copa wrote: > Running doveadm on Alpine Linux will show various underlinking errors: > /usr/bin/doveadm: symbol 'acl_user_module': can't resolve symbol in > lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. > /usr/bin/doveadm: symbol 'acl_identifier_parse': can't resolve symbol > in lib '/usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so'. .. > Which means that lib10_doveadm_acl_plugin.so should be directly linked > to lib01_acl_plugin.so. But it is not: .. > Below is a patch that should solve it. I think I once did that with imap_quota plugin for quota plugin. The problem was that then it was possible to set "mail_plugins=imap_quota", which automatically loaded the quota plugin, but didn't call its quota_plugin_init() function, which caused some strange behavior (crashed probably). With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. Maybe there are some other ways to make this work more nicely. There would still be time to change it for v2.2 if you have good ideas. :) From natanael.copa at gmail.com Wed Oct 10 15:17:58 2012 From: natanael.copa at gmail.com (Natanael Copa) Date: Wed, 10 Oct 2012 14:17:58 +0200 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> References: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> Message-ID: On Wed, Oct 10, 2012 at 12:22 PM, Timo Sirainen wrote: > On 10.10.2012, at 13.00, Natanael Copa wrote: > >> Running doveadm on Alpine Linux will show various underlinking errors: > With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. Why does it need to load all the plugins? Why not only try to load those who are enabled? > Maybe there are some other ways to make this work more nicely. There would still be time to change it for v2.2 if you have good ideas. :) Link statically? Or only link in the .o files that has the needed symbols? I'm not familiar with the code, but how about adding an char* adm_module to the module struct, and then instead of trying to dlopen all in doveadm dir, loop through all the already loaded modules and dlopen(loaded_module->adm_module)? -- Natanael Copa From listas at adminlinux.com.br Wed Oct 10 15:46:56 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Wed, 10 Oct 2012 09:46:56 -0300 Subject: [Dovecot] About ManageSieve and TLS In-Reply-To: <506B40F9.1000905@rename-it.nl> References: <506B3653.5020804@adminlinux.com.br> <506B40F9.1000905@rename-it.nl> Message-ID: <50756E40.2040408@adminlinux.com.br> It works ! Thanks Stephan! On 02-10-2012 16:31, Stephan Bosch wrote: > If you really want to, you can disable TLS for ManageSieve > specifically by putting a ssl=no inside the protocol sieve {} section. -- Thiago Henrique adminlinux.com.br From listas at adminlinux.com.br Wed Oct 10 16:39:27 2012 From: listas at adminlinux.com.br (3.listas@adminlinux.com.br) Date: Wed, 10 Oct 2012 10:39:27 -0300 Subject: [Dovecot] Irrelevant information filling logs Message-ID: <50757A8F.4030508@adminlinux.com.br> Hi, I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. There is a lot of logs of "quota exceeded" like this: Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. Is there a way to send specific Dovecot errors on specific files or just discard them? Thanks -- Thiago Henrique adminlinux.com.br From robert at schetterer.org Wed Oct 10 17:14:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 10 Oct 2012 16:14:15 +0200 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <507582B7.5040400@schetterer.org> Am 10.10.2012 15:39, schrieb 3.listas at adminlinux.com.br: > Hi, > > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My > mailbox server is shared by ~ 10k domains. It works fine with ~50k > accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, > user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: > msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded > (mailbox for user is full) > > These messages are not important to me. But these messages fill the log > files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or just > discard them? > > Thanks > -- > Thiago Henrique > adminlinux.com.br no sure if you can disable this notice without loosing other wanted ones by dovecot log settings if using rsyslog you can do it with discard action http://www.rsyslog.com/discarding-unwanted-messages/ -- Best Regards MfG Robert Schetterer From tibby at tibby.hu Wed Oct 10 17:29:08 2012 From: tibby at tibby.hu (Tibby) Date: Wed, 10 Oct 2012 16:29:08 +0200 (CEST) Subject: [Dovecot] I need a quota expert In-Reply-To: <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> References: <20121010142523.871B11AE881D@dovecot.org> <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> Message-ID: <648980957.4179.1349879348102.JavaMail.root@tibby.hu> I'm having problems with dovecot quota configuration. If I Include the driver mysql in dovecot-sql.conf I'm getting this error: dovecot: dict: Error in configuration file /etc/dovecot/dovecot-sql.conf line 1: Unknown setting: driver dovecot: dict: Failed to initialize dictionary 'quotadict' If I Don't include it: dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf Now What??? How Should I configure quota ? My SQL has a mail db with a user table. The table has email, password, quota_kb fields. my dovecot.conf: ---------------- protocols = imap imaps pop3 pop3s log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir disable_plaintext_auth = no ssl_cert_file = /etc/ssl/certs/dovecot.pem ssl_key_file = /etc/ssl/private/dovecot.pem namespace private { separator = . prefix = INBOX. inbox = yes } protocol imap { mail_plugins = quota imap_quota } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster at domain.tld mail_plugins = sieve quota global_script_path = /home/vmail/globalsieverc mail_plugin_dir = /usr/lib/dovecot/modules/lda } protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } dict { quotadict = mysql:/etc/dovecot/dovecot-sql.conf } plugin { quota = dict:user::proxy::quotadict quota_rule = *:storage=10M:messages=1000 } auth default { user = root passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb sql { args = /etc/dovecot/dovecot-sql.conf } socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } mail_uid = 5000 mail_gid = 5000 my dovecot-sql.conf: ---------------- driver = mysql connect = host=127.0.0.1 dbname=mail user=****** password=****** default_pass_scheme = CRYPT password_query = SELECT email as user, password, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u'; user_query = SELECT CONCAT('/home/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u' map { pattern = priv/quota/storage table = user username_field = user value_field = quota_kb } SOMEBODY PLEASE HELP ME!!! :) From jbates at brightok.net Wed Oct 10 17:37:16 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:37:16 -0500 Subject: [Dovecot] lmtp proxy logging Message-ID: <5075881C.4060905@brightok.net> The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. Jack From roundcube222 at alaadin.org Wed Oct 10 17:40:12 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 10 Oct 2012 17:40:12 +0300 Subject: [Dovecot] Dovecot doesnot disconnect at end of pop3 session Message-ID: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> Hello, I have 3 users checking same email account using pop3 (outlook express) suddenly after some hours , users are unable to disconnect from pop3 after checking mail So i telnet to the pop3 and found that the issue is true but i do not know the reason telnet mymail.com 110 user username pass password list quit all commands work except the quit command, Dovecot hang at this point !!! Why Dovecot hang on receiving the Quit command !!! and doesnot disconnect !! Any help is greatly appreciated. Thanks. From jbates at brightok.net Wed Oct 10 17:43:02 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:43:02 -0500 Subject: [Dovecot] I need a quota expert In-Reply-To: <648980957.4179.1349879348102.JavaMail.root@tibby.hu> References: <20121010142523.871B11AE881D@dovecot.org> <1304454090.4171.1349879210446.JavaMail.root@tibby.hu> <648980957.4179.1349879348102.JavaMail.root@tibby.hu> Message-ID: <50758976.8060302@brightok.net> Looking at the wiki: http://wiki2.dovecot.org/Quota/Dict It appears that the conf files for dict are different than the sql files for auth. This is why you are getting error messages. The auth config needs the driver option, the dict config does not. Create a file specifically for dict and point to that. Use the above wiki's examples for format. Jack On 10/10/2012 9:29 AM, Tibby wrote: > I'm having problems with dovecot quota configuration. > If I Include the driver mysql in dovecot-sql.conf I'm getting this error: > dovecot: dict: Error in configuration file /etc/dovecot/dovecot-sql.conf line 1: Unknown setting: driver > dovecot: dict: Failed to initialize dictionary 'quotadict' > > If I Don't include it: > dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf > dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf > > Now What??? > How Should I configure quota ? > > My SQL has a mail db with a user table. > The table has email, password, quota_kb fields. > > my dovecot.conf: > ---------------- > protocols = imap imaps pop3 pop3s > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/home/vmail/%d/%n/Maildir > disable_plaintext_auth = no > ssl_cert_file = /etc/ssl/certs/dovecot.pem > ssl_key_file = /etc/ssl/private/dovecot.pem > namespace private { > separator = . > prefix = INBOX. > inbox = yes > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol lda { > log_path = /home/vmail/dovecot-deliver.log > auth_socket_path = /var/run/dovecot/auth-master > postmaster_address = postmaster at domain.tld > mail_plugins = sieve quota > global_script_path = /home/vmail/globalsieverc > mail_plugin_dir = /usr/lib/dovecot/modules/lda > } > protocol pop3 { > pop3_uidl_format = %08Xu%08Xv > mail_plugins = quota > } > dict { > quotadict = mysql:/etc/dovecot/dovecot-sql.conf > } > plugin { > quota = dict:user::proxy::quotadict > quota_rule = *:storage=10M:messages=1000 > } > auth default { > user = root > passdb sql { > args = /etc/dovecot/dovecot-sql.conf > } > userdb sql { > args = /etc/dovecot/dovecot-sql.conf > } > socket listen { > master { > path = /var/run/dovecot/auth-master > mode = 0600 > user = vmail > } > client { > path = /var/spool/postfix/private/auth > mode = 0660 > user = postfix > group = postfix > } > } > } > mail_uid = 5000 > mail_gid = 5000 > > > > > > my dovecot-sql.conf: > ---------------- > driver = mysql > connect = host=127.0.0.1 dbname=mail user=****** password=****** > default_pass_scheme = CRYPT > password_query = SELECT email as user, password, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u'; > > user_query = SELECT CONCAT('/home/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u' > > map { > pattern = priv/quota/storage > table = user > username_field = user > value_field = quota_kb > } > > > SOMEBODY PLEASE HELP ME!!! :) From jbates at brightok.net Wed Oct 10 17:47:24 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 09:47:24 -0500 Subject: [Dovecot] Dovecot doesnot disconnect at end of pop3 session In-Reply-To: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> References: <98fbed2e480fab9c830ac14e210f65f1@Coptics.org> Message-ID: <50758A7C.1050501@brightok.net> On 10/10/2012 9:40 AM, Robert JR wrote: > > telnet mymail.com 110 > user username > pass password > list > quit > > all commands work except the quit command, Dovecot hang at this point !!! > > Why Dovecot hang on receiving the Quit command !!! and doesnot > disconnect !! > Please include your dovecot version. Also, does the list command end with a line that contains a period? You might want to include dovecot -n as well, which will show everyone what your mail storage type is and other relevant information. Jack From roundcube222 at alaadin.org Wed Oct 10 18:07:57 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 10 Oct 2012 18:07:57 +0300 Subject: [Dovecot] Fwd: Re: Dovecot doesnot disconnect at end of pop3 session Message-ID: On 2012-10-10 17:47, Jack Bates wrote: > On 10/10/2012 9:40 AM, Robert JR wrote: > >> telnet mymail.com 110 user username pass password list quit all >> commands work except the quit command, Dovecot hang at this point >> !!! >> Why Dovecot hang on receiving the Quit command !!! and doesnot >> disconnect !! > Please include your dovecot version. Also, does the list command end > with a line that contains a period? You might want to include dovecot > -n > as well, which will show everyone what your mail storage type is and > other relevant information. Jack # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.9.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login auth_username_format = %Lu disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mal:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = References: Message-ID: <50759753.5040301@brightok.net> On 10/10/2012 10:07 AM, Robert JR wrote: > > list command return msg number and size only. > Thanks Below is the proper format you should see. The line with a period at the end is required to show that the list command has completed. If you do not get that, then the server hasn't successfully completed the list command to accept new commands. Check your logs for any mention of the connection and errors that might have generated. If you are using NFS, there are several issues with locking that can be problematic. If you are getting the '.' line, then try other commands besides quit. It is important to determine if the code is ignoring input or if it has a problem with the actual quit code. Also, I notice you mentioned several hours. It would be useful to know if this occurs on startup or if it fails over time. It is also useful to know if it effects all accounts or just some of the active accounts. It is also useful to know if there are people currently logged in on those accounts when you perform your test. telnet localhost 110 Trying ::1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user **** +OK pass **** +OK Logged in. list +OK 3 messages: 1 2821 2 5907 3 11171 . quit +OK Logging out. Connection closed by foreign host. From slitt at troubleshooters.com Wed Oct 10 20:35:08 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Wed, 10 Oct 2012 13:35:08 -0400 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <20121010133508.6f836584@mydesk> On Wed, 10 Oct 2012 10:39:27 -0300, 3.listas at adminlinux.com.br said: > Hi, > > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. > My mailbox server is shared by ~ 10k domains. It works fine with ~50k > accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, > user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: > msgid=unspecified: failed to store into mailbox 'INBOX': Quota > exceeded (mailbox for user is full) > > These messages are not important to me. But these messages fill the > log files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or > just discard them? If all else fails, you could cat the log through a grep -v to filter out the quota exceeded messages, and then pipe it to less for viewing. SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From jbates at brightok.net Wed Oct 10 21:09:25 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 10 Oct 2012 13:09:25 -0500 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <5075B9D5.8030007@brightok.net> On 10/10/2012 8:39 AM, 3.listas at adminlinux.com.br wrote: > > Is there a way to send specific Dovecot errors on specific files or > just discard them? syslog/rsyslog filters are the best method, in my opinion. This would allow you to filter to another file or discard things you aren't interested in all together. I personally like quota messages in another logfile. Then I can tail the file with a script and issue changes so that the MTA servers reject emails to that user instead of trying to deliver, then periodically check quotas for users we currently block to determine when it is okay to accept emails again. It really lowers the bounces on the MTA servers. Jack From tss at iki.fi Wed Oct 10 22:37:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 10 Oct 2012 22:37:25 +0300 Subject: [Dovecot] Irrelevant information filling logs In-Reply-To: <50757A8F.4030508@adminlinux.com.br> References: <50757A8F.4030508@adminlinux.com.br> Message-ID: <5B596E8E-0721-4EE9-81E2-FC7F308B3C5C@iki.fi> On 10.10.2012, at 16.39, 3.listas at adminlinux.com.br wrote: > I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. > > There is a lot of logs of "quota exceeded" like this: > > Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) > > These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. > > Is there a way to send specific Dovecot errors on specific files or just discard them? Upgrade to v2.1, they are logged with info level there. (I think v2.0 also logs them with info level if you don't use Sieve.) From dovecot at freakout.de Thu Oct 11 09:43:18 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Thu, 11 Oct 2012 08:43:18 +0200 (CEST) Subject: [Dovecot] dovecot cores Message-ID: <201210110643.q9B6hIF6003121@bongo.freakout.de> Hi dovecot-community, i have set up a new dovecot server - everything is self-compiled and with newest versions. dovecot daemon dies every night: Bugreport Mail - dovecot cores with sig11 my dovecot daemon cores every night - no message in log file. Please help. It runs in an OpenVZ container - therefore reiserfs is reported as filesystem. root at glen ~]# cat /etc/dovecot/dovecot.conf # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.8.2.el5.028stab101.1PAE i686 CentOS release 5.8 (Final) reiserfs auth_mechanisms = plain login cram-md5 apop default_login_user = nobody disable_plaintext_auth = no first_valid_uid = 300 listen = * mail_gid = 332 mail_home = /var/dovecot mail_location = maildir:/var/spool/mail/virtual/%d/%n mail_plugins = quota mail_uid = 332 passdb { driver = sql args = /etc/dovecot/sql.conf } plugin { quota = maildir:User quota } protocols = imap pop3 ssl_cert = This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /opt/dovecot-2.1.10-3/sbin/dovecot...(no debugging symbols found)...done. [New Thread 20049] Reading symbols from /opt/mysql/lib/libmysqlclient.so.18...done. Loaded symbols for /opt/mysql/lib/libmysqlclient.so.18 Reading symbols from /opt/dovecot/lib/libdovecot.so.0...done. Loaded symbols for /opt/dovecot/lib/libdovecot.so.0 Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /opt/mysql/lib/libstrings.so...done. Loaded symbols for /opt/mysql/lib/libstrings.so Reading symbols from /opt/zlib/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /opt/zlib/lib/libz.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /opt/ssp/lib/libssp.so.0...done. Loaded symbols for /opt/ssp/lib/libssp.so.0 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/opt/dovecot/sbin/dovecot'. Program terminated with signal 11, Segmentation fault. #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 (gdb) bt #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 #1 0xb7f131c0 in master_instance_list_add_line (list=0x9d48880, line=0x9d540c8 "1349762052\tdovecot\t/var/dovecot/run\t") at master-instance.c:85 #2 0xb7f1331f in master_instance_list_refresh (list=0x9d48880) at master-instance.c:115 #3 0xb7f13620 in master_instance_write_init (list=0x9d48880, dotlock_r=0xbff6fb60) at master-instance.c:173 #4 0xb7f1390d in master_instance_list_set_name (list=0x9d48880, base_dir=0x8055300 "/var/dovecot/run", name=0x8055323 "dovecot") at master-instance.c:252 #5 0x0804c347 in net_accept () at network.c:502 #6 0xb7f3255b in io_loop_handle_timeouts_real (ioloop=0x9d465b0) at ioloop.c:354 #7 0xb7f325de in io_loop_handle_timeouts (ioloop=0x9d465b0) at ioloop.c:367 #8 0xb7f338b6 in io_loop_handler_run (ioloop=0x9d465b0) at ioloop-poll.c:171 #9 0xb7f326d3 in io_loop_run (ioloop=0x9d465b0) at ioloop.c:398 #10 0xb7f17304 in master_service_run (service=0x9d464e0, callback=0) at master-service.c:543 #11 0x0804d375 in net_accept () at network.c:502 #12 0xb7d84e9c in __libc_start_main () from /lib/libc.so.6 #13 0x0804b501 in net_accept () at network.c:502 (gdb) q Regards Axel From calestyo at scientia.net Thu Oct 11 05:18:34 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 04:18:34 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup (ignore) In-Reply-To: <1349921426.3341.175.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> Message-ID: <1349921914.3341.183.camel@fermat.scientia.net> Oops... that was obivously not intended for dovecot but AOX mailing list,... where I ask around similar questions. Sorry for the noise =) Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From talanchor at mail.ru Thu Oct 11 13:21:09 2012 From: talanchor at mail.ru (=?UTF-8?B?LiAu?=) Date: Thu, 11 Oct 2012 14:21:09 +0400 Subject: [Dovecot] =?utf-8?q?memory_allocation_in_new_thread?= Message-ID: <1349950869.22650094@f123.mail.ru> Hi! I have some problems with memory allocation. I create new thread in cidir storage and call malloc(), and it fails to allocate even 1 byte. What can cause this problem? Dovecot vesrion is: 2.1.10 (130563b592c9+) Sample code looks like this (I also link to pthread with: -pthread): #define TEST_MALLOC() \ { \ ??? void *p; \ ??? p = malloc(1); \ ??? if (!p) { \ ??? ??? i_info("%s: malloc() failed", __FUNCTION__); \ ??? } else { \ ??? ??? i_info("%s: malloc() succeeded", __FUNCTION__); \ ??? ??? free(p); \ ??? } \ } #include static void *test_pthread_malloc_func(void *data_) { ??? TEST_MALLOC(); ??? return NULL; } static void test_pthread_malloc() { ??? int ret; ??? pthread_t tid; ??? TEST_MALLOC(); ??? ret = pthread_create(&tid, NULL, test_pthread_malloc_func, NULL); ??? if (ret) { ??????? i_info("failed to start thread"); ??? } else { ??????? pthread_join(tid, NULL); ??? } } I call test_pthread_malloc() function. It produces following output: Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc: malloc() succeeded Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc_func: malloc() failed From calestyo at scientia.net Thu Oct 11 05:10:26 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 04:10:26 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup Message-ID: <1349921426.3341.175.camel@fermat.scientia.net> Hi folks. Perhaps you find some time to look into this,... I'm trying to plan my mail system and would have some questions. The overall idea is about the following: - There is a (internet) server, which is the MTA (which will be postfix) and imap and/or pop3 server (which shall be, guess, dovecot ... AOX comes in later). - Any spam filtering, virus-scanning, mail filtering happens on that server. - I have not yet decided on whether to use maildrop for this or Sieve. Maildrop is nice, but has one major deficiency which I don't know how to work around. - A few clients (that means I won't serve 100 of users) connect to that server via imap and should see all mails, etc. already in some fancy sorted hierarchy (that means filtered into different directories). So far nothing complicated. But now... - Apart from spam, I never delete mail; and because I'm subscribed to many lists, I get a lot of mail. - Storage on my server is limited and it's located somewhere at my ISP, so I generally do not trust it with respect to safety... For both reasons, I want the canonical archive of all mail to be at home at some local server. - The local server should also be an imap server, so that I can access the archive from may computers at home. - The local server won't be available from the internet. - The local archive should have the same folder hierarchy as the internet server (I'd prefer not to filter twice). - The local archive might be AOX... Mail Flow: - I generally want to have _all_ mail (which is not sorted out because of being spam) to be archived at the local server. - But(!) I want to selectively keep (in addition) mail at the internet server. For example I may want to select the folder that contains all mail form some friend to be kept online completely. But I may want to decide that mailinglists keep only the last 10 days and/or 1000 messages of mail. - The idea is, that the local server regularly (when it is online/running) catches new mail from the internet server... and stores it in the archive. - So apart from new mail that has not yet been read, that local archive contains always all mails that are also on the internet server... the later may contain (for specific directories) the same, or just parts of. - The MUAs will then have two imap accounts, one to the internet server and one to the local archive,... each one being usable, depending on where I am. 1) This is where my first problem arises: How can I implement that mail flow, especially: - How do I secure that all mail is read from the internet server (i.e. that nothing is "forgotten"? - How do I make sure that no mails are retrieved twice (or more)? A problem which I often had with pop, when the mail client crashed during sync? - Further it must be secured, that when I delete something on the internet server, it is NOT deleted on the local server (on the next mail-fetching).... this is why I don't use the word "sync". a) One stupid solution would be, that I duplicate all mail on the online server,... one part is for staying online, one part is for being fetched to the local archive. As soon as it was fetched... that copy gets removed (always). That solution would give a clean and secured separation of both? b) I don't think offlineimap or any other caching-like solution is the right thing... especially as one must always fear that such a cache may be accidentally wiped. Are there better solutions than (a)? 2) Problem would be already a refinement of a working solution for (1) (but obviously not when using (1).(a) ). When e.g. reply to or forward a mail using the online server,... and that mail had already been fetched,... can I make the flag synced? 3) Is AOX suitable for the local server? - I couldn't use maildir locally, because I loose just to much space to the block fragmentation. But I guess I won't have this problem with the DB backend. a) Is AOX able to manage a really big folder hierarchy that basically ever keeps growing... with easily several 100k mails per folder... and that is in total already over 100GB? I read that e.g. dovecot would have more performance problems with that. b) I would prefer to have fast full text search. Does AOX provide this? I read that IMAP has limitations which make full text search not really usable via it? Further, I remember some dovecot wiki page that showed a comparison which said that both do not perfectly implement imap. Thanks a lot, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From robert at schetterer.org Thu Oct 11 15:56:34 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 11 Oct 2012 14:56:34 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <1349921426.3341.175.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> Message-ID: <5076C202.7080202@schetterer.org> Am 11.10.2012 04:10, schrieb Christoph Anton Mitterer: > 3) Is AOX suitable for the local server? > - I couldn't use maildir locally, because I loose just to much space to > the block fragmentation. But I guess I won't have this problem with the DB backend. > > > a) Is AOX able to manage a really big folder hierarchy that basically > ever keeps growing... with easily several 100k > mails per folder... and that is in total already over 100GB? > > I read that e.g. dovecot would have more performance problems with that. > > > b) I would prefer to have fast full text search. Does AOX provide > this? > > I read that IMAP has limitations which make full text search not really usable via it? > > > Further, I remember some dovecot wiki page that showed a comparison which said > that both do not perfectly implement imap. Christoph, sorry, what exact is AOX, and what is its relation to the dovecot list.... youre looking to an overall mail setup so split your questions up to software you wanna use and ask the related mail list, perhaps hire some mail consultant advice you -- Best Regards MfG Robert Schetterer From robert at schetterer.org Thu Oct 11 15:57:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 11 Oct 2012 14:57:11 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup (ignore) In-Reply-To: <1349921914.3341.183.camel@fermat.scientia.net> References: <1349921426.3341.175.camel@fermat.scientia.net> <1349921914.3341.183.camel@fermat.scientia.net> Message-ID: <5076C227.9040303@schetterer.org> Am 11.10.2012 04:18, schrieb Christoph Anton Mitterer: > Oops... that was obivously not intended for dovecot but AOX mailing > list,... where I ask around similar questions. > > Sorry for the noise =) > > > Cheers, > Chris. > ups answered exact about this *g -- Best Regards MfG Robert Schetterer From raabe at froglogic.com Thu Oct 11 16:03:55 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 11 Oct 2012 15:03:55 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <5076C202.7080202@schetterer.org> References: <1349921426.3341.175.camel@fermat.scientia.net> <5076C202.7080202@schetterer.org> Message-ID: <5076C3BB.1090307@froglogic.com> Am 11.10.2012 14:56, schrieb Robert Schetterer: > Am 11.10.2012 04:10, schrieb Christoph Anton Mitterer: >> 3) Is AOX suitable for the local server? [..] > Christoph, sorry, what exact is AOX, and what is its relation to the > dovecot list.... I suppose he meant Archiveopteryx (another IMAP server). -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dovecot at freakout.de Thu Oct 11 16:19:27 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Thu, 11 Oct 2012 15:19:27 +0200 (CEST) Subject: [Dovecot] iterate_query does not use userdb - mail_location not found Message-ID: <201210111319.q9BDJRV3005614@bongo.freakout.de> Hi dovecot-comminity, can't get iterate_query working. doveadm cannot find mail_location which comes from userdb query. dovecot itself works fine with sql. Whats wrong? please help: [root at glen exim]# /opt/dovecot/bin/doveadm search -A mailbox Trash savedbefore 90d doveadm(uwe at mitmachnet.de): Error: user uwe at mitmachnet.de: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=(not set) doveadm(uwe at mitmachnet.de): Error: User init failed dovecot.conf: passdb { driver = sql args = /etc/dovecot/sql.conf } userdb { driver = sql args = /etc/dovecot/sql.conf } sql.conf: driver = mysql connect = host=much dbname=toarx user=exim password=xxxxxxxx default_pass_scheme = PLAIN password_query = select user as username, password, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' user_query = select user as username, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' iterate_query = select email as user from vusers Cheers Axel From jbates at brightok.net Thu Oct 11 17:28:40 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 11 Oct 2012 09:28:40 -0500 Subject: [Dovecot] iterate_query does not use userdb - mail_location not found In-Reply-To: <201210111319.q9BDJRV3005614@bongo.freakout.de> References: <201210111319.q9BDJRV3005614@bongo.freakout.de> Message-ID: <5076D798.80102@brightok.net> On 10/11/2012 8:19 AM, dovecot at freakout.de wrote: > sql.conf: > driver = mysql > connect = host=much dbname=toarx user=exim password=xxxxxxxx > default_pass_scheme = PLAIN > password_query = select user as username, password, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' > user_query = select user as username, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' > iterate_query = select email as user from vusers > Only prefetch in the password_query prefixes with userdb_. Some things such as doveadm commands and lmtp delivery don't use passdb but userdb directly. The user_query should not have the userdb_ prefix. Also, according to the wiki, iterate_query returns username, not user. http://wiki2.dovecot.org/AuthDatabase/SQL Jack From calestyo at scientia.net Thu Oct 11 19:39:49 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Thu, 11 Oct 2012 18:39:49 +0200 Subject: [Dovecot] some questions on AOX or rather a mail system setup In-Reply-To: <5076C3BB.1090307@froglogic.com> References: <1349921426.3341.175.camel@fermat.scientia.net> <5076C202.7080202@schetterer.org> <5076C3BB.1090307@froglogic.com> Message-ID: <1349973589.3370.4.camel@fermat.scientia.net> Hi. Sorry folks for the stupid postings... At first I posted what should go to the AOX list accidentally here and then our institute's MTA hat some issues yesterday, so the mail[0] where I already tried to explain the wrong posting, came much earlier than the wrong post itself. Guess you see why I need a better mail system ;) Sorry, Chris. [0] http://dovecot.org/pipermail/dovecot/2012-October/068740.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From btb at bitrate.net Thu Oct 11 20:35:00 2012 From: btb at bitrate.net (btb) Date: Thu, 11 Oct 2012 13:35:00 -0400 Subject: [Dovecot] imap proxy setup - "killed with signal 11" Message-ID: <50770344.70905@bitrate.net> hi- i'm setting up an imap proxy in front of a novell groupwise server. it seems to so far be partially working, but dovecot is having trouble in certain cases. i expect that it's ultimately due to what i believe is a very poor implementation of imap provided by groupwise [at least based on other experiences in the past] - but that's a big part of why i'd like to have dovecot in between it and clients. below is information collected during starting of dovecot, opening/initial connection from a client [os x mail.app], closing of the client, and stopping of dovecot. os is ubuntu 12.10 development/beta, dovecot is 2.1.7 courtesy of ubuntu's packages. log entries: Oct 11 13:24:33 halo dovecot: master: Dovecot v2.1.7 starting up Oct 11 13:24:49 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14171, TLS, session= Oct 11 13:24:50 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14174, TLS, session= Oct 11 13:24:51 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14176, TLS, session= Oct 11 13:24:51 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14178, TLS, session= Oct 11 13:24:51 halo dovecot: imap(jdoe): Connection closed in=16 out=350 Oct 11 13:24:52 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14180, TLS, session= Oct 11 13:24:52 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:52 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14176 killed with signal 11 (core dumped) Oct 11 13:24:57 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:57 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14178 killed with signal 11 (core dumped) Oct 11 13:24:57 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14182, TLS, session= Oct 11 13:24:58 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:24:58 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14180 killed with signal 11 (core dumped) Oct 11 13:25:03 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:03 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14182 killed with signal 11 (core dumped) Oct 11 13:25:03 halo dovecot: imap-login: Login: user=, method=PLAIN, rip=10.68.40.110, lip=10.59.1.53, mpid=14184, TLS, session= Oct 11 13:25:03 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:03 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14184 killed with signal 11 (core dumped) Oct 11 13:25:09 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Oct 11 13:25:09 halo dovecot: imap(jdoe): Fatal: master: service(imap): child 14174 killed with signal 11 (core dumped) [repeats] Oct 11 13:25:27 halo dovecot: dns-client: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: dns-client: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: master: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: auth: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: config: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: ssl-params: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: anvil: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) Oct 11 13:25:27 halo dovecot: log: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) gdb backtrace: ~ >gdb /usr/lib/dovecot/imap /var/cache/imapproxy/jdoe/core GNU gdb (GDB) 7.5-ubuntu Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug/usr/lib/dovecot/imap...done. done. [New LWP 13939] warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `dovecot/imap'. Program terminated with signal 11, Segmentation fault. #0 0x0000000000000000 in ?? () (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00007fc7f6cb611e in imap_parser_reset (parser=0x7fc7f8a0f3a0) at imap-parser.c:93 No locals. #2 0x00007fc7f6f7ada7 in imapc_connection_input_reset (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:664 No locals. #3 0x00007fc7f6f7c6f4 in imapc_connection_input_untagged (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:908 imap_args = 0x7fc7f8a0f4f8 name = 0x7fc7f8a0f5d0 "" value = parser = 0x0 reply = {name = 0x7fc7f8a0f5d0 "", num = 11, args = 0x7fc7f8a0f4f8, file_args = 0x7fc7f8a0d5d0, file_args_count = 0, resp_text_key = 0x0, resp_text_value = 0x0, untagged_box_context = 0x7fc7f8a1ad70} ret = #4 0x00007fc7f6f7d25e in imapc_connection_input_one (conn=0x7fc7f8a0d270) at imapc-connection.c:1061 tag = 0x7fc7f8a0f5c0 "" ret = -1 #5 imapc_connection_input_pending (conn=0x7fc7f8a0d270) at imapc-connection.c:1407 _data_stack_cur_id = 6 ret = #6 0x00007fc7f6f7d2c2 in imapc_connection_input (conn=0x7fc7f6c8f798) at imapc-connection.c:1100 errstr = ret = #7 0x00007fc7f6cdf006 in io_loop_call_io (io=0x7fc7f8a23800) at ioloop.c:379 ioloop = 0x7fc7f8a23630 t_id = 5 #8 0x00007fc7f6cdfcb7 in io_loop_handler_run (ioloop=ioloop at entry=0x7fc7f8a23630) at ioloop-epoll.c:213 ctx = 0x7fc7f8a23850 events = 0x7fc7f8a245e0 event = 0x7fc7f8a238c0 list = 0x7fc7f8a24320 io = tv = {tv_sec = 299, tv_usec = 999402} events_count = msecs = ret = 1 i = call = #9 0x00007fc7f6cdea18 in io_loop_run (ioloop=0x7fc7f8a23630) at ioloop.c:398 No locals. #10 0x00007fc7f6f7a0f7 in imapc_client_run_pre (client=) at imapc-client.c:142 connp = 0x7fc7f8a0cfe0 prev_ioloop = 0x7fc7f89e3670 #11 imapc_client_run (client=0x7fc7f8a0ce80) at imapc-client.c:161 No locals. #12 0x00007fc7f6f79254 in imapc_storage_run (storage=0x7fc7f8a0be60) at imapc-storage.c:118 No locals. #13 0x00007fc7f6f78311 in imapc_sync_index (ctx=0x7fc7f8a1fc70) at imapc-sync.c:351 mbox = 0x7fc7f8a1ad70 sync_rec = {uid1 = 47, uid2 = 0, type = (unknown: 4171272512), add_flags = 199 '\307', remove_flags = 127 '\177', keyword_idx = 0, guid_128 = "\000\000\000\000)\235\312\366\307\177\000\000p\255\241", } seq1 = 32767 seq2 = 0 #14 imapc_sync_begin (force=, ctx_r=, mbox=0x7fc7f8a1ad70) at imapc-sync.c:422 ctx = 0x7fc7f8a1fc70 sync_flags = ret = #15 imapc_sync (mbox=0x7fc7f8a1ad70) at imapc-sync.c:464 sync_ctx = force = #16 imapc_mailbox_sync_init (box=0x7fc7f8a1ad70, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at imapc-sync.c:498 mbox = 0x7fc7f8a1ad70 capabilities = changes = false ret = #17 0x00007fc7f6f8bd43 in mailbox_sync_init (box=0x7fc7f8a1ad70, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at mail-storage.c:1320 _data_stack_cur_id = 4 ctx = #18 0x00007fc7f6f8be67 in mailbox_sync (box=, flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at mail-storage.c:1368 ctx = status = {sync_delayed_expunges = 0} #19 0x00007fc7f74475d2 in select_open (readonly=false, mailbox=, ctx=0x7fc7f8a0da98) at cmd-select.c:296 client = 0x7fc7f8a13d30 status = {messages = 4171084000, recent = 32711, unseen = 0, uidvalidity = 0, uidnext = 13, first_unseen_seq = 0, first_recent_uid = 4137782496, last_cached_seq = 32711, highest_modseq = 13, keywords = 0x7fc7f6cd06fb , permanent_flags = 4171118192, nonpermanent_modseqs = 1, permanent_keywords = 1, allow_new_keywords = 1} flags = MAILBOX_FLAG_DROP_RECENT ret = #20 cmd_select_full (cmd=, readonly=) at cmd-select.c:419 ---Type to continue, or q to quit--- client = ctx = args = 0x7fc7f8a18598 list_args = 0x7fc7f89db0e0 mailbox = 0x7fc7f89db310 "Trash" ret = 1 __FUNCTION__ = "cmd_select_full" #21 0x00007fc7f744b29c in command_exec (cmd=cmd at entry=0x7fc7f8a0d9a0) at imap-commands.c:148 hook = 0x7fc7f89e4cd0 ret = #22 0x00007fc7f744a2ee in client_command_input (cmd=0x7fc7f6c8f798) at imap-client.c:682 client = 0x7fc7f8a13d30 command = __FUNCTION__ = "client_command_input" #23 0x00007fc7f744a39a in client_command_input (cmd=0x7fc7f8a0d9a0) at imap-client.c:733 client = 0x7fc7f8a13d30 command = __FUNCTION__ = "client_command_input" #24 0x00007fc7f744a5fd in client_handle_next_command (remove_io_r=, client=0x7fc7f8a13d30) at imap-client.c:774 size = 19 #25 client_handle_input (client=client at entry=0x7fc7f8a13d30) at imap-client.c:786 _data_stack_cur_id = 3 ret = 112 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #26 0x00007fc7f744aef5 in client_input (client=0x7fc7f8a13d30) at imap-client.c:825 cmd = output = 0x7fc7f8a0d868 bytes = 19 __FUNCTION__ = "client_input" #27 0x00007fc7f6cdf006 in io_loop_call_io (io=0x7fc7f8a11c90) at ioloop.c:379 ioloop = 0x7fc7f89e3670 t_id = 2 #28 0x00007fc7f6cdfcb7 in io_loop_handler_run (ioloop=ioloop at entry=0x7fc7f89e3670) at ioloop-epoll.c:213 ctx = 0x7fc7f89e39e0 events = 0x7fc7f8a245e0 event = 0x7fc7f89e3a50 list = 0x7fc7f8a0f380 io = tv = {tv_sec = 1739, tv_usec = 996790} events_count = msecs = ret = 1 i = call = #29 0x00007fc7f6cdea18 in io_loop_run (ioloop=0x7fc7f89e3670) at ioloop.c:398 No locals. #30 0x00007fc7f6ccb463 in master_service_run (service=0x7fc7f89e3520, callback=callback at entry=0x7fc7f7452f70 ) at master-service.c:544 No locals. #31 0x00007fc7f74428c7 in main (argc=1, argv=0x7fc7f89e3370) at main.c:389 set_roots = {0x7fc7f7658d20 , 0x0} login_set = {auth_socket_path = 0x7fc7f89db070 "/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x7fc7f7452e10 , failure_callback = 0x7fc7f7452b60 } service_flags = storage_service_flags = username = 0x0 c = (gdb) config: >doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-17-generic x86_64 Ubuntu quantal (development branch) first_valid_gid = 999 first_valid_uid = 999 imapc_host = backend.example.com last_valid_gid = 999 last_valid_uid = 999 log_timestamp = "%d.%m.%Y %H.%M.%S " login_greeting = dovecot ready mail_gid = imapproxy mail_home = /var/cache/imapproxy/%u mail_location = imapc:%h/%n/Maildir mail_uid = imapproxy passdb { args = host=backend.example.com default_fields = userdb_imapc_user=%u userdb_imapc_password=%w driver = imap } protocols = " imap" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } ssl = required ssl_cert = References: <50757A8F.4030508@adminlinux.com.br> <5B596E8E-0721-4EE9-81E2-FC7F308B3C5C@iki.fi> Message-ID: <50771D40.5000105@rename-it.nl> On 10/10/2012 9:37 PM, Timo Sirainen wrote: > On 10.10.2012, at 16.39, 3.listas at adminlinux.com.br wrote: > >> I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. >> >> There is a lot of logs of "quota exceeded" like this: >> >> Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user at mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) >> >> These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. >> >> Is there a way to send specific Dovecot errors on specific files or just discard them? > Upgrade to v2.1, they are logged with info level there. (I think v2.0 also logs them with info level if you don't use Sieve.) Nope, that is unfortunately not going to help right now. At least not until the following change I made yesterday is released: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/5c1ce25596ed Of course you can patch it if you're in a hurry. Regards, Stephan. From tss at iki.fi Thu Oct 11 22:48:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 22:48:34 +0300 Subject: [Dovecot] memory allocation in new thread In-Reply-To: <1349950869.22650094@f123.mail.ru> References: <1349950869.22650094@f123.mail.ru> Message-ID: <66C27C1C-8C0F-4835-9E73-CB5D22DFCB3F@iki.fi> On 11.10.2012, at 13.21, . . wrote: > Hi! I have some problems with memory allocation. > I create new thread in cidir storage and call malloc(), and it fails to allocate even 1 byte. > What can cause this problem? No idea. Dovecot in general isn't designed to work with threads. > Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc: malloc() succeeded > Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc_func: malloc() failed Well, or the one thing I can think of you to try: Set default_vsz_limit=0 Also straceing the process could show what exactly fails. From tss at iki.fi Thu Oct 11 22:52:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 22:52:01 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210110643.q9B6hIF6003121@bongo.freakout.de> References: <201210110643.q9B6hIF6003121@bongo.freakout.de> Message-ID: On 11.10.2012, at 9.43, dovecot at freakout.de wrote: > Core was generated by `/opt/dovecot/sbin/dovecot'. OK.. > #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 > #1 0xb7f131c0 in master_instance_list_add_line (list=0x9d48880, line=0x9d540c8 "1349762052\tdovecot\t/var/dovecot/run\t") > at master-instance.c:85 Dovecot code is calling str_to_time() from libmysqlclient, instead of from Dovecot's internal code. Not the first time mysql conflicted with Dovecot code. This could be worked around, but .. why is your dovecot binary linked with libmysqlclient? Only auth and dict binaries should be. From tss at iki.fi Thu Oct 11 23:10:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 23:10:56 +0300 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <50752C97.1010209@froglogic.com> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> Message-ID: <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> On 10.10.2012, at 11.06, Frerich Raabe wrote: > I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: > > namespace public { > separator = / > prefix = Lists/ > location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > subscriptions = no > } > > Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) You'll need v2.2 and its INDEXPVT setting. From stephan at rename-it.nl Thu Oct 11 23:35:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 11 Oct 2012 22:35:21 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50753E85.5060904@dada.eu> References: <50753E85.5060904@dada.eu> Message-ID: <50772D89.4050601@rename-it.nl> On 10/10/2012 11:23 AM, Sandro Tosi wrote: > Hello, > we're scouting if it's possible to use Pigeonhole (currently v0.3.1, > as this will be provided with an upcoming Debian package) with MySQL > dict lookups with the mail setup we're designing. > > Our (main) goals are: > > 1. store the filters on the database That is possible with some limitations. > 2. allow each user to enable/disable any of the filters set we provide > (it's a static set of some general filters, available to all the > users; we're currently not providing the possibility to users to write > their own filters) Will one or multiple scripts be active at the same time? > For point 1) we already see[1] that's possible, but it uses the map > construct that might not fit with our current database structure: we > have a domain table (storing the domain info) and a mailbox table > (storing the mailbox info, but the username is composed by the local > part, stored in this table, and the domain part is a FK to the domain > table, using an id). > > Do you think it's possible to run a join query on domain+mailbox to > retrieve the mailbox_id needed to query the table for the filters? Or > do we have to create the filter table and store the local at domain.ext > info there ("relaxing" the integrity relationships between tables)? My SQL is a bit rusty, but afaik this is possible with a JOIN or a nested query. > How do we specify which filters are enabled for any given user? We > originally thought of an "Enabled" field on the filter table, but in > the example in the doc[1] I hadn't seen a way to do that: it seems > like the filter list is specified in the proxy definition - am I > wrong? How can we do that? The above suggests that you would like to activate multiple Sieve scripts at the same time. That is currently not possible with the dict Script location. It is on my TODO list, but I am not sure when it will be ready (definitely not for coming Debian stable). For Dovecot v2.2 the new :optional tag for the Sieve include command could be used in - combination with the dict Sieve script location type - to provide some hackish solution. Unfortunately, in your case that is still not helpful, because v2.2 is not even in beta stage. :/ Regards, Stephan. From raabe at froglogic.com Thu Oct 11 23:38:51 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Thu, 11 Oct 2012 22:38:51 +0200 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> Message-ID: Am 11.10.2012 um 22:10 schrieb Timo Sirainen: > On 10.10.2012, at 11.06, Frerich Raabe wrote: >> I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: >> >> namespace public { >> separator = / >> prefix = Lists/ >> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >> subscriptions = no >> } >> >> Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) > > You'll need v2.2 and its INDEXPVT setting. Hm, you mean the feature introduced by http://hg.dovecot.org/dovecot-2.2/rev/dbd42f7198eb ? Is there some discussion of the feature somewhere? The commit log is a bit unclear to me, it says 'Per-user flags can now be stored in private index files.' however http://wiki2.dovecot.org/SharedMailboxes/Public says 'By making each user have their own private index files, you can make the \Seen flag private for the users.' (using the INDEX setting). Makes me wonder - the Wiki talks about 'private index files' when talking about 'INDEX' and the commit says 'private index files' talking about INDEXPVT - what is the difference? :-) -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dave.mehler at gmail.com Thu Oct 11 23:40:11 2012 From: dave.mehler at gmail.com (David Mehler) Date: Thu, 11 Oct 2012 16:40:11 -0400 Subject: [Dovecot] per-user quotas Message-ID: Hello, I've got quotas set up on an all-user basis on my system, it's a Postfix, Dovecot, Mysql virtual users setup. Currently I have each user getting a 1GB quota with these settings in 90-quota.conf: plugin { quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M } plugin { quota = maildir:User quota } While this works it's not what I want for all users. Say I add a virtual user called user1 to the mysql database but he's a test user and I only want that user to have 25 megabytes of storage, reading the wiki on quotas per-user items such as for example in a database overrides the global items above, is this right? If so, I'm hoping I'm not going to have to redo my entire user database, some users will have per-user quotas while I'll let others have the global quota. Thanks. Dave. From tss at iki.fi Thu Oct 11 23:46:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 11 Oct 2012 23:46:01 +0300 Subject: [Dovecot] Shared Squat index for public mailboxes In-Reply-To: References: <507523BC.9050004@froglogic.com> <5075288D.9080304@schetterer.org> <50752C97.1010209@froglogic.com> <914B946A-2A50-40DB-BD89-B6281E8D5A12@iki.fi> Message-ID: <99F7FC09-E21F-4808-9796-E2AAC31CDED0@iki.fi> On 11.10.2012, at 23.38, Frerich Raabe wrote: > Am 11.10.2012 um 22:10 schrieb Timo Sirainen: >> On 10.10.2012, at 11.06, Frerich Raabe wrote: >>> I already use this; as I mentioned, the index files of the public readonly mailbox is stored per-user so that each user has his own set of \Seen flags. Here's my public namespace: >>> >>> namespace public { >>> separator = / >>> prefix = Lists/ >>> location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists >>> subscriptions = no >>> } >>> >>> Alas, this means that *all* index files (including the Squat index) is stored per-user whereas I'd just to have just *some* of them per-user. :-) >> >> You'll need v2.2 and its INDEXPVT setting. > > Hm, you mean the feature introduced by http://hg.dovecot.org/dovecot-2.2/rev/dbd42f7198eb ? Yes. > Is there some discussion of the feature somewhere? http://markmail.org/message/45jxf363ffrubonv has some. > The commit log is a bit unclear to me, it says 'Per-user flags can now be stored in private index files.' however http://wiki2.dovecot.org/SharedMailboxes/Public says 'By making each user have their own private index files, you can make the \Seen flag private for the users.' (using the INDEX setting). > > Makes me wonder - the Wiki talks about 'private index files' when talking about 'INDEX' and the commit says 'private index files' talking about INDEXPVT - what is the difference? :-) You can have both! Shared indexes having the shared stuff (including squat indexes), while the private indexes only have the per-user flags, nothing else. For example with sdbox/mdbox you couldn't even have set per-user INDEX location or it would have just broken. From tibby at tibby.hu Thu Oct 11 23:49:11 2012 From: tibby at tibby.hu (Tibby) Date: Thu, 11 Oct 2012 22:49:11 +0200 Subject: [Dovecot] per-user quotas In-Reply-To: References: Message-ID: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> Hello! What's the version of your dovecot? 1.2.X ? or 2.0 ? Tibby On Oct 11, 2012, at 10:40 PM, David Mehler wrote: > Hello, > > I've got quotas set up on an all-user basis on my system, it's a > Postfix, Dovecot, Mysql virtual users setup. Currently I have each > user getting a 1GB quota with these settings in 90-quota.conf: > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } > plugin { > quota = maildir:User quota > } > > While this works it's not what I want for all users. Say I add a > virtual user called user1 to the mysql database but he's a test user > and I only want that user to have 25 megabytes of storage, reading the > wiki on quotas per-user items such as for example in a database > overrides the global items above, is this right? > > If so, I'm hoping I'm not going to have to redo my entire user > database, some users will have per-user quotas while I'll let others > have the global quota. > > Thanks. > Dave. From dave.mehler at gmail.com Fri Oct 12 00:43:57 2012 From: dave.mehler at gmail.com (David Mehler) Date: Thu, 11 Oct 2012 17:43:57 -0400 Subject: [Dovecot] per-user quotas In-Reply-To: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> References: <00C7D335-D72F-491A-91D5-0D9AD7E1D06C@tibby.hu> Message-ID: Hello, I'm running Dovecot 2.1.10. Thanks. Dave. On 10/11/12, Tibby wrote: > Hello! > > What's the version of your dovecot? 1.2.X ? or 2.0 ? > > Tibby > > On Oct 11, 2012, at 10:40 PM, David Mehler wrote: > >> Hello, >> >> I've got quotas set up on an all-user basis on my system, it's a >> Postfix, Dovecot, Mysql virtual users setup. Currently I have each >> user getting a 1GB quota with these settings in 90-quota.conf: >> >> plugin { >> quota_rule = *:storage=1G >> quota_rule2 = Trash:storage=+100M >> } >> plugin { >> quota = maildir:User quota >> } >> >> While this works it's not what I want for all users. Say I add a >> virtual user called user1 to the mysql database but he's a test user >> and I only want that user to have 25 megabytes of storage, reading the >> wiki on quotas per-user items such as for example in a database >> overrides the global items above, is this right? >> >> If so, I'm hoping I'm not going to have to redo my entire user >> database, some users will have per-user quotas while I'll let others >> have the global quota. >> >> Thanks. >> Dave. > > From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 02:01:36 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 01:01:36 +0200 Subject: [Dovecot] Multiple Maildir? In-Reply-To: <5074CAC2.2030507@perkel.com> References: <5074A247.6080307@perkel.com> <20121009224218.GA11401@daniel.localdomain> <5074CAC2.2030507@perkel.com> Message-ID: <20121011230136.GA9153@daniel.localdomain> Hi Marc, Marc Perkel wrote: > On 10/9/2012 3:42 PM, Daniel Parthey wrote: > >Marc Perkel wrote: > >>if the mail location doesn't exist > >>then I want to try a second mail location: > >>mail_location = maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >You might do this with a script which exports the MAIL environment > >variable and then executes the service binary: > >http://wiki2.dovecot.org/MailLocation#Custom_mailbox_location_detection > But how do I pick up the name and domain parameters to test the directory? You can get the username and any other userdb value from the environment, have a look at the environment section: http://wiki2.dovecot.org/PostLoginScripting#Running_environment Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 02:24:08 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 01:24:08 +0200 Subject: [Dovecot] Feature Request In-Reply-To: <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> References: <5074D090.3010909@perkel.com> <4DCEDBD8-83A3-4B97-9289-1FB9E7049702@iki.fi> <5074E01B.8030001@perkel.com> <331B0406-804A-4481-96B5-F857D7A5ADA9@iki.fi> Message-ID: <20121011232408.GA9444@daniel.localdomain> Timo Sirainen wrote: > On 10.10.2012, at 5.40, Marc Perkel wrote: > > >>> It would be handy (for me) if there were a userdb where a directory structure defined the db. > >>> > >>> userdb stat { > >>> mail_location=maildir:/fakedir/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >>> } > >>> > >>> userdb stat { > >>> mail_location=maildir:/email/%d/%n:INBOX=/email/%d/%n:LAYOUT=fs > >>> } > >>> > >>> The idea being that if the first directory doesn't exist then it will try the second one. > >> You could already implement this as userdb checkpassword script. > > > > Can you give me an example? > > Something like this: > > userdb { > driver = checkpasword > args = /usr/local/bin/userdb.sh > } Here is the documentation about how to implement a checkpassword script: http://wiki2.dovecot.org/AuthDatabase/CheckPassword Dovecot sets some environment variables that the script may use. All of the AUTH_* variables are available as AUTH_ extra fields in the environment: http://wiki2.dovecot.org/Variables#line-30 Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Fri Oct 12 03:15:39 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 12 Oct 2012 02:15:39 +0200 Subject: [Dovecot] per-user quotas In-Reply-To: References: Message-ID: <20121012001539.GA10473@daniel.localdomain> Hi Dave, David Mehler wrote: > I've got quotas set up on an all-user basis on my system, it's a > Postfix, Dovecot, Mysql virtual users setup. Currently I have each > user getting a 1GB quota with these settings in 90-quota.conf: > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } > plugin { > quota = maildir:User quota > } > > While this works it's not what I want for all users. Say I add a > virtual user called user1 to the mysql database but he's a test user > and I only want that user to have 25 megabytes of storage, reading the > wiki on quotas per-user items such as for example in a database > overrides the global items above, is this right? > > If so, I'm hoping I'm not going to have to redo my entire user > database, some users will have per-user quotas while I'll let others > have the global quota. You can just make your SQL query a bit more sophisticated in order to fit your needs. MySQL supports SQL CASE statement and default value with ELSE: http://dev.mysql.com/doc/refman/5.1/en/case.html This example sets quota to unlimited if mail comes in via port 20025, otherwise is uses the quota_bytes and quota_message columns: user_query = SELECT username AS user, \ home as home, \ uid as uid, \ gid as gid, \ CASE '%a' \ WHEN '20025' THEN '*:bytes=0:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota_bytes AS CHAR), ':messages=', CAST(quota_message AS CHAR)) \ END AS `quota_rule` \ FROM dovecot_users \ WHERE username='%u' Regards Daniel -- https://plus.google.com/103021802792276734820 From dovecot at freakout.de Fri Oct 12 09:50:16 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Fri, 12 Oct 2012 08:50:16 +0200 (CEST) Subject: [Dovecot] dovecot cores Message-ID: <201210120650.q9C6oGSG005182@bongo.freakout.de> According to Timo Sirainen: > On 11.10.2012, at 9.43, dovecot at freakout.de wrote: > > Dovecot code is calling str_to_time() from libmysqlclient, > instead of from Dovecot's internal code. > Not the first time mysql conflicted with Dovecot code. > This could be worked around, but .. why is your dovecot > binary linked with libmysqlclient? > Only auth and dict binaries should be. > but dovecot's configure script does not allow to specify the mysql libs and headers explictly - only by global CPPFLAGS and LDFLAGS extensions, which are used for all binaries - when i tried to specify: ./configure --prefix=/opt/dovecot --sysconfdir=/etc/dovecot --mandir=/opt/dovecot/man \ --docdir=/opt/dovecot/doc --libexecdir=/opt/dovecot/sbin --datadir=/opt/dovecot \ --with-rundir=/var/dovecot/run --with-statedir=/var/dovecot/state \ --with-mysql=/opt/mysql/bin/mysql_config checking for ... checking pkg-config is at least version 0.9.0... yes configure: error: --with-mysql=path not supported. You may want to use instead: CPPFLAGS=-I/opt/mysql/bin/mysql_config/include LDFLAGS=-L/opt/mysql/bin/mysql_config/lib ./configure --with-mysql i followed the hint from the configure script above and run into the core-dumps due to symbol clash str_to_time. How to work around with mysql in non-standard location? Thanks Axel From dovecot-list at mohtex.net Fri Oct 12 10:25:59 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 14:25:59 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120650.q9C6oGSG005182@bongo.freakout.de> References: <201210120650.q9C6oGSG005182@bongo.freakout.de> Message-ID: <5077C607.3010308@mohtex.net> dovecot at freakout.de wrote the following on 12.10.2012 13:50: > According to Timo Sirainen: >> On 11.10.2012, at 9.43, dovecot at freakout.de wrote: >> >> Dovecot code is calling str_to_time() from libmysqlclient, >> instead of from Dovecot's internal code. >> Not the first time mysql conflicted with Dovecot code. >> This could be worked around, but .. why is your dovecot >> binary linked with libmysqlclient? >> Only auth and dict binaries should be. >> > but dovecot's configure script does not allow to specify the > mysql libs and headers explictly - only by global CPPFLAGS and > LDFLAGS extensions, which are used for all binaries - when i > tried to specify: > > ./configure --prefix=/opt/dovecot --sysconfdir=/etc/dovecot --mandir=/opt/dovecot/man \ > --docdir=/opt/dovecot/doc --libexecdir=/opt/dovecot/sbin --datadir=/opt/dovecot \ > --with-rundir=/var/dovecot/run --with-statedir=/var/dovecot/state \ > --with-mysql=/opt/mysql/bin/mysql_config > checking for ... > checking pkg-config is at least version 0.9.0... yes > configure: error: --with-mysql=path not supported. You may want to use instead: > CPPFLAGS=-I/opt/mysql/bin/mysql_config/include LDFLAGS=-L/opt/mysql/bin/mysql_config/lib ./configure --with-mysql > > i followed the hint from the configure script above and run > into the core-dumps due to symbol clash str_to_time. > > How to work around with mysql in non-standard location? > > Thanks > Axel This one works for me for mysql in a non-standard location (my.cnf is in /etc): CPPFLAGS='-I/opt/mysql/include/mysql' LDFLAGS='-L/opt/mysql/lib/mysql -lmysqlclient -lz -lcrypt -lnsl -lm' ./configure --with-mysql ..... Rds Tamsy From tss at iki.fi Fri Oct 12 10:34:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:34:08 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120702.q9C72saS005717@bongo.freakout.de> References: <201210120702.q9C72saS005717@bongo.freakout.de> Message-ID: <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > According to Timo Sirainen: >> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? >> > > ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" > LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" -lmysqlclient shouldn't be in LDFLAGS. > if i omit "-lmysqlclient" (seems to be the reason for the hassle) i get: > > libtool: link: gcc4 -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include -o .libs/auth auth.o auth-cache.o auth-client-connection.o > auth-master-connection.o auth-postfix-connection.o mech-otp-skey-common.o mech-plain-common.o auth-penalty.o auth-request.o auth-request-handler.o auth-settings.o auth-stream.o auth-worker-client.o auth-worker-server.o db-checkpassword.o db-dict.o db-sql.o db-passwd-file.o main.o mech.o mech-anonymous.o mech-plain.o mech-login.o > mech-cram-md5.o mech-digest-md5.o mech-external.o mech-gssapi.o mech-ntlm.o mech-otp.o mech-scram-sha1.o mech-skey.o mech-rpa.o mech-apop.o mech-winbind.o passdb.o passdb-blocking.o passdb-bsdauth.o passdb-cache.o passdb-checkpassword.o passdb-dict.o passdb-passwd.o passdb-passwd-file.o passdb-pam.o passdb-shadow.o passdb-sia.o > passdb-vpopmail.o passdb-sql.o passdb-static.o passdb-template.o userdb.o userdb-blocking.o userdb-checkpassword.o userdb-dict.o userdb-nss.o userdb-passwd.o userdb-passwd-file.o userdb-prefetch.o userdb-static.o userdb-vpopmail.o userdb-sql.o userdb-template.o db-ldap.o passdb-ldap.o userdb-ldap.o -Wl,--export-dynamic -L/opt > /zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib libpassword.a ../lib-ntlm/libntlm.a ../lib-otp/libotp.a ../../src/lib-sql/.libs/libsql.a ../../src/lib-dovecot/.libs/libdovecot.so -lcrypt -ldl -Wl,-rpath -Wl,/opt/dovecot/lib > ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': > /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' >>>> ... tons of other undefined reference to mysqlclient > /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' > collect2: error: ld returned 1 exit status I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. From tss at iki.fi Fri Oct 12 10:39:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:39:07 +0300 Subject: [Dovecot] imap proxy setup - "killed with signal 11" In-Reply-To: <50770344.70905@bitrate.net> References: <50770344.70905@bitrate.net> Message-ID: <560FF304-5D04-44D1-AEC2-8DE8DC3F0943@iki.fi> On 11.10.2012, at 20.35, btb wrote: > i'm setting up an imap proxy in front of a novell groupwise server. it seems to so far be partially working, but dovecot is having trouble in certain cases. i expect that it's ultimately due to what i believe is a very poor implementation of imap provided by groupwise [at least based on other experiences in the past] - but that's a big part of why i'd like to have dovecot in between it and clients. > > os is ubuntu 12.10 development/beta, dovecot is 2.1.7 courtesy of ubuntu's packages. There have been a couple of imapc fixes since v2.1.7. It's possible that the crash is fixed by one of them. > Oct 11 13:24:52 halo dovecot: imap(jdoe): Error: imapc: Mailbox 'Trash' state corrupted: Expunged message reappeared in session (uid=6282 < next_uid=6283) Could you get imapc rawlogs where this happens? Point imapc_rawlog_dir setting to some directory. > #0 0x0000000000000000 in ?? () > No symbol table info available. > #1 0x00007fc7f6cb611e in imap_parser_reset (parser=0x7fc7f8a0f3a0) at imap-parser.c:93 > No locals. > #2 0x00007fc7f6f7ada7 in imapc_connection_input_reset (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:664 > No locals. > #3 0x00007fc7f6f7c6f4 in imapc_connection_input_untagged (conn=conn at entry=0x7fc7f8a0d270) at imapc-connection.c:908 This backtrace unfortunately doesn't make it very clear what the problem is. I'd guess it's trying to use already freed memory (one such bug was already fixed). From tss at iki.fi Fri Oct 12 10:40:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:40:36 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5075881C.4060905@brightok.net> References: <5075881C.4060905@brightok.net> Message-ID: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> On 10.10.2012, at 17.37, Jack Bates wrote: > The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. > > My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. From tss at iki.fi Fri Oct 12 10:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:45:05 +0300 Subject: [Dovecot] [PATCH] dovadm plugins underlinking In-Reply-To: References: <806FA839-00C7-47BB-9FCC-400B1E230DA6@iki.fi> Message-ID: On 10.10.2012, at 15.17, Natanael Copa wrote: > On Wed, Oct 10, 2012 at 12:22 PM, Timo Sirainen wrote: >> On 10.10.2012, at 13.00, Natanael Copa wrote: >> >>> Running doveadm on Alpine Linux will show various underlinking errors: > >> With doveadm the similar behavior is done for pretty much the same reason. doveadm tries to load all of the plugins, and it intentionally fails for those that fail to load due to not being enabled in mail_plugins setting. doveadm acl command shouldn't work if acl plugin isn't enabled. > > Why does it need to load all the plugins? Why not only try to load > those who are enabled? doveadm has two types of commands: mail commands and non-mail commands. The mail_plugins can add new mail features, and doveadm plugins can add more doveadm commands, which use the new mail features. But doveadm can also have plugins that add non-mail commands, which don't need anything in mail_plugins. So both cases would need to work.. From dovecot-list at mohtex.net Fri Oct 12 10:45:25 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 14:45:25 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> Message-ID: <5077CA95.1060506@mohtex.net> Timo Sirainen wrote the following on 12.10.2012 14:34: > On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > >> According to Timo Sirainen: >>> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? >>> >> ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" >> LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" > -lmysqlclient shouldn't be in LDFLAGS. > >> if i omit "-lmysqlclient" (seems to be the reason for the hassle) i get: >> >> libtool: link: gcc4 -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include -o .libs/auth auth.o auth-cache.o auth-client-connection.o >> auth-master-connection.o auth-postfix-connection.o mech-otp-skey-common.o mech-plain-common.o auth-penalty.o auth-request.o auth-request-handler.o auth-settings.o auth-stream.o auth-worker-client.o auth-worker-server.o db-checkpassword.o db-dict.o db-sql.o db-passwd-file.o main.o mech.o mech-anonymous.o mech-plain.o mech-login.o >> mech-cram-md5.o mech-digest-md5.o mech-external.o mech-gssapi.o mech-ntlm.o mech-otp.o mech-scram-sha1.o mech-skey.o mech-rpa.o mech-apop.o mech-winbind.o passdb.o passdb-blocking.o passdb-bsdauth.o passdb-cache.o passdb-checkpassword.o passdb-dict.o passdb-passwd.o passdb-passwd-file.o passdb-pam.o passdb-shadow.o passdb-sia.o >> passdb-vpopmail.o passdb-sql.o passdb-static.o passdb-template.o userdb.o userdb-blocking.o userdb-checkpassword.o userdb-dict.o userdb-nss.o userdb-passwd.o userdb-passwd-file.o userdb-prefetch.o userdb-static.o userdb-vpopmail.o userdb-sql.o userdb-template.o db-ldap.o passdb-ldap.o userdb-ldap.o -Wl,--export-dynamic -L/opt >> /zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib libpassword.a ../lib-ntlm/libntlm.a ../lib-otp/libotp.a ../../src/lib-sql/.libs/libsql.a ../../src/lib-dovecot/.libs/libdovecot.so -lcrypt -ldl -Wl,-rpath -Wl,/opt/dovecot/lib >> ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': >> /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' >>>>> ... tons of other undefined reference to mysqlclient >> /usr/src/rpm/BUILD/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' >> collect2: error: ld returned 1 exit status > I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). From tss at iki.fi Fri Oct 12 10:50:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 10:50:06 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <5077CA95.1060506@mohtex.net> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> <5077CA95.1060506@mohtex.net> Message-ID: <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> On 12.10.2012, at 10.45, Tamsy wrote: > Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". > > Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). If you run configure without adding the -lmysqlclient, what do you get with: egrep -i 'mysql|auth_libs|sql_libs' Makefile I guess the problem is that I shouldn't have copy&pasted the mysql detection code from php, and configure somehow passes successfully without actually setting any MYSQL_LIBS.. From dovecot-list at mohtex.net Fri Oct 12 11:04:46 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 12 Oct 2012 15:04:46 +0700 Subject: [Dovecot] dovecot cores In-Reply-To: <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> References: <201210120702.q9C72saS005717@bongo.freakout.de> <8F50313A-D1E7-4690-A483-1071FEBCAB86@iki.fi> <5077CA95.1060506@mohtex.net> <450E316F-8AF0-49D7-BB19-2D6BBF92FCF0@iki.fi> Message-ID: <5077CF1E.4060809@mohtex.net> Timo Sirainen wrote the following on 12.10.2012 14:50: > On 12.10.2012, at 10.45, Tamsy wrote: > >> Axel, please let us know whether one of these works: "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient". >> >> Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and only by putting -lmysqlclient in LDFLAGS as described before Dovecot compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a non-standart location). > If you run configure without adding the -lmysqlclient, what do you get with: > > egrep -i 'mysql|auth_libs|sql_libs' Makefile > > I guess the problem is that I shouldn't have copy&pasted the mysql detection code from php, and configure somehow passes successfully without actually setting any MYSQL_LIBS.. Just ran configure without adding the -lmysqlclient (CPPFLAGS='-I/opt/mysql/include/mysql' LDFLAGS='-L/opt/mysql/lib/mysql -lz -lcrypt -lnsl -lm' ./configure --with-mysql.... egrep -i 'mysql|auth_libs|sql_libs' Makefile says: AUTH_LIBS = CPPFLAGS = -I/opt/mysql/include/mysql LDFLAGS = $(NOPLUGIN_LDFLAGS) -L/opt/mysql/lib/mysql -lz -lcrypt -lnsl -lm MYSQL_CFLAGS = MYSQL_CONFIG = NO MYSQL_LIBS = PGSQL_LIBS = SQL_LIBS = sql_drivers = mysql scan-build -o scan-reports ../configure --with-ldap=auto --with-pgsql=auto --with-mysql=auto --with-sqlite=auto --with-solr=auto --with-gssapi=auto --with-libwrap=auto; \ configure runs without error but make ends with: ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_get_error': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:469: undefined reference to `mysql_errno' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:470: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_get_field_value_binary': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:436: undefined reference to `mysql_fetch_lengths' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_fetch_fields': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:383: undefined reference to `mysql_num_fields' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:384: undefined reference to `mysql_fetch_fields' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_next_row': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:366: undefined reference to `mysql_fetch_row' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:370: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_result_free': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:351: undefined reference to `mysql_free_result' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_do_query': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:233: undefined reference to `mysql_query' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:237: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_query_s': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:315: undefined reference to `mysql_affected_rows' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:316: undefined reference to `mysql_store_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:321: undefined reference to `mysql_next_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:332: undefined reference to `mysql_free_result' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:327: undefined reference to `mysql_errno' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_exec': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:284: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_escape_string': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:274: undefined reference to `mysql_real_escape_string' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:268: undefined reference to `mysql_escape_string' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_connect': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:83: undefined reference to `mysql_options' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:87: undefined reference to `mysql_options' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:92: undefined reference to `mysql_ssl_set' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:110: undefined reference to `mysql_real_connect' /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:123: undefined reference to `mysql_error' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_deinit_v': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:226: undefined reference to `mysql_close' ../../src/lib-sql/.libs/libsql.a(driver-mysql.o): In function `driver_mysql_parse_connect_string': /usr/local/src/dovecot-2.1.10/src/lib-sql/driver-mysql.c:198: undefined reference to `mysql_init' collect2: ld returned 1 exit status make[3]: *** [auth] Error 1 make[3]: Leaving directory `/usr/local/src/dovecot-2.1.10/src/auth' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/dovecot-2.1.10/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/dovecot-2.1.10' make: *** [all] Error 2 From dovecot at freakout.de Fri Oct 12 11:10:20 2012 From: dovecot at freakout.de (dovecot at freakout.de) Date: Fri, 12 Oct 2012 10:10:20 +0200 (CEST) Subject: [Dovecot] dovecot cores In-Reply-To: <5077CA95.1060506@mohtex.net> Message-ID: <201210120810.q9C8AK7V007314@bongo.freakout.de> According to Tamsy: > Timo Sirainen wrote the following on 12.10.2012 14:34: > > On 12.10.2012, at 10.02, dovecot at freakout.de wrote: > > > >> According to Timo Sirainen: > >>> Simply specifying -I or -L paths doesn't link with libmysql. What exactly did you use for CPPFLAGS/LDFLAGS/configure? > >>> > >> ok - i specified: CFLAGS="-I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include" > >> LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib -lmysqlclient" > > -lmysqlclient shouldn't be in LDFLAGS. > > > > I'm not sure why it's doing that. It really shouldn't. You could try SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or MYSQL_LIBS=-lmysqlclient if one of them helps. > Axel, please let us know whether one of these works: > "SQL_LIBS=-lmysqlclient or AUTH_LIBS=-lmysqlclient or > MYSQL_LIBS=-lmysqlclient". > > Since Dovecot 1.x all the way up to 2.1.10 I had trouble with this and > only by putting -lmysqlclient in LDFLAGS as described before Dovecot > compiles without error (Ubuntu Server 8.04 & 10.04, mySQL in a > non-standart location). > SQL_LIBS=-lmysqlclient => not working AUTH_LIBS=-lmysqlclient => not working MYSQL_LIBS=-lmysqlclient \ LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib" \ ./configure --prefix=%{_prefix} \ --sysconfdir=%{_etcdir} --mandir=%{_mandir} --docdir=%{_docdir} --libexecdir=%{_sbindir} --datadir=%{_prefix} \ --with-rundir=/var/dovecot/run \ --with-statedir=/var/dovecot/state \ --with-mysql => WORKING BUT: [axel at joe rpm]$ ldd BUILD/dovecot-2.1.10-root/opt/dovecot-2.1.10-5/sbin/dovecot libdovecot.so.0 => /opt/dovecot/lib/libdovecot.so.0 (0x00993000) libgcc_s.so.1 => /opt/gcc4/lib/libgcc_s.so.1 (0x0092c000) libc.so.6 => /lib/libc.so.6 (0x00ebf000) >>>>! libmysqlclient.so.18 => /opt/mysql/lib/libmysqlclient.so.18 (0x001cc000) libdl.so.2 => /lib/libdl.so.2 (0x00ae3000) libssp.so.0 => /opt/ssp/lib/libssp.so.0 (0x0057b000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x002c6000) libstrings.so => /opt/mysql/lib/libstrings.so (0x0057e000) libz.so.1 => /opt/zlib/lib/libz.so.1 (0x00110000) libpthread.so.0 => /lib/libpthread.so.0 (0x00b08000) libm.so.6 => /lib/libm.so.6 (0x00135000) dovecot still seems to be linked with the mysqlclient! i have just compiled - not tried the binaries - the core dump occurs only in the night! > If you run configure without adding the -lmysqlclient, what do you get with: egrep -i 'mysql|auth_libs|sql_libs' Makefile [axel at joe dovecot-2.1.10]$ egrep -i 'mysql|auth_libs|sql_libs' Makefile AUTH_LIBS = -lcrypt -lmysqlclient CFLAGS = -std=gnu99 -g -I/opt/zlib/include -I/opt/ssl/include -I/opt/mysql/include -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/opt/ssl/include LDFLAGS = $(NOPLUGIN_LDFLAGS) -L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib MYSQL_CFLAGS = MYSQL_CONFIG = NO MYSQL_LIBS = -lmysqlclient PGSQL_LIBS = SQL_LIBS = -lmysqlclient sql_drivers = mysql scan-build -o scan-reports ../configure --with-ldap=auto --with-pgsql=auto --with-mysql=auto --with-sqlite=auto --with-solr=auto --with-gssapi=auto --with-libwrap=auto; \ Cheers Axel From kjonca at o2.pl Fri Oct 12 11:01:49 2012 From: kjonca at o2.pl (Kamil =?iso-8859-2?Q?Jo=F1ca?=) Date: Fri, 12 Oct 2012 10:01:49 +0200 Subject: [Dovecot] [sieve] - counting headers Message-ID: <87wqyw6rv6.fsf@alfa.kjonca> In some of my maildrop filters I have rules with weighted scoring[1], but only to count headers (for example to count "Received:" header) ie. all these rules are of form "/pattern/:h,1" Can dovecot sieve do this? KJ [1] http://www.courier-mta.org/maildrop/maildropfilter.html -- http://blogdebart.pl/2012/06/24/hiena/ Wiesz, tryb tekstowy w Linuksie ma si? tak do DOSu jak F-117A do paralotni. (c) Dawid Kuroczko From busseniu at in.tum.de Fri Oct 12 17:10:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Fri, 12 Oct 2012 16:10:28 +0200 Subject: [Dovecot] INBOX.INBOX.Sent causes problems in 2.0 Message-ID: <507824D4.7080303@in.tum.de> Hi, we have our namespace rooted at "INBOX.". Sometimes, users have folders like "INBOX.INBOX.Sent" or "INBOX.INBOX.INBOX.Sent". I do not know why these folders are created; I suspect it is due to buggy MUAs. If a mailbox like INBOX.INBOX.Sent exists, then in some cases Dovecot counts all messages in INBOX twice. This behavior can be demonstrated as follows: 1) Create a folder named "INBOX.INBOX.Sent" (using the IMAP CREATE command) 2) Save a message to "INBOX" (in my example the message is 7 MB) 3) dovecot-quota contains: priv/quota/storage 7129025 priv/quota/messages 1 4) doveadm quota recalc -u user1 5) Now dovecot-quota contains priv/quota/storage 14258050 priv/quota/messages 2 I.e. the user will experience that his quota fills up very fast. "doveadm -f flow fetch -u user1 'mailbox guid' ALL" prints: mailbox=INBOX guid=040ce73645177850497d000040c59ffc mailbox=INBOX guid=040ce73645177850497d000040c59ffc However, "doveadm -f flow fetch -u user1 'mailbox guid' mailbox INBOX" prints the message only once: mailbox=INBOX guid=040ce73645177850497d000040c59ffc How can we prevent this kind of confusion? What I'd like most is prevent the creation of these weird folders. Most MUAs have problems listing them properly, especially if both "INBOX.Sent" and "INBOX.INBOX.Sent" exist. Dovecot 2.1 does not seem to count anything twice. Cheers, Christoph Config: # 2.0.21: /usr/local/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (6224) # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mail mail_plugins = quota mail_uid = vmail namespace default { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { quota = dict:ROOT::file:%h/dovecot-quota quota_rule = *:storage=5G } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } service imap { process_limit = 5000 } ssl_cert = Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From thefantaman at gmail.com Fri Oct 12 18:38:59 2012 From: thefantaman at gmail.com (thefantaman) Date: Fri, 12 Oct 2012 08:38:59 -0700 (PDT) Subject: [Dovecot] /var/run/dovecot/auth-userdb failed Message-ID: <1350056339814-38093.post@n4.nabble.com> Hi, I work on test server and if I send an email on log i read lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: /var/run/dovecot/auth-userdb, euid is not dir owner) This is my dovecot.conf: auth_mechanisms = plain login info_log_path = /var/log/dovecot listen = 0.0.0.0 log_path = /var/log/dovecot login_greeting = Dovecot IMAP Server ready. mail_location = maildir:/home/vmail/%d/%u mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-ldap.conf.ext.fabry driver = ldap } protocols = pop3 sieve imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = root group = root } } ssl_cert = HELP! I have installed sendmail, dovecot, and squirrel mail. The squirrel mail portion of it works just fine, but I would like to have Mozilla Thunderbird as a client. Whenever I try and connect to the server it says "Thunderbird failed to find the settings for your email account." We do have an MX record in DNS pointing to our server. We are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 We are doing this for a class project and it is due by Tuesday 10/16/2012.. Thanks, Justin From ben at indietorrent.org Fri Oct 12 20:12:57 2012 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 12 Oct 2012 13:12:57 -0400 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <50784F99.5080201@indietorrent.org> On 10/12/2012 1:09 PM, Justin Vore wrote: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email account." > We do have an MX record in DNS pointing to our server. We are using > Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 > We are doing this for a class project and it is due by Tuesday 10/16/2012.. > > > Thanks, > > > Justin > I have found Thunderbird's automatic setting detection mechanism to be rather unreliable. Try entering the settings manually. -Ben From arne at drlinux.no Fri Oct 12 20:20:46 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Fri, 12 Oct 2012 19:20:46 +0200 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <5078516E.2040505@drlinux.no> Den 12.10.2012 19:09, skrev Justin Vore: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email account." > We do have an MX record in DNS pointing to our server. We are using > Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 > We are doing this for a class project and it is due by Tuesday 10/16/2012.. This explains how to set up autoconfigure in TB. You need a DNS record and a little work on a webhost. https://developer.mozilla.org/en-US/docs/Thunderbird/Autoconfiguration Arne -- Arne K. Haaje http://www.drlinux.no/ ::: arne at drlinux.no LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From lists at kokelnet.de Fri Oct 12 22:48:03 2012 From: lists at kokelnet.de (Tobias Hachmer) Date: Fri, 12 Oct 2012 21:48:03 +0200 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: Am 12.10.2012 19:09, schrieb Justin Vore: > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email > account." > We do have an MX record in DNS pointing to our server. Enter settings manually or set up automx (http://www.automx.org/). Regards, Tobias Hachmer From tss at iki.fi Fri Oct 12 23:07:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 12 Oct 2012 23:07:08 +0300 Subject: [Dovecot] dovecot cores In-Reply-To: <201210120810.q9C8AK7V007314@bongo.freakout.de> References: <201210120810.q9C8AK7V007314@bongo.freakout.de> Message-ID: <371D7F4F-3534-4F52-B106-165A487E1828@iki.fi> On 12.10.2012, at 11.10, dovecot at freakout.de wrote: > MYSQL_LIBS=-lmysqlclient \ > LDFLAGS="-L/opt/zlib/lib -L/opt/ssl/lib -L/opt/mysql/lib" \ > ./configure --prefix=%{_prefix} \ > --sysconfdir=%{_etcdir} --mandir=%{_mandir} --docdir=%{_docdir} --libexecdir=%{_sbindir} --datadir=%{_prefix} \ > --with-rundir=/var/dovecot/run \ > --with-statedir=/var/dovecot/state \ > --with-mysql > => WORKING I think this is the correct fix for this. Basically same as what I committed now: http://hg.dovecot.org/dovecot-2.1/rev/c8d55ba25f39 > BUT: > > [axel at joe rpm]$ ldd BUILD/dovecot-2.1.10-root/opt/dovecot-2.1.10-5/sbin/dovecot > libdovecot.so.0 => /opt/dovecot/lib/libdovecot.so.0 (0x00993000) > libgcc_s.so.1 => /opt/gcc4/lib/libgcc_s.so.1 (0x0092c000) > libc.so.6 => /lib/libc.so.6 (0x00ebf000) >>>>> ! libmysqlclient.so.18 => /opt/mysql/lib/libmysqlclient.so.18 (0x001cc000) > libdl.so.2 => /lib/libdl.so.2 (0x00ae3000) > libssp.so.0 => /opt/ssp/lib/libssp.so.0 (0x0057b000) > /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x002c6000) > libstrings.so => /opt/mysql/lib/libstrings.so (0x0057e000) > libz.so.1 => /opt/zlib/lib/libz.so.1 (0x00110000) > libpthread.so.0 => /lib/libpthread.so.0 (0x00b08000) > libm.so.6 => /lib/libm.so.6 (0x00135000) > > dovecot still seems to be linked with the mysqlclient! Maybe the old LDFLAGS was cached somehow? Or maybe the rpm build does something strange? I don't see how that could happen otherwise. From mark at xwax.org Sat Oct 13 00:20:03 2012 From: mark at xwax.org (Mark Hills) Date: Fri, 12 Oct 2012 22:20:03 +0100 (BST) Subject: [Dovecot] dsync ignores ssh-agent Message-ID: <1210122159360.19545@vega.localdomain> I use IMAP over SSH, in pre-auth. I wanted to use dsync to offline mail to my laptop. dsync v2.1.10 would always ask for my SSH key/passphrase, ignoring ssh-agent. The culprit is the env_clean() in the stack below. Reading the source, I saw DOVECOT_PRESERVE_ENVS. When used as follows in my script it enables dsync to find my ssh-agent: export DOVECOT_PRESERVE_ENVS="SSH_AGENT_PID SSH_AUTH_SOCK" dsync mirror ssh imap.example.com /home/mark/opt/dovecot/bin/dsync and it now works without asking for password every time. I'm posting here so that anyone else googling for the same problem will hopefully find this, as I couldn't find anything about this in the docs. Also I'm interested in why dsync so aggressively cleans the environment; I tried a naive removal of env_clean() but this breaks basic functions. With this dsync is working very well for offline mail -- combined with alpine and a local exim for the outbound queue :) Thanks -- Mark Breakpoint 2, env_clean () at env-util.c:59 59 if (clearenv() < 0) (gdb) bt #0 env_clean () at env-util.c:59 #1 0xb7df10fc in master_service_env_clean () at master-service.c:454 #2 0xb7df26d4 in master_service_exec_config (service=0x809e7d0, input=0xbffff7e4) at master-service-settings.c:103 #3 0xb7df29be in config_exec_fallback (service=0x809e7d0, input=0xbffff7e4) at master-service-settings.c:153 #4 0xb7df2b65 in master_service_open_config (service=0x809e7d0, input=0xbffff7e4, path_r=0xbffff780, error_r=0xbffff7d8) at master-service-settings.c:206 #5 0xb7df3130 in master_service_settings_read (service=0x809e7d0, input=0xbffff7e4, output_r=0xbffff7dc, error_r=0xbffff7d8) at master-service-settings.c:345 #6 0x0805c672 in doveadm_read_settings () at doveadm.c:275 #7 0x0805c7d6 in main (argc=5, argv=0x809e1c0) at doveadm.c:342 From gedalya at gedalya.net Sat Oct 13 03:45:29 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 12 Oct 2012 20:45:29 -0400 Subject: [Dovecot] Help! In-Reply-To: References: Message-ID: <5078B9A9.6040707@gedalya.net> This has nothing to do with dovecot or with any server. Thunderbird tries to guess settings such as your IMAP and SMTP server addresses, ports, TLS, authentication scheme, etc. Sometimes none of the guess attempts matches your settings. That's not an actual problem. Just enter the settings manually. If your priority is to make Thunderbird quickly and automatically configure your email accounts, read here https://wiki.mozilla.org/Thunderbird:Autoconfiguration Note that this is Thunderbird-specific, Microsoft Outlook, Blackberry etc. each have their own autoconfiguration schemes. On 10/12/2012 01:09 PM, Justin Vore wrote: > HELP! > > I have installed sendmail, dovecot, and squirrel mail. The squirrel > mail portion of it works just fine, but I would like to have Mozilla > Thunderbird as a client. Whenever I try and connect to the server it > says "Thunderbird failed to find the settings for your email > account." We do have an MX record in DNS pointing to our server. We > are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version > 8.14.4-2ubuntu2 We are doing this for a class project and it is due by > Tuesday 10/16/2012.. > > > Thanks, > > > Justin From robert at schetterer.org Sat Oct 13 08:31:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 13 Oct 2012 07:31:54 +0200 Subject: [Dovecot] Help! In-Reply-To: <5078B9A9.6040707@gedalya.net> References: <5078B9A9.6040707@gedalya.net> Message-ID: <5078FCCA.2090606@schetterer.org> Am 13.10.2012 02:45, schrieb Gedalya: > This has nothing to do with dovecot or with any server. > Thunderbird tries to guess settings such as your IMAP and SMTP server > addresses, ports, TLS, authentication scheme, etc. Sometimes none of the > guess attempts matches your settings. That's not an actual problem. Just > enter the settings manually. > > If your priority is to make Thunderbird quickly and automatically > configure your email accounts, read here > https://wiki.mozilla.org/Thunderbird:Autoconfiguration > > Note that this is Thunderbird-specific, Microsoft Outlook, Blackberry > etc. each have their own autoconfiguration schemes. you may use http://www.automx.org/ for that > > > On 10/12/2012 01:09 PM, Justin Vore wrote: >> HELP! >> >> I have installed sendmail, dovecot, and squirrel mail. The squirrel >> mail portion of it works just fine, but I would like to have Mozilla >> Thunderbird as a client. Whenever I try and connect to the server it >> says "Thunderbird failed to find the settings for your email >> account." We do have an MX record in DNS pointing to our server. We >> are using Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version >> 8.14.4-2ubuntu2 We are doing this for a class project and it is due by >> Tuesday 10/16/2012.. >> >> >> Thanks, >> >> >> Justin > -- Best Regards MfG Robert Schetterer From alessio at skye.it Sat Oct 13 11:16:27 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 10:16:27 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= Message-ID: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> Hi, I'm running dovecot 2.1.10 on Debian 6. When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] If I add -D to doveadm I can see this: doveadm(myuser at mydomain.com): Debug: Added userdb setting: plugin/quota_rule=*:backend=524288000S doveadm(myuser at mydomain.com): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/2/mydomain.com/myuser doveadm(myuser at mydomain.com): Debug: Quota root: name=UserQuota backend=maildir args= doveadm(myuser at mydomain.com): Debug: Quota rule: root=UserQuota mailbox=* bytes=524288000 messages=0 doveadm(myuser at mydomain.com): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 doveadm(myuser at mydomain.com): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota doveadm(myuser at mydomain.com): Debug: dict quota: user=myuser at mydomain.com, uri=proxy::quota, noenforcing=1 doveadm(myuser at mydomain.com): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir doveadm(myuser at mydomain.com): Debug: maildir++: root=/home/vpopmail/domains/2/mydomain.com/myuser/Maildir, index=, control=, inbox=/home/vpopmail/domains/2/mydomain.com/myuser/Maildir, alt= doveadm(myuser at mydomain.com): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 doveadm(myuser at mydomain.com): Debug: acl: acl username = myuser at mydomain.com doveadm(myuser at mydomain.com): Debug: acl: owner = 1 doveadm(myuser at mydomain.com): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls doveadm(myuser at mydomain.com): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u doveadm(myuser at mydomain.com): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= doveadm(myuser at mydomain.com): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 doveadm(myuser at mydomain.com): Debug: acl: acl username = myuser at mydomain.com doveadm(myuser at mydomain.com): Debug: acl: owner = 0 doveadm(myuser at mydomain.com): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=276 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=277 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=278 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=279 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=280 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=281 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=282 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=283 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=284 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=285 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=286 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=287 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=288 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=289 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=290 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=291 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=292 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=293 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=294 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=295 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=296 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=297 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=298 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=299 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=300 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=301 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=302 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=303 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=304 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=305 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=306 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=307 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=308 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=309 doveadm(myuser at mydomain.com): Debug: expunge: box=Trash uid=310 doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found doveadm(myuser at mydomain.com): Debug: Namespace : Using permissions from /home/vpopmail/domains/2/mydomain.com/myuser/Maildir: mode=0700 gid=-1 doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Drafts/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Spam/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Sent/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/.Trash/dovecot-acl not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found doveadm(myuser at mydomain.com): Debug: acl vfile: file /home/vpopmail/domains/2/mydomain.com/myuser/Maildir/dovecot-acl not found Segmentation fault If I disable "acl" plugin in dovecot all works fine. I think that this problem is connected with "Dovecot deliver Segmentation fault when arrive the first message" http://www.dovecot.org/list/dovecot/2012-September/068343.html that I'm still experiencing. How can get core dumps from "doveadm"? This is my dovecot config with acl enabled: # 2.1.10: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /usr/local/dovecot-2.1/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } ssl_cert = References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> Message-ID: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> On 13.10.2012, at 11.16, Alessio Cecchi wrote: > I'm running dovecot 2.1.10 on Debian 6. > > When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" > > [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] The most helpful way to get this fixed is to get a gdb backtrace: http://dovecot.org/bugreport.html From alessio at skye.it Sat Oct 13 13:48:46 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 12:48:46 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= In-Reply-To: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> Message-ID: <9a747f967d6b70da5a1551a82a017112@skye.it> Il 2012-10-13 10:42 Timo Sirainen ha scritto: > On 13.10.2012, at 11.16, Alessio Cecchi wrote: > >> I'm running dovecot 2.1.10 on Debian 6. >> >> When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it >> crash with "Segmentation fault" >> >> [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 >> sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] > > The most helpful way to get this fixed is to get a gdb backtrace: > http://dovecot.org/bugreport.html Hi Timo, I'm unable to get core dump from doveadm, I start dovecot after run "ulimit -c unlimited" and set echo "/tmp/%p" > /proc/sys/kernel/core_pattern, so core dumps is enable: Oct 13 12:38:02 master: Info: Dovecot v2.1.10 starting up Oct 13 12:38:18 auth-worker(5000): Info: mysql(localhost): Connected to database vpopmail Oct 13 12:38:18 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:38:26 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:38:29 dict: Info: mysql(109.168.113.139): Connected to database dovecot Oct 13 12:39:51 dict: Info: mysql(109.168.113.139): Connected to database dovecot but when doveadm stops with "Segmentation fault" I'm unable to find any dump file and no information in dovecot.log. Can you help me? Thanks From c at roessner-network-solutions.com Sat Oct 13 14:22:30 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Sat, 13 Oct 2012 13:22:30 +0200 Subject: [Dovecot] Help! In-Reply-To: <50784F99.5080201@indietorrent.org> References: <50784F99.5080201@indietorrent.org> Message-ID: Hi, >> I have installed sendmail, dovecot, and squirrel mail. The squirrel >> mail portion of it works just fine, but I would like to have Mozilla >> Thunderbird as a client. Whenever I try and connect to the server it >> says "Thunderbird failed to find the settings for your email account." >> We do have an MX record in DNS pointing to our server. We are using >> Ubuntu 12.0.4.1, and have Dovecot 2.2.6 sendmail version 8.14.4-2ubuntu2 >> We are doing this for a class project and it is due by Tuesday 10/16/2012.. > > I have found Thunderbird's automatic setting detection mechanism to be > rather unreliable. > > Try entering the settings manually. I am one of the automx developers. Have a look at http://www.automx.org. It is open source. I also finished setting up a test server, so you can try with mail address automx at automx.org, pw: automx and see how it works. Kind regards -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From alessio at skye.it Sat Oct 13 15:38:41 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sat, 13 Oct 2012 14:38:41 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> Il 2012-10-02 21:28 Timo Sirainen ha scritto: > On 19.9.2012, at 16.07, Alessio Cecchi wrote: > >> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > > This backtrace is rather weird. Could you also do (instead of bt > full): > > fr 1 > p *ns > p *ns.user > p *auser > > It crashes because auser->dict = NULL, but it should never be NULL. Hi Timo, this is a new backtrace: root at demo-vpop ~ # /home/vpopmail/bin/vadduser test160 at qboxdns.it qweqweroot at demo-vpop ~ # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test160 at qboxdns.it Segmentation fault (core dumped) root at demo-vpop ~ # gdb /usr/local/dovecot-2.1/libexec/dovecot/deliver /tmp/10923 GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/local/dovecot-2.1/libexec/dovecot/deliver...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-lda.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-lda.so.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libbz2.so.1.0...(no debugging symbols found)...done. Loaded symbols for /lib/libbz2.so.1.0 Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib90_sieve_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib90_sieve_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/libdovecot-sieve.so.0...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/libdovecot-sieve.so.0 Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver -d test160 at qboxdns.it'. Program terminated with signal 11, Segmentation fault. #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 221 if (dict->dict == NULL) (gdb) fr 1 #1 0x00007f9edac761b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x2496520) at acl-backend-vfile-acllist.c:297 297 (void)acl_lookup_dict_rebuild(auser->acl_lookup_dict); (gdb) p *ns $1 = {next = 0x2496860, refcount = 1, type = NAMESPACE_PRIVATE, flags = 8235, prefix = 0x24961b0 "", prefix_len = 0, alias_for = 0x0, alias_chain_next = 0x0, user = 0x249a700, owner = 0x249a700, list = 0x249ef40, storage = 0x2496210, set = 0x249b4f0, unexpanded_set = 0x249ab58, mail_set = 0x249b200, destroyed = 0} (gdb) p *ns.user $2 = {pool = 0x249a6e0, v = {deinit = 0x7f9edac7a280 }, vlast = 0x249bf38, refcount = 1, username = 0x249a7b8 "test160 at qboxdns.it", _home = 0x249bb60 "/home/vpopmail/domains/qboxdns.it/test160", uid = 89, gid = 89, service = 0x249bb90 "lda", local_ip = 0x0, remote_ip = 0x0, var_expand_table = 0x249bb98, error = 0x0, set_info = 0x2482ce8, unexpanded_set = 0x249a7d0, set = 0x249b168, namespaces = 0x2496130, storages = 0x24a1e20, hooks = {arr = {buffer = 0x249beb0, element_size = 8}, v = 0x249beb0, v_modifiable = 0x249beb0}, mountpoints = 0x0, module_contexts = {arr = {buffer = 0x249bb00, element_size = 8}, v = 0x249bb00, v_modifiable = 0x249bb00}, home_looked_up = 1, admin = 0, autocreated = 0, initialized = 1, mail_debug = 0, inbox_open_error_logged = 0, fuzzy_search = 0, dsyncing = 0} (gdb) p *auser $3 = {module_ctx = {super = {deinit = 0x7f9edaa68190 }, reg = 0x7f9edaa68190}, master_user = 0x0, acl_env = 0x249bd88 "vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300", groups = 0x0, acl_lookup_dict = 0x0} (gdb) and also but full for safety: (gdb) bt full #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 ns = ids_arr = {arr = {buffer = 0x0, element_size = 38363440}, v = 0x0, v_modifiable = 0x0} ids = 0x24787e0 i = dest = ret = -601327851 #1 0x00007f9edac761b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x2496520) at acl-backend-vfile-acllist.c:297 auser = 0x249bf10 iter = 0x0 acllist_path = 0x24787e0 "/home/vpopmail/domains/qboxdns.it/test160/Maildir/dovecot-acl-list" ret = ns = 0x2496130 output = 0x0 st = {st_dev = 2051, st_ino = 663856, st_nlink = 1, st_mode = 33152, st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1350131151, tv_nsec = 0}, st_mtim = {tv_sec = 1350131151, tv_nsec = 0}, st_ctim = {tv_sec = 1350131151, tv_nsec = 0}, __unused = {0, 0, 0}} path = 0x24783a8 ---Type to continue, or q to quit--- file_mode = 384 dir_mode = 448 gid = 4294967295 list = info = rootdir = 0x24787a0 "Sent" origin = 0x249f4c0 "/home/vpopmail/domains/qboxdns.it/test160/Maildir" fd = 8 #2 acl_backend_vfile_acllist_rebuild (backend=0x2496520) at acl-backend-vfile-acllist.c:311 acllist_path = #3 0x00007f9edac76563 in acl_backend_vfile_acllist_refresh (backend=0x2496520) at acl-backend-vfile-acllist.c:153 __FUNCTION__ = "acl_backend_vfile_acllist_refresh" #4 0x00007f9edac766d5 in acl_backend_vfile_acllist_verify (backend=0x0, name=0x2496800 "", mtime=0) at acl-backend-vfile-acllist.c:343 acllist = #5 0x00007f9edac750b8 in acl_backend_vfile_object_refresh_cache ( _aclobj=0x24967c0) at acl-backend-vfile.c:858 old_validity = validity = {global_validity = {last_check = 0, last_read_time = 1350131151, last_mtime = 0, last_size = 0}, local_validity = {last_check = 0, last_read_time = 0, ---Type to continue, or q to quit--- last_mtime = 0, last_size = 0}, mailbox_validity = { last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}} mtime = 0 ret = 38387472 #6 0x00007f9edac7325e in acl_backend_get_default_rights (backend=0x2496520, mask_r=0x28) at acl-backend.c:164 No locals. #7 0x00007f9edac795bd in acl_mailbox_try_list_fast (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:107 alist = nonowner_list_ctx = ret = backend = 0x2496520 acl_mask = 0x1 ns = 0x2496130 update_ctx = {iter_ctx = 0x7f9edc4bf2c8, tree_ctx = 0x7f9edcbdda88, glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, update_only = 0, match_parents = 0} name = #8 acl_mailbox_list_iter_init (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 _data_stack_cur_id = 2 ---Type to continue, or q to quit--- ctx = 0x2498e60 pool = i = inboxcase = #9 0x00007f9edc538d33 in mailbox_list_iter_init_multiple (list=0x249ef40, patterns=0x7fff89037330, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at mailbox-list-iter.c:158 ctx = ret = __FUNCTION__ = "mailbox_list_iter_init_multiple" #10 0x00007f9edc539459 in mailbox_list_iter_init (list=0x0, pattern=, flags=1350131151) at mailbox-list-iter.c:58 patterns = {0x7f9edaa696dc "*", 0x0} #11 0x00007f9edaa64370 in quota_count_namespace (root=0x2496cb0, bytes_r=, count_r=0x7fff890373d0) at quota-count.c:73 ctx = 0x7f9edc270ef3 info = #12 quota_count (root=0x2496cb0, bytes_r=, count_r=0x7fff890373d0) at quota-count.c:111 i = 0 ret = 0 #13 0x00007f9edaa657ce in dict_quota_count (root=0x0, want_bytes=true, value_r=0x7fff89037418) at quota-dict.c:113 ---Type to continue, or q to quit--- dt = bytes = 0 count = 0 #14 0x00007f9edaa6595a in dict_quota_update_callback ( ret=, context=0x249bf10) at quota-dict.c:178 value = 1 #15 0x00007f9edc244258 in client_dict_finish_transaction (dict=0x249eb30, line_r=) at dict-client.c:265 ctx = 0x24da1b0 #16 client_dict_read_one_line (dict=0x249eb30, line_r=) at dict-client.c:356 id = 1 line = ret = 0 __FUNCTION__ = "client_dict_read_one_line" #17 0x00007f9edc244565 in client_dict_wait (_dict=) at dict-client.c:520 dict = 0x249eb30 line = 0x0 ret = #18 0x00007f9edaa65ab5 in dict_quota_deinit (_root=) at quota-dict.c:90 root = 0x2496cb0 ---Type to continue, or q to quit--- #19 0x00007f9edaa61c72 in quota_root_deinit (root=0x0) at quota.c:240 pool = 0x249e900 #20 0x00007f9edaa636e1 in quota_deinit (_quota=0x249bf40) at quota.c:335 quota = 0x2496940 i = 2 #21 0x00007f9edaa681dd in quota_user_deinit (user=0x249a700) at quota-storage.c:412 quser = 0x249bf38 quota_set = 0x2499270 #22 0x00007f9edc53388e in mail_user_unref (_user=) at mail-user.c:153 user = 0x249a700 __FUNCTION__ = "mail_user_unref" #23 0x0000000000402de2 in main (argc=3, argv=0x247e370) at main.c:481 set_roots = {0x604640, 0x0} ctx = {pool = 0x247ef70, set = 0x24817e8, session = 0x247ef90, dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, src_envelope_sender = 0x0, dest_user = 0x0, dest_addr = 0x247e3c2 "test160 at qboxdns.it", final_dest_addr = 0x247e3c2 "test160 at qboxdns.it", dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tried_default_save = true, saved_mail = false, save_dest_mail = false, mailbox_full = false, ---Type to continue, or q to quit--- dsn = false} service_flags = user = 0x247e3c2 "test160 at qboxdns.it" errstr = 0x0 path = 0x7fff89037748 "\351\a" storage_service = 0x24803b0 service_user = 0x2480d58 service_input = {module = 0x4034d5 "lda", service = 0x4034d5 "lda", username = 0x247e3c2 "test160 at qboxdns.it", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = 0x2496210 user_source = destaddr_source = 0x403594 "user at hostname" process_euid = stderr_rejection = false ---Type to continue, or q to quit--- ret = c = error = MAIL_ERROR_NONE (gdb) and this the dovecot configuration: # dovecot -n # 2.1.9: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_size = 512 k auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota zlib acl maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=109.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_default = /usr/local/dovecot-2.1/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } ssl_cert = References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <83B37619-1CE8-4C5D-8147-A3C0E1C99CDC@iki.fi> Message-ID: <6730bf79a50779c9bd33311e50ccce9e@skye.it> Il 2012-10-02 22:15 Timo Sirainen ha scritto: > On 2.10.2012, at 22.28, Timo Sirainen wrote: > >> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >> >>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 >> >> This backtrace is rather weird. Could you also do (instead of bt >> full): > > Also, can you reproduce the crash always by running "doveadm quota > recalc -u user at domain"? If first add a news user, than run quota recalc and after deliver the first message "deliver" not crash: # vpopmail/bin/vadduser test10 at qboxdns.it # doveadm quota recalc -u test110 at qboxdns.it # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test10 at qboxdns.it When add a new user without quota recalc deliver crash: # /home/vpopmail/bin/vadduser test12 at qboxdns.it # cat /root/testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d test12 at qboxdns.it Segmentation fault (core dumped) # Hope this will useful From simon.buongiorno at gmail.com Sun Oct 14 05:07:29 2012 From: simon.buongiorno at gmail.com (simon.buongiorno at gmail.com) Date: Sat, 13 Oct 2012 22:07:29 -0400 Subject: [Dovecot] POP UIDL Message-ID: Hi I run a small mail server with Dovecot and postfix. Several of the accounts are popped by an external email provide I use, mostly out habit and also for a bit of redundancy. I have no details on that external server except that it uses exim. Lately, it's been repopping mail from accounts (mail is left on the server so I can use IMAP) at fairly frequent, but undetermined, intervals. Before I take it to them, I want to be sure it's not an error on my side. How can I be sure Dovecot does not have a problem with the UIDL list causing this external server to repop the mail? For the record, I'm not inclined to think it's a Dovecot issue, but since that's the bit I can fix, I'd like to be sure. Cheers Simon From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 16:52:18 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 15:52:18 +0200 Subject: [Dovecot] [sieve] - counting headers In-Reply-To: <87wqyw6rv6.fsf@alfa.kjonca> References: <87wqyw6rv6.fsf@alfa.kjonca> Message-ID: <20121014135218.GA7602@daniel.localdomain> Kamil Jo?ca wrote: > In some of my maildrop filters I have rules with weighted scoring[1], > but only to count headers (for example to count "Received:" header) > ie. all these rules are of form "/pattern/:h,1" > > Can dovecot sieve do this? http://tools.ietf.org/rfc/rfc5231.txt To check the number of received fields in the header, the following test may be used: header :count "ge" :comparator "i;ascii-numeric" ["received"] ["3"] Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 17:24:22 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 16:24:22 +0200 Subject: [Dovecot] /var/run/dovecot/auth-userdb failed In-Reply-To: <1350056339814-38093.post@n4.nabble.com> References: <1350056339814-38093.post@n4.nabble.com> Message-ID: <20121014142422.GA8080@daniel.localdomain> thefantaman wrote: > I work on test server and if I send an email on log i read > > lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: > Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: > /var/run/dovecot/auth-userdb, euid is not dir owner) > > unix_listener auth-userdb { > mode = 0600 > user = root > group = root > } > } The problem is that LDA (local delivery agent or lmtp service) is not able to look up the destination mailbox in userdb. The socket /var/run/dovecot/auth-userdb is currently only readable or writable by user root since mode is set to 0600, not readable or writable by other groups. http://wiki2.dovecot.org/LDA#Virtual_users You'll need to set up a auth-userdb socket for dovecot-lda so it knows where to find mailboxes for the users. LDA is running under the virtual mailbox user and group "vmail", so you need to grant this user or group access to /var/run/dovecot/auth-userdb. You could do this by using group memberships and set mode = 0660 or simply make it world-readable-writable with mode = 0666: unix_listener auth-userdb { mode = 0666 user = root group = root } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 14 17:38:30 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 14 Oct 2012 16:38:30 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: <9a747f967d6b70da5a1551a82a017112@skye.it> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> Message-ID: <20121014143830.GA8425@daniel.localdomain> Alessio Cecchi wrote: > I'm unable to get core dump from doveadm, I start dovecot after run > "ulimit -c unlimited" and set echo "/tmp/%p" > > /proc/sys/kernel/core_pattern, so core dumps is enable: > > but when doveadm stops with "Segmentation fault" I'm unable to find > any dump file and no information in dovecot.log. On Debian try to enable coredumps in /etc/default/dovecot and start dovecot as usual via init script. Also watch out for core dumps and segfaults in /var/log/kern.log Regards Daniel -- https://plus.google.com/103021802792276734820 From alessio at skye.it Sun Oct 14 18:58:40 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sun, 14 Oct 2012 17:58:40 +0200 Subject: [Dovecot] =?utf-8?q?Segmentation_fault_in_doveadm_with_lib01=5Fac?= =?utf-8?q?l=5Fplugin=2Eso?= In-Reply-To: <20121014143830.GA8425@daniel.localdomain> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> <20121014143830.GA8425@daniel.localdomain> Message-ID: Il 2012-10-14 16:38 Daniel Parthey ha scritto: > Alessio Cecchi wrote: >> I'm unable to get core dump from doveadm, I start dovecot after run >> "ulimit -c unlimited" and set echo "/tmp/%p" > >> /proc/sys/kernel/core_pattern, so core dumps is enable: >> >> but when doveadm stops with "Segmentation fault" I'm unable to find >> any dump file and no information in dovecot.log. > > On Debian try to enable coredumps in /etc/default/dovecot and start > dovecot as usual via init script. Also watch out for core dumps > and segfaults in /var/log/kern.log Thanks, my dovecot installation is build from source. Dovecot start fine with core dumps enabled but doveadm don't return "Core dumped" when crash. From dave at boostpro.com Sun Oct 14 21:30:10 2012 From: dave at boostpro.com (Dave Abrahams) Date: Sun, 14 Oct 2012 14:30:10 -0400 Subject: [Dovecot] Search for substring in header? Message-ID: Hi, According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), if I do a search for "TO isocpp.org" it should find all the messages whose To: field contains the string "isocpp.org", but dovecot is returning me an empty list. However, a search for "TO tm at isocpp.org" produces a long list of messages. What am I doing wrong? TIA, Dave -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From slusarz at curecanti.org Mon Oct 15 04:59:50 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Sun, 14 Oct 2012 19:59:50 -0600 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: Message-ID: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Quoting Dave Abrahams : > Hi, > > According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), > if I do a search for "TO isocpp.org" it should find all the messages > whose To: field contains the string "isocpp.org", but dovecot is > returning me an empty list. However, a search for "TO tm at isocpp.org" > produces a long list of messages. What am I doing wrong? First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. Second, your assumption is correct - TO should do a substring search. But this works fine for me (using version 2.1.10). michael From sandro.tosi at dada.eu Mon Oct 15 10:40:48 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Mon, 15 Oct 2012 09:40:48 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50772D89.4050601@rename-it.nl> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> Message-ID: <507BBE00.9010007@dada.eu> Hi Stephan, thanks a lot for your reply. On 10/11/2012 10:35 PM, Stephan Bosch wrote: > On 10/10/2012 11:23 AM, Sandro Tosi wrote: >> Hello, >> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >> as this will be provided with an upcoming Debian package) with MySQL >> dict lookups with the mail setup we're designing. >> >> Our (main) goals are: >> >> 1. store the filters on the database > That is possible with some limitations. Are the ones below the only limitatios (ie one script per user) or are there any other worth knowing? >> 2. allow each user to enable/disable any of the filters set we provide >> (it's a static set of some general filters, available to all the >> users; we're currently not providing the possibility to users to write >> their own filters) > Will one or multiple scripts be active at the same time? Yep, the idea is that any user could have multiple scripts active at the same time, and we'd like also to give them an ordering, so like managing a sort of priority (the lower the priority the sooner the script is executed, or the other way around, doesn't matter). Ideally, we have a set of several scripts and each user can select to enable only some of them, and choose the order of their executions. >> For point 1) we already see[1] that's possible, but it uses the map >> construct that might not fit with our current database structure: we >> have a domain table (storing the domain info) and a mailbox table >> (storing the mailbox info, but the username is composed by the local >> part, stored in this table, and the domain part is a FK to the domain >> table, using an id). >> >> Do you think it's possible to run a join query on domain+mailbox to >> retrieve the mailbox_id needed to query the table for the filters? Or >> do we have to create the filter table and store the local at domain.ext >> info there ("relaxing" the integrity relationships between tables)? > > My SQL is a bit rusty, but afaik this is possible with a JOIN or a > nested query. Ah no well, I mean, using map { } constructs :) The example for Sieve-MySQL only shows 2 maps, but given we've never used them, we'd want to know if a "map cascade" would work, so implementing the joins in multiple steps: selecting the ids with a map and the subsequent would use that id to exec the join and so on. >> How do we specify which filters are enabled for any given user? We >> originally thought of an "Enabled" field on the filter table, but in >> the example in the doc[1] I hadn't seen a way to do that: it seems >> like the filter list is specified in the proxy definition - am I >> wrong? How can we do that? > > The above suggests that you would like to activate multiple Sieve > scripts at the same time. That is currently not possible with the dict > Script location. It is on my TODO list, but I am not sure when it will > be ready (definitely not for coming Debian stable). I see, I think that some others would wonder the same, so you might also want to extend the doc to state that explicitly. Maybe you may want to include something in your TODO list to handle the ordering in case of multiple scripts. In our situation, what would you suggest? We're now thinking of keeping the scripts list on a separate table, and merge the "user selected ones" in a single script to write in the filters table. Is that what would you suggest? Is there a better solution? Cheers, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From stocton12 at yahoo.com Mon Oct 15 15:46:09 2012 From: stocton12 at yahoo.com (b m) Date: Mon, 15 Oct 2012 05:46:09 -0700 (PDT) Subject: [Dovecot] (no subject) Message-ID: <1350305169.43664.YahooMailNeo@web125703.mail.ne1.yahoo.com> Hi. I'm using dovecot 2.0.18 and I'm trying to authenticate through a CAS server (until now authentication was through MS Active Directory). I could not find anywhere some examples, so here is what i have done so far. -install phpcas and pam_cas -edit /etc/pam.d/dovecot ????????????????? auth??? sufficient????? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf -edit /etc.pam_cas.conf ????????????????? host mycas.mydomain.com ????????????????? port 443 ????????????????? uriValidate /cas/proxyValidate ????????????????? ssl on ????????????????? proxy ??????????????????????? ????????????????? trusted_ca /etc/cert/certificate.pem ????????????????? debug on - and finally dovecot.conf which I'm sure is complety wrong ????????????? userdb { ? ? ? ? ?? ?? args = /etc/dovecot/dovecot-ldap.conf ? ? ? ? ?? ?? driver = ldap ????????????? } ???????????? passdb { ???????????? driver = pam ? ? ? ? ? ?? args = cache_key=%u dovecot ???????????? } What I get in log is Oct 15 15:39:58 auth-worker: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): lookup service=dovecot Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): #1/1 style=1 msg=Password: Oct 15 15:39:58 auth-worker: Info: pam(user,127.0.0.1): pam_authenticate() failed: Permission denied Oct 15 15:40:00 auth: Debug: client out: FAIL??? 1??? user=user Oct 15 15:40:00 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 15 15:40:00 auth: Debug: auth client connected (pid=9019) Any ideas? Thanks. From linuxpencil at hotmail.com Mon Oct 15 16:01:04 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 09:01:04 -0400 Subject: [Dovecot] Can't Start Dovecot Message-ID: Hi; I just installed dovecot from yum on CentOS5. ps wax grep "dovecot" only brings up the grep The command "dovecot" is not recognized. # ls /usr/local/bin/dove* doveadm doveconf No dovecot. What up? TIA, John From s.lazzaris at interactive.eu Mon Oct 15 16:13:45 2012 From: s.lazzaris at interactive.eu (Simone Lazzaris) Date: Mon, 15 Oct 2012 15:13:45 +0200 Subject: [Dovecot] Plugin hooks in login process Message-ID: <1947528.35zxeZD9k1@orion> Hi all; I've setup dovecot (2.1.10) in a cluster configuration. We have two servers acting as frontend which authenticates users and proxy them to other two servers which handles the "real" work. Users credentials are on a mysql cluster; we have one master, in which read/write queries are processed, and many replicated slave, which process read-only queries. The frontend servers reads users credentials from the read-only mysql slaves. I'd like to execute a query once the client is verified to update the last login data. Right now, that query is executed on the backend servers, via a post-login service: protocols = imap service imap-postlogin { executable = script-login /usr/local/etc/dovecot/postlogin.sh unix_listener imap-postlogin { group = vchkpw mode = 0600 user = vpopmail } } service imap { executable = imap imap-postlogin process_limit = 2048 } Problem is, if I execute the update on the backend, I miss the information regarding the original IP, as I only see the IP of the proxies. I haven't been able to launch the postlogin service on the frontend, so I figured that I can try to write a plugin - that also seems to me the cleanest solution. Looking in the dovecot source code, I noticed that there aren't any hooks in the execution path used by the proxies; I am missing something ? I am the only one missing the presence of this hooks in the auth/proxy process ? I've also thought of a workaround for this problem. One way is to monitor the dovecot logs on the frontend and execute the update asyncronously. One other way is to query directly the main mysql server of the cluster and adjust the query making it call a stored procedure that updates the information in case of successful login. But I'd really prefer to create a plugin, that I'd be willing to share. I attach the configuration of the servers (front and back) generated via postfix -n. Thanks in advance for any help. -- Simone Lazzaris | Responsabile aree datacenter e VoIP Interactive Network srl | via Roggia Vignola 9, 24047 Treviglio (BG) Tel. 0363 1970352 | Fax 0363.1971971 | www.interactive.eu -------------- next part -------------- # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686-bigmem i686 Debian 6.0.2 auth_debug = yes auth_verbose = yes auth_verbose_passwords = plain base_dir = /var/run/dovecot/ default_login_user = nobody director_doveadm_port = 9091 director_mail_servers = AAA.BBB.CCC.DDD EEE.FFF.GGG.HHH director_servers = XXX.YYY.ZZZ.WWW disable_plaintext_auth = no listen = * log_path = /var/log/dovecot passdb { args = /usr/local/etc/dovecot/sql.conf driver = sql } protocols = imap service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director service_count = 0 } ssl_cert = From dave at boostpro.com Mon Oct 15 16:23:08 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 06:23:08 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: on Sun Oct 14 2012, Michael M Slusarz wrote: > Quoting Dave Abrahams : > >> Hi, >> >> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >> if I do a search for "TO isocpp.org" it should find all the messages >> whose To: field contains the string "isocpp.org", but dovecot is >> returning me an empty list. However, a search for "TO tm at isocpp.org" >> produces a long list of messages. What am I doing wrong? > > First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. Thanks for pointing me to the right one. > Second, your assumption is correct - TO should do a substring search. > But this works fine for me (using version 2.1.10). Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the same empty result with either of these two commands: UID SEARCH TO isocpp.org UID SEARCH TO "isocpp.org" Am I formatting the command wrongly? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From linuxpencil at hotmail.com Mon Oct 15 16:37:09 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 09:37:09 -0400 Subject: [Dovecot] Can't Start Dovecot In-Reply-To: References: Message-ID: Never mind. The command /etc/init.d/dovecot start would work; however, something else is using the port. Tracking it down. John > From: linuxpencil at hotmail.com > To: dovecot at dovecot.org > Date: Mon, 15 Oct 2012 09:01:04 -0400 > Subject: [Dovecot] Can't Start Dovecot > > > > Hi; > I just installed dovecot from yum on CentOS5. > ps wax grep "dovecot" only brings up the grep > The command "dovecot" is not recognized. > # ls /usr/local/bin/dove* > doveadm doveconf > No dovecot. What up? > TIA, > John > From h.reindl at thelounge.net Mon Oct 15 16:42:00 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 15 Oct 2012 15:42:00 +0200 Subject: [Dovecot] Can't Start Dovecot In-Reply-To: References: Message-ID: <507C12A8.1040702@thelounge.net> netstat --numeric-hosts --numeric-ports --programs -u -t -l will list all listening ports and as root also the exectueable /etc/init.d/dovecot star is they way to go never start a service by it's binary without a good reason without knowing exactly how it is supposed to work BTW: the dovecot binary lives in /sbin/ not /bin/ [root at srv:~]$ ps aux | grep dovecot root 1843 0.0 0.0 19548 1520 ? Ss 11:58 0:00 /usr/sbin/dovecot -F i am generally wonder about /usr/local as you said you installed with yum - typically distributions packages are using /usr/bin, /usr/sbin/ and not /usr/local Am 15.10.2012 15:37, schrieb John Reddy: > Never mind. The command > /etc/init.d/dovecot start > would work; however, something else is using the port. Tracking it down. > John > >> From: linuxpencil at hotmail.com >> To: dovecot at dovecot.org >> Date: Mon, 15 Oct 2012 09:01:04 -0400 >> Subject: [Dovecot] Can't Start Dovecot >> >> >> >> Hi; >> I just installed dovecot from yum on CentOS5. >> ps wax grep "dovecot" only brings up the grep >> The command "dovecot" is not recognized. >> # ls /usr/local/bin/dove* >> doveadm doveconf >> No dovecot. What up? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From list at airstreamcomm.net Mon Oct 15 17:36:11 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 15 Oct 2012 09:36:11 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> Message-ID: <507C1F5B.2060002@airstreamcomm.net> On 10/12/12 2:40 AM, Timo Sirainen wrote: > On 10.10.2012, at 17.37, Jack Bates wrote: > >> The logging on lmtp and lmtp proxy is pretty limited from what I can see. It seems to handle errors, Connect, Disconnect, and in the case of lmtp delivery, it logs where an email is saved to. The lmtp may be enough, "connect, saved user, saved user..., disconnect", but I was curious if it is worth while to add more info logging for the proxy, primarily which recipients are sent to which proxy. I was thinking of local patching it, but I'll generate up something more inline with official code if it is desired. >> >> My thought is to show 1 entry for each recipient, and the destination server chosen. If I recall correctly, the proxy code doesn't actually listen in on the conversation, so logging results would probably complicate the code. > I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. > > +1 for adding this detail to logging for LMTP. From dave at boostpro.com Mon Oct 15 18:08:59 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 08:08:59 -0700 Subject: [Dovecot] fts = squat solr Message-ID: I don't know if this was supposed to have changed with dovecot2, but http://wiki.dovecot.org/Plugins/FTS shows fts = squat solr so, since I have the lucene plugin?"fts = lucene" works by itself?I tried fts = squat lucene but: $ doveadm index '*' doveadm(dave): Error: fts: Failed to initialize backend 'squat lucene': Unknown backend So, is that syntax obsolete, is the wiki wrong, or am I doing something wrong? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Mon Oct 15 18:36:30 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 08:36:30 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: on Mon Oct 15 2012, Dave Abrahams wrote: > on Sun Oct 14 2012, Michael M Slusarz wrote: > >> Quoting Dave Abrahams : >> >>> Hi, >>> >>> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >>> if I do a search for "TO isocpp.org" it should find all the messages >>> whose To: field contains the string "isocpp.org", but dovecot is >>> returning me an empty list. However, a search for "TO tm at isocpp.org" >>> produces a long list of messages. What am I doing wrong? >> >> First, you referenced the wrong RFC - RFC 2060 has been obsoleted by RFC 3501. > > Thanks for pointing me to the right one. > >> Second, your assumption is correct - TO should do a substring search. >> But this works fine for me (using version 2.1.10). > > Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the > same empty result with either of these two commands: > > UID SEARCH TO isocpp.org > > UID SEARCH TO "isocpp.org" > > Am I formatting the command wrongly? Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the same result. baffled-ly y'rs, -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From howellrepaja at gmail.com Mon Oct 15 18:43:25 2012 From: howellrepaja at gmail.com (Howell Repaja) Date: Mon, 15 Oct 2012 23:43:25 +0800 Subject: [Dovecot] Dovecot Authentication Problem Can't Make it Work Message-ID: Hi All, I am struggling for 2 weeks solving authentication problem in dovecot. logs from /etc/mail/maillog Oct 15 18:00:35 localhost dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 18:00:35 localhost dovecot: auth: Debug: auth client connected (pid=26723) Oct 15 18:00:35 localhost dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011lip=10.0.0.123#011rip=88.22.197.66#011lport=110#011rport=2358#011resp=AGhvd2VsbEB0b3VyZm9yeW91LmluZm8AanVtb25n Oct 15 18:00:35 localhost dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 18:00:35 localhost dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 18:00:35 localhost dovecot: auth: Debug: pam(howell at mydomain.info,88.22.33.66): lookup service=dovecot Oct 15 18:00:35 localhost dovecot: auth: Debug: pam(howell at mydomain.info,88.22.197.66): #1/1 style=1 msg=Password: Oct 15 18:00:37 localhost dovecot: auth: pam(howell at mydomain.info,88.22.197.66): unknown user Oct 15 18:00:39 localhost dovecot: auth: Debug: client out: FAIL#0111#011user=howell at mydomain.info#011reason=Password : Oct 15 18:00:39 localhost dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=>, method=PLAIN, rip=88.22.197.66, lip=10.0.0.123 Oct 15 18:01:05 localhost sendmail[26722]: q9FA15LB026722: [88.22.197.66] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.el6.x86_64 x86_64 CentOS release 6.3 (Final) ext4 auth_debug_passwords = yes auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb disable_plaintext_auth = no last_valid_gid = 10 last_valid_uid = 650 listen = * login_greeting = Dovecot ready for you. mail_debug = yes mail_location = mbox:/var/spool/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = setcred=yes failure_show_msg=yes cache_key=%u dovecot driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } service imap-login { inet_listener imap { port = 143 } } service pop3-login { inet_listener pop3 { port = 110 } } ssl_cert = Hi.I'm very sorry for the repost but I forgot the subject. So,? I'm using dovecot 2.0.18 and I'm trying to authenticate through a CAS server (until now authentication was through MS Active Directory). I could not find anywhere some examples, so here is what i have done so far. -install phpcas and pam_cas -edit /etc/pam.d/dovecot ????????????????? auth??? sufficient????? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf -edit /etc.pam_cas.conf ????????????????? host mycas.mydomain.com ????????????????? port 443 ????????????????? uriValidate /cas/proxyValidate ????????????????? ssl on ????????????????? proxy ??????????????????????? ????????????????? trusted_ca /etc/cert/certificate.pem ????????????????? debug on - and finally dovecot.conf which I'm sure is complety wrong ????????????? userdb { ? ? ? ? ?? ?? args = /etc/dovecot/dovecot-ldap.conf ? ? ? ? ?? ?? driver = ldap ????????????? } ???????????? passdb { ???????????? driver = pam ? ? ? ? ? ?? args = cache_key=%u dovecot ???????????? } What I get in log is Oct 15 15:39:58 auth-worker: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_pgsql.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Oct 15 15:39:58 auth-worker: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): lookup service=dovecot Oct 15 15:39:58 auth-worker: Debug: pam(user,127.0.0.1): #1/1 style=1 msg=Password: Oct 15 15:39:58 auth-worker: Info: pam(user,127.0.0.1): pam_authenticate() failed: Permission denied Oct 15 15:40:00 auth: Debug: client out: FAIL??? 1??? user=user Oct 15 15:40:00 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 15 15:40:00 auth: Debug: auth client connected (pid=9019) Any ideas? Thanks. From user+dovecot at localhost.localdomain.org Mon Oct 15 20:04:19 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 15 Oct 2012 19:04:19 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> <9a747f967d6b70da5a1551a82a017112@skye.it> <20121014143830.GA8425@daniel.localdomain> Message-ID: <507C4213.8000205@localhost.localdomain.org> On 10/14/2012 05:58 PM Alessio Cecchi wrote: > Thanks, my dovecot installation is build from source. Dovecot start > fine with core dumps enabled but doveadm don't return "Core dumped" when > crash. In your terminal emulator enter the following commands: ulimit -c unlimited doveadm ? Regards, Pascal -- The trapper recommends today: decade.1228919 at localdomain.org From linuxpencil at hotmail.com Mon Oct 15 21:10:12 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 14:10:12 -0400 Subject: [Dovecot] Where'd the Mail Go? Message-ID: Hi; I ran this: echo "Hello me" | mail -s "Dovecot test" $USER then created a bash script: for mbox in /var/mail/$USER /var/spool/mail/$USER ~/mbox ~/mail/* ~/*; do grep -q "Dovecot test" $mbox && echo "mbox: $mbox" done grep -q "Dovecot test" ~/Maildir/new/* 2>/dev/null && echo "Maildir: ~/Maildir" and ran it but couldn't find where the mail went. I checked /var/mail/my_user_name and it wasn't there, either. How do I find it? TIA, John From slusarz at curecanti.org Mon Oct 15 22:00:11 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 15 Oct 2012 13:00:11 -0600 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> Message-ID: <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Quoting Dave Abrahams : > on Mon Oct 15 2012, Dave Abrahams wrote: > >> on Sun Oct 14 2012, Michael M Slusarz wrote: >> >>> Quoting Dave Abrahams : >>> >>>> Hi, >>>> >>>> According to the IMAP spec (http://tools.ietf.org/html/rfc2060#page-37), >>>> if I do a search for "TO isocpp.org" it should find all the messages >>>> whose To: field contains the string "isocpp.org", but dovecot is >>>> returning me an empty list. However, a search for "TO tm at isocpp.org" >>>> produces a long list of messages. What am I doing wrong? >>> >>> First, you referenced the wrong RFC - RFC 2060 has been obsoleted >>> by RFC 3501. >> >> Thanks for pointing me to the right one. >> >>> Second, your assumption is correct - TO should do a substring search. >>> But this works fine for me (using version 2.1.10). >> >> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >> same empty result with either of these two commands: >> >> UID SEARCH TO isocpp.org >> >> UID SEARCH TO "isocpp.org" >> >> Am I formatting the command wrongly? > > Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the > same result. Lucene for sure does not support subtext searching. Squat used to... but IIRC things may have changed for v2.1. Try the wiki. michael From jbates at brightok.net Mon Oct 15 22:07:07 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 15 Oct 2012 14:07:07 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> Message-ID: <507C5EDB.7050401@brightok.net> On 10/12/2012 2:40 AM, Timo Sirainen wrote: > would probably complicate the code. > I don't think this would be difficult to implement. Probably just a few lines of code. Yeah, could be useful. > > Commented logs below. I did 3 different types of connections. Let me know what you think. Because I'm logging the proxy host itself, it can be IP or name depending on the configuration. If you like it, want minor changes, additional logging, let me know and I'll adjust the code. As is, this is a one liner. Jack Oct 12 19:03:45 compiler dovecot: lmtp(18568): Connect from ::1 Connection succeeds using static proxy to lmtp.example.com (default in this config). Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from lmtp.example.com(test): 250 2.5.0 command succeeded Connection succeeds but user invalid using director mapping Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from 192.168.1.3(test2): 550 5.1.1 User doesn't exist: test2 Current error reporting Oct 12 19:04:14 compiler dovecot: lmtp(18568): Error: lmtp client: connect(192.168.1.4, 7025) failed: No route to host Connection failed and what we returned to client Oct 12 19:04:14 compiler dovecot: lmtp(18568): Reply from 192.168.1.4(test42): 451 4.4.0 Remote server not answering (connect) Oct 12 19:04:16 compiler dovecot: lmtp(18568): Disconnect from ::1: Client quit (in reset) From tom at whyscream.net Mon Oct 15 22:56:19 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 15 Oct 2012 21:56:19 +0200 Subject: [Dovecot] Dovecot Authentication Problem Can't Make it Work In-Reply-To: References: Message-ID: <507C6A63.2000301@whyscream.net> On 15/10/12 17:43, Howell Repaja wrote: > Hi All, > > I am struggling for 2 weeks solving authentication problem in dovecot. > > logs from /etc/mail/maillog > Oct 15 18:00:37 localhost dovecot: auth: > pam(howell at mydomain.info,88.22.197.66): > unknown user > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-279.el6.x86_64 x86_64 CentOS release 6.3 (Final) ext4 > passdb { > args = setcred=yes failure_show_msg=yes cache_key=%u dovecot > driver = pam > } > userdb { > driver = passwd > } Pam says that you have no such user 'howell at tourforyou.info'. Pam most probably talks to /etc/passwd (and friends), which means that you either need to login with a valid valid system username listed in /etc/passwd, or you need to setup some other userdb/passdb that supports full email addresses as usernames. -- Tom From jbates at brightok.net Mon Oct 15 23:10:59 2012 From: jbates at brightok.net (Jack Bates) Date: Mon, 15 Oct 2012 15:10:59 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507C5EDB.7050401@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> Message-ID: <507C6DD3.2000309@brightok.net> On 10/15/2012 2:07 PM, Jack Bates wrote: > On 10/12/2012 2:40 AM, Timo Sirainen wrote: >> would probably complicate the code. >> I don't think this would be difficult to implement. Probably just a >> few lines of code. Yeah, could be useful. >> >> > If there's no argument over the last email, confirm and check this patch. It's not the overall logging I would like, but the lmtp code isn't as mature as pop3/imap and the proxy is a quick and dirty on the lmtp code. Both need a good revamp, preferably with x-session support and perhaps logging rip/lip similar to how we do pop3/imap logins. I think we should also work on adjusting all logging for services using x-session to also log the proxy ip. rip,lip,pip. As I get time I'll look at it. This patch is just to keep us from having no useful logging in lmtp proxy. Based on lmtp pid, one can at least follow the connect, the proxy replies, and the disconnect of a session. --- dovecot-2.1.10/src/lmtp/lmtp-proxy.c 2012-10-12 19:46:49.688952484 +0000 +++ dovecot-2.1.10/src/lmtp/lmtp-proxy.c-new 2012-10-12 19:48:51.751932325 +0000 @@ -160,6 +160,8 @@ static bool lmtp_proxy_send_data_replies break; o_stream_send_str(proxy->client_output, t_strconcat(rcpt[i]->reply, "\r\n", NULL)); + i_info("proxy(%s): proxy host=%s: status=%s",rcpt[i]->address, + rcpt[i]->conn->set.host,rcpt[i]->reply); } o_stream_uncork(proxy->client_output); proxy->next_data_reply_idx = i; From daniel.parthey at informatik.tu-chemnitz.de Tue Oct 16 00:27:46 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 15 Oct 2012 23:27:46 +0200 Subject: [Dovecot] Where'd the Mail Go? In-Reply-To: References: Message-ID: <20121015212746.GA8899@daniel.localdomain> Hi John, John Reddy wrote: > I ran this: > echo "Hello me" | mail -s "Dovecot test" $USER > and ran it but couldn't find where the mail went. I checked /var/mail/my_user_name and it wasn't there, either. How do I find it? I'm sorry to tell you this is rather off-topic on the dovecot list, since the mail is routed and delivered by your MTA. In the logfiles of your MTA (mail transport agent) you should find hints where the mail went. /var/log/postfix/... /var/log/exim/... Regards Daniel -- https://plus.google.com/103021802792276734820 From dave at boostpro.com Tue Oct 16 01:20:29 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 15:20:29 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Message-ID: on Mon Oct 15 2012, Michael M Slusarz wrote: > Quoting Dave Abrahams : > >> on Mon Oct 15 2012, Dave Abrahams wrote: >> >>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>> >>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>> same empty result with either of these two commands: >>> >>> UID SEARCH TO isocpp.org >>> >>> UID SEARCH TO "isocpp.org" >>> >>> Am I formatting the command wrongly? >> >> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >> same result. > > Lucene for sure does not support subtext searching. Squat used to... > but IIRC things may have changed for v2.1. Try the wiki. Sorry, but what does "try the wiki" mean? Which indexer are you using, that successfully finds the substring match? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 01:35:06 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 15:35:06 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search Message-ID: According to the IMAP spec if I do a search for "TO isocpp.org" it should find all the messages whose To: field contains the string "isocpp.org", but dovecot is returning me an empty list. However, a search for "TO tm at isocpp.org" produces a long list of messages. This behavior is present if I *even load* the lucene fts plugin. Note that lucene isn't in use (fts = squat); it's merely loaded. This behavior goes away if I don't load fts_lucene. Dovecot configuration with dovecot -n: --8<---------------cut here---------------start------------->8--- # 2.1.6: /usr/local/stow/dovecot-2.1.6/etc/dovecot/dovecot.conf # OS: Darwin 11.4.2 x86_64 hfs default_internal_user = _dovecot default_login_user = _dovenull mail_gid = 20 mail_location = mdbox:/Users/dave/Library/Data/LocalIMAP/mdbox mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = fts fts_squat fts_lucene zlib mail_uid = 501 maildir_very_dirty_syncs = yes namespace { inbox = yes location = prefix = separator = . subscriptions = yes type = private } passdb { args = uid=501 gid=20 home=/Users/dave nopassword=y driver = static } plugin { fts = squat zlib_save = gz zlib_save_level = 6 } protocols = imap ssl = no protocol imap { mail_plugins = fts fts_squat fts_lucene zlib } --8<---------------cut here---------------end--------------->8--- Dovecot version: 2.1.6 Operating system or Linux distribution name: MacOS X 10.7, 10.8 CPU architecture (x86 or something else?): x86_64 Filesystem you used (especially if you use NFS or not): Mac Some kind of description of what you were doing and with what IMAP client.: Searching -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From tss at iki.fi Tue Oct 16 03:09:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:09:03 +0300 Subject: [Dovecot] Advanced dovecot tricks - spam review/release In-Reply-To: <20121005194847.GA15222@daniel.localdomain> References: <506C9685.8070906@perkel.com> <506CCD7C.6070507@perkel.com> <506CF443.5080904@perkel.com> <72776FD9-C636-44E9-9EB7-A459FEBA12D4@iki.fi> <506E9003.7030201@krausam.de> <69BC7FF9-A2EA-407E-A31F-E53F3036327F@iki.fi> <20121005194847.GA15222@daniel.localdomain> Message-ID: <5FF6D55F-3B53-4CAD-ACBA-FC334E09F159@iki.fi> On 5.10.2012, at 22.48, Daniel Parthey wrote: > Timo Sirainen wrote: >> -i changes to dovecot.conf used by the given instance name > > This does not seem to work, at least not with version 2.1.10: Fixed: http://hg.dovecot.org/dovecot-2.1/rev/0262ede193e5 From tss at iki.fi Tue Oct 16 03:12:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:12:14 +0300 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> Message-ID: <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> On 4.10.2012, at 16.00, Benedetto Vassallo wrote: > All works fine, but with the new version it seems that dovecot don't do hardlinks when deliver a message to multiple users. The hard linking is done only when the directory permissions match. > mail_location = maildir:~/MailDir:LAYOUT=fs > > I tryed using lmtp directly issuing 'telnet localhost 24' and sending a test message to 3 recipients. > Then issuing a 'ls -il' in the "new" directory of that users, I saw the inode was not the same. What are the permissions of the MailDir directory for user1/user2? ls -ld /home/user1/MailDir ls -ld /home/user2/MailDir From tss at iki.fi Tue Oct 16 03:15:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:15:05 +0300 Subject: [Dovecot] Dovecot Hangs while mutile users download mail for same account using pop3 In-Reply-To: <793760c2702e89acc526a66c0b543293@Coptics.org> References: <793760c2702e89acc526a66c0b543293@Coptics.org> Message-ID: On 8.10.2012, at 18.12, Robert JR wrote: > I have a weird problem in dovecot, Dovecot Hangs while multiple users download mail for same account using pop3 > > Three persons use 1 same email , and three of them use outlook express to check > That specific mail .. Also some times one of the three users check the mail for this > Account using imap (squirrel mail) Make sure you have pop3_lock_session=no and.. > to=, orig_to=, relay=local, delay=357, delays=338/0.01/0/19, dsn=4.2.0, status=deferred (cannot update mailbox /var/mail/sales for user sales. unable to lock for exclusive access: Resource temporarily unavailable) The problem may simply be that you're using mbox format. POP3 protocol itself wasn't meant for simultaneous access (it's actually disallowed by the RFC) and with mbox format Dovecot optimizes it in a way that probably locks the mailbox exclusively for the whole duration of the session. From linuxpencil at hotmail.com Tue Oct 16 03:15:05 2012 From: linuxpencil at hotmail.com (John Reddy) Date: Mon, 15 Oct 2012 20:15:05 -0400 Subject: [Dovecot] Where'd the Mail Go? In-Reply-To: <20121015212746.GA8899@daniel.localdomain> References: , <20121015212746.GA8899@daniel.localdomain> Message-ID: > In the logfiles of your MTA (mail transport agent) you should find > hints where the mail went. Hmm. I guess I'll work on postfix then, and come back later ;) John From tss at iki.fi Tue Oct 16 03:35:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:35:57 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> Message-ID: <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> On 13.10.2012, at 15.38, Alessio Cecchi wrote: > Il 2012-10-02 21:28 Timo Sirainen ha scritto: >> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >> >>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 This should fix it: http://hg.dovecot.org/dovecot-2.1/rev/41aac09497ee From tss at iki.fi Tue Oct 16 03:37:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:37:38 +0300 Subject: [Dovecot] POP UIDL In-Reply-To: References: Message-ID: <30E57D11-39A4-43FC-9CF2-91644ADEF950@iki.fi> On 14.10.2012, at 5.07, simon.buongiorno at gmail.com wrote: > I run a small mail server with Dovecot and postfix. Several of the accounts are popped by an external email provide I use, mostly out habit and also for a bit of redundancy. I have no details on that external server except that it uses exim. > > Lately, it's been repopping mail from accounts (mail is left on the server so I can use IMAP) at fairly frequent, but undetermined, intervals. Before I take it to them, I want to be sure it's not an error on my side. How can I be sure Dovecot does not have a problem with the UIDL list causing this external server to repop the mail? > > For the record, I'm not inclined to think it's a Dovecot issue, but since that's the bit I can fix, I'd like to be sure. dovecot -n output would have helped. Anyway, some POP3 clients become confused if there are duplicate UIDLs. Recent v2.1 versions have a pop3_uidl_duplicates setting to avoid those. From tss at iki.fi Tue Oct 16 03:41:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:41:14 +0300 Subject: [Dovecot] Plugin hooks in login process In-Reply-To: <1947528.35zxeZD9k1@orion> References: <1947528.35zxeZD9k1@orion> Message-ID: <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> On 15.10.2012, at 16.13, Simone Lazzaris wrote: > Problem is, if I execute the update on the backend, I miss the information > regarding the original IP, as I only see the IP of the proxies. This is easy to solve: Set login_trusted_networks setting to point to your proxies, and you'll see the original IP. > Looking in the dovecot source code, I noticed that there aren't any hooks in > the execution path used by the proxies; I am missing something ? I am the only > one missing the presence of this hooks in the auth/proxy process ? The login processes aren't really meant to have any plugins. From tss at iki.fi Tue Oct 16 03:42:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:42:51 +0300 Subject: [Dovecot] fts = squat solr In-Reply-To: References: Message-ID: <933F50FC-B8F6-4A02-B738-9109B83D9D33@iki.fi> On 15.10.2012, at 18.08, Dave Abrahams wrote: > I don't know if this was supposed to have changed with dovecot2, but > http://wiki.dovecot.org/Plugins/FTS shows Read wiki2 for Dovecot v2. > fts = squat solr > > so, since I have the lucene plugin?"fts = lucene" works by itself?I > tried > > fts = squat lucene > > but: > > $ doveadm index '*' > doveadm(dave): Error: fts: Failed to initialize backend 'squat lucene': Unknown backend > > So, is that syntax obsolete, is the wiki wrong, or am I doing something > wrong? The syntax is obsolete. From tss at iki.fi Tue Oct 16 03:52:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 03:52:50 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 1.35, Dave Abrahams wrote: > According to the IMAP spec if I do a search for "TO isocpp.org" it > should find all the messages whose To: field contains the string > "isocpp.org", but dovecot is returning me an empty list. However, a > search for "TO tm at isocpp.org" produces a long list of messages. This specific problem can be solved by: plugin { fts_lucene = whitespace_chars=@. } > This > behavior is present if I *even load* the lucene fts plugin. > Note that lucene isn't in use (fts = squat); it's merely loaded. This > behavior goes away if I don't load fts_lucene. I don't really see how that's possible. Although a quick test shows me that fts_squat seems to be completely broken with me for some reason. From dave at boostpro.com Tue Oct 16 06:44:10 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:44:10 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } Wow; OK, Google tells me that's documented at http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now because I knew what to look for. This might be good enough for me, but still doesn't make it conforming to the IMAP spec, right? IIUC the spec says you can search for arbitrary strings without regard to word boundaries. >> This behavior is present if I *even load* the lucene fts plugin. >> Note that lucene isn't in use (fts = squat); it's merely loaded. >> This behavior goes away if I don't load fts_lucene. > > I don't really see how that's possible. Although a quick test shows me > that fts_squat seems to be completely broken with me for some reason. I don't know what to tell ya. Tests confirm it for me. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 06:45:53 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:45:53 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } OK, Google tells me that's documented at http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now because I knew what to look for. I suggest doing something to make that more discoverable. This might be good enough for me, but still doesn't make it conforming to the IMAP spec, right? IIUC the spec says you can search for arbitrary strings without regard to word boundaries. >> This behavior is present if I *even load* the lucene fts plugin. >> Note that lucene isn't in use (fts = squat); it's merely loaded. >> This behavior goes away if I don't load fts_lucene. > > I don't really see how that's possible. Although a quick test shows me > that fts_squat seems to be completely broken with me for some reason. I don't know what to tell ya. Tests confirm it for me. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave at boostpro.com Tue Oct 16 06:51:40 2012 From: dave at boostpro.com (Dave Abrahams) Date: Mon, 15 Oct 2012 20:51:40 -0700 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search References: Message-ID: on Mon Oct 15 2012, Timo Sirainen wrote: > On 16.10.2012, at 1.35, Dave Abrahams wrote: > >> According to the IMAP spec if I do a search for "TO isocpp.org" it >> should find all the messages whose To: field contains the string >> "isocpp.org", but dovecot is returning me an empty list. However, a >> search for "TO tm at isocpp.org" produces a long list of messages. > > This specific problem can be solved by: > > plugin { > fts_lucene = whitespace_chars=@. > } Do I also need plugin { fts = lucene } or are these mutually exclusive, or...? It's not clear from http://wiki2.dovecot.org/Plugins/FTS/Lucene -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From dave.mehler at gmail.com Tue Oct 16 09:30:34 2012 From: dave.mehler at gmail.com (David Mehler) Date: Tue, 16 Oct 2012 02:30:34 -0400 Subject: [Dovecot] per-user quotas In-Reply-To: <20121012040136.GA13561@daniel.localdomain> References: <20121012001539.GA10473@daniel.localdomain> <20121012040136.GA13561@daniel.localdomain> Message-ID: Hello, Thanks for your replies so far. Still having issues with per-user quotas. To my Mysql virtual_users table I've added a column quota_kb and for a test user I've added in a value of 250000 going for a 250 megabyte quota. I've tried various sql queries they're returning empty sets not pulling out the information needed. Thanks. Dave. On 10/12/12, Daniel Parthey wrote: > Hi Dave, > > David Mehler wrote: >> Thanks for your reply. I've written you directly as it is sounding >> like at least for now this isn't dovecot it's well a mysql issue with >> design. I don't have anything in my database setup with regards quota >> I've included it below. I've got one virtual mail user called vmail >> with UID/GID of 5000 who owns all the virtual mailboxes. If I'm >> understanding what I've read in the link, the dovecot wiki and your >> message since anything in a user section of Mysql will override the >> global configuration in 90-quota.conf, that being the case I should >> add an extra column to virtual users? Once that's done adjust the >> userdb query for dovecot to return quota information? > > Yes, the quota should be stored in an additional userdb column, > or you need at least an SQL statement which takes a username > and returns a quota rule. > > Quota of 0 is interpreted as "unlimited" by dovecot. > As already said, if the value in the quota column for the > user is 0, your SQL statement could also return a different > value (default quota) instead of 0 (using MySQL CASE/ELSE statement). > > Here is a short documentation on how it is done with Tine 2.0 Groupware: > http://www.tine20.org/wiki/index.php/Admins/Mailserver_integration > (it is a simple example which does not support a default quota fallback, > since Tine 2.0 writes the default quota as a value into the column > when the user is created) > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From tss at iki.fi Tue Oct 16 09:43:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 09:43:08 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 6.45, Dave Abrahams wrote: >>> According to the IMAP spec if I do a search for "TO isocpp.org" it >>> should find all the messages whose To: field contains the string >>> "isocpp.org", but dovecot is returning me an empty list. However, a >>> search for "TO tm at isocpp.org" produces a long list of messages. >> >> This specific problem can be solved by: >> >> plugin { >> fts_lucene = whitespace_chars=@. >> } > > OK, Google tells me that's documented at > http://wiki2.dovecot.org/Plugins/FTS/Lucene but I only found it now > because I knew what to look for. I suggest doing something to make that > more discoverable. That is the only page where there is any information about fts-lucene. I made it a bit clearer in that page now that whitespace_chars should be used as default. > This might be good enough for me, but still doesn't make it conforming > to the IMAP spec, right? IIUC the spec says you can search for > arbitrary strings without regard to word boundaries. It doesn't conform to the IMAP spec, correct. But nobody cares about that anymore. Everyone violates it. From tss at iki.fi Tue Oct 16 09:43:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 16 Oct 2012 09:43:41 +0300 Subject: [Dovecot] [BUG] Lucene plugin breaks header substring search In-Reply-To: References: Message-ID: On 16.10.2012, at 6.51, Dave Abrahams wrote: >> plugin { >> fts_lucene = whitespace_chars=@. >> } > > Do I also need > > plugin { > fts = lucene > } > > or are these mutually exclusive, or...? It's not clear from > http://wiki2.dovecot.org/Plugins/FTS/Lucene fts setting selects which backend to use. fts_lucene gives settings to that backend. From benedetto.vassallo at unipa.it Tue Oct 16 10:11:53 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Tue, 16 Oct 2012 09:11:53 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> Message-ID: <20121016091153.15601eysq5n040qh@webmail.unipa.it> Def. Quota Timo Sirainen : > On 4.10.2012, at 16.00, Benedetto Vassallo wrote: > >> All works fine, but with the new version it seems that dovecot >> don't do hardlinks when deliver a message to multiple users. > > The hard linking is done only when the directory permissions match. > >> mail_location = maildir:~/MailDir:LAYOUT=fs >> >> I tryed using lmtp directly issuing 'telnet localhost 24' and >> sending a test message to 3 recipients. >> Then issuing a 'ls -il' in the "new" directory of that users, I saw >> the inode was not the same. > > What are the permissions of the MailDir directory for user1/user2? > > ls -ld /home/user1/MailDir > ls -ld /home/user2/MailDir > > Thank you for your reply. They are different groups: drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ I tryed to issue: chgrp -R mail /home/user1/MailDir chgrp -R mail /home/user2/MailDir chgrp -R mail /home/user3/MailDir but nothing changed. Any idea? Thank you. -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From thefantaman at gmail.com Tue Oct 16 10:28:31 2012 From: thefantaman at gmail.com (Fabrizio Monti) Date: Tue, 16 Oct 2012 09:28:31 +0200 Subject: [Dovecot] /var/run/dovecot/auth-userdb failed In-Reply-To: <20121014142422.GA8080@daniel.localdomain> References: <1350056339814-38093.post@n4.nabble.com> <20121014142422.GA8080@daniel.localdomain> Message-ID: Thank you very much Daniel, I solved the problem. In this moment I have a problem with a maildir, I used a format "/home/vmail/%d/%u" and the first user make maildir he is owner "/home/vmail/%d" and second user don't make a maildir. Now, I look for the solution. Best regards. 2012/10/14 Daniel Parthey > thefantaman wrote: > > I work on test server and if I send an email on log i read > > > > lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: > > Permission denied (euid=8135(vmail) egid=8135(vmail) missing +r perm: > > /var/run/dovecot/auth-userdb, euid is not dir owner) > > > > unix_listener auth-userdb { > > mode = 0600 > > user = root > > group = root > > } > > } > > The problem is that LDA (local delivery agent or lmtp service) > is not able to look up the destination mailbox in userdb. > > The socket /var/run/dovecot/auth-userdb is currently only > readable or writable by user root since mode is set to 0600, > not readable or writable by other groups. > > http://wiki2.dovecot.org/LDA#Virtual_users > > You'll need to set up a auth-userdb socket for dovecot-lda so it > knows where to find mailboxes for the users. LDA is running under > the virtual mailbox user and group "vmail", so you need to grant > this user or group access to /var/run/dovecot/auth-userdb. > > You could do this by using group memberships and set mode = 0660 > or simply make it world-readable-writable with mode = 0666: > > unix_listener auth-userdb { > mode = 0666 > user = root > group = root > } > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From raabe at froglogic.com Tue Oct 16 10:30:26 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Tue, 16 Oct 2012 09:30:26 +0200 Subject: [Dovecot] Search for substring in header? In-Reply-To: References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> Message-ID: <507D0D12.6000205@froglogic.com> Am 10/16/2012 12:20 AM, schrieb Dave Abrahams: > > on Mon Oct 15 2012, Michael M Slusarz wrote: > >> Quoting Dave Abrahams : >> >>> on Mon Oct 15 2012, Dave Abrahams wrote: >>> >>>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>>> >>>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>>> same empty result with either of these two commands: >>>> >>>> UID SEARCH TO isocpp.org >>>> >>>> UID SEARCH TO "isocpp.org" >>>> >>>> Am I formatting the command wrongly? >>> >>> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >>> same result. >> >> Lucene for sure does not support subtext searching. Squat used to... >> but IIRC things may have changed for v2.1. Try the wiki. > > Sorry, but what does "try the wiki" mean? > Which indexer are you using, that successfully finds the substring match? I don't know what Michael had in mind, but I also seemed to recall that the 'Squat' plugin used to be the only FTS plugin which suppotred substring matches. http://wiki2.dovecot.org/Plugins/FTS/Squat explains: "The main difference between Squat indexes and the others is that Squat provides support for substring searches, while pretty much all other FTS indexes support only matching from the beginning of words. By strictly reading the IMAP RFC it requires substring matching, so to optimize regular TEXT and BODY searches you must use Squat with Dovecot v2.0. [..] However, almost all other commonly used IMAP servers no longer care about this requirement, so Dovecot v2.1 also no longer makes this distinction." I'm not sure how to read this, but I can imagine (and maybe that's what Michael was hinting at) that the Squat plugin for Dovecot >= 2.1 no longer supports substring matches as required by the IMAP RFC whereas previous versions do. P.S.: I wish this list would have a Reply-To configured. :-) -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From dave at boostpro.com Tue Oct 16 12:35:42 2012 From: dave at boostpro.com (Dave Abrahams) Date: Tue, 16 Oct 2012 02:35:42 -0700 Subject: [Dovecot] Search for substring in header? References: <20121014195950.Horde.YfHWG4F5lbhQe24WunFhYDA@bigworm.curecanti.org> <20121015130011.Horde.dI_3e4F5lbhQfF0718zEMQA@bigworm.curecanti.org> <507D0D12.6000205@froglogic.com> Message-ID: on Tue Oct 16 2012, Frerich Raabe wrote: > Am 10/16/2012 12:20 AM, schrieb Dave Abrahams: >> >> on Mon Oct 15 2012, Michael M Slusarz wrote: >> >>> Quoting Dave Abrahams : >>> > >>>> on Mon Oct 15 2012, Dave Abrahams wrote: >>>> >>>>> on Sun Oct 14 2012, Michael M Slusarz wrote: >>>>> >>>>> Using 2.1.6 and 2.1.9 built --with-clucene --with-libstemmer, I get the >>>>> same empty result with either of these two commands: >>>>> >>>>> UID SEARCH TO isocpp.org >>>>> >>>>> UID SEARCH TO "isocpp.org" >>>>> >>>>> Am I formatting the command wrongly? >>>> >>>> Incidentally, if I turn of fts_lucene and turn on fts_squat, I get the >>>> same result. >>> >>> Lucene for sure does not support subtext searching. Squat used to... >>> but IIRC things may have changed for v2.1. Try the wiki. >> >> Sorry, but what does "try the wiki" mean? >> Which indexer are you using, that successfully finds the substring match? > > I don't know what Michael had in mind, but I also seemed to recall > that the 'Squat' plugin used to be the only FTS plugin which suppotred > substring matches. http://wiki2.dovecot.org/Plugins/FTS/Squat > explains: > > "The main difference between Squat indexes and the others is that > Squat provides support for substring searches, while pretty much all > other FTS indexes support only matching from the beginning of > words. By strictly reading the IMAP RFC it requires substring > matching, so to optimize regular TEXT and BODY searches you must use > Squat with Dovecot v2.0. [..] However, almost all other commonly used > IMAP servers no longer care about this requirement, so Dovecot v2.1 > also no longer makes this distinction." > > I'm not sure how to read this, but I can imagine (and maybe that's > what Michael was hinting at) that the Squat plugin for Dovecot >= 2.1 > no longer supports substring matches as required by the IMAP RFC > whereas previous versions do. Well, it worked for me in 2.1.6 and 2.1.9. However, http://wiki2.dovecot.org/Plugins/FTS claims squat is "obsolete" in 2.1.x, my colleague is reporting (to me) crashes with squat during indexing, and Timo just posted that squat "seems to be completely broken for some reason." -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From alessio at skye.it Tue Oct 16 16:36:37 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 16 Oct 2012 15:36:37 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <48a2d35bf6a59f8a7ea472386a2b2ce4@skye.it> <6A1D6DAC-144F-4463-94B4-ABD0F35F9DD3@iki.fi> Message-ID: <507D62E5.60006@skye.it> Il 16/10/2012 02:35, Timo Sirainen ha scritto: > On 13.10.2012, at 15.38, Alessio Cecchi wrote: > >> Il 2012-10-02 21:28 Timo Sirainen ha scritto: >>> On 19.9.2012, at 16.07, Alessio Cecchi wrote: >>> >>>> #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( >>>> backend=0x1944240) at acl-backend-vfile-acllist.c:297 > This should fix it: http://hg.dovecot.org/dovecot-2.1/rev/41aac09497ee > Timo, with this patch works fine! Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From towern at gmail.com Tue Oct 16 16:39:13 2012 From: towern at gmail.com (tower) Date: Tue, 16 Oct 2012 15:39:13 +0200 Subject: [Dovecot] Problem with quota update in dovecot 1.2 Message-ID: <507D6381.2030703@gmail.com> Hi A have a problem with quota2 table, working under postfixadmin. When I login into imap server with thunderbird client everything works fine, quota is update when i receive a message, but when I move message to trash or any other folder value for messages field in table quota2 increases +1 instead decreases or have still this same value. Only if I close my imap client value descending -2. I wish to table quota2 will be updated immediately after any operation on inbox. Is there any mistake in my configuration? Thanks for any advice. ============================================================== dovecot -n print: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-686 i686 Debian 6.0.3 ext3 log_path: /var/log/dovecot/error.log info_log_path: /var/log/dovecot/info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login first_valid_uid: 100 mail_privileged_group: Debian-exim mail_uid: 101 mail_gid: 103 mail_location: maildir:/var/mail/virtual/%d/%n/Maildir mail_cache_min_mail_count: 100 mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): autocreate quota imap_quota trash mail_plugins(imap): autocreate quota imap_quota trash mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): tb-extra-mailbox-sep imap_client_workarounds(imap): tb-extra-mailbox-sep imap_client_workarounds(pop3): imap_id_log(default): * imap_id_log(imap): * imap_id_log(pop3): lda: postmaster_address: postmaster at mydomain.com mail_plugins: quota log_path: /var/log/dovecot/lda-errors.log info_log_path: /var/log/dovecot/deliver.log auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: Debian-exim plugin: autocreate: INBOX autocreate2: Sent autocreate3: Trash autocreate4: Drafts autocreate5: Junk autocreate6: Spam autosubscribe: INBOX autosubscribe2: Sent autosubscribe3: Trash autosubscribe4: Drafts autosubscribe5: Junk autosubscribe6: Spam quota: dict:user::proxy::quotadict quota_rule: Trash:storage=+20%% quota_rule2: Spam:ignore quota_rule3: Drafts:storage=+5%% quota_rule4: Sent:storage=+15%% quota_rule5: Junk:storage=+10%% quota_warning: storage=100%% quota-exceeded 100 %u quota_warning2: storage=95%% quota-warning 95 %u quota_warning3: storage=90%% quota-warning 90 %u quota_warning4: storage=85%% quota-warning 85 %u quota_warning5: storage=80%% quota-warning 80 %u quota_warning6: storage=75%% quota-warning 75 %u quota_warning7: storage=70%% quota-warning 70 %u quota_warning8: storage=65%% quota-warning 65 %u trash: /etc/dovecot/dovecot-trash.conf mail_log_events: delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_group_events: no mail_log_fields: uid box msgid size subject from dict: quotadict: mysql:/etc/dovecot/dovecot-dict-quota.conf ============================================= cat /etc/dovecot/dovecot-sql.conf driver = mysql connect = host=localhost dbname=eximdb user=eximdbadm password=************* default_pass_scheme = PLAIN password_query = \ SELECT username as user, domain, password \ FROM mailbox WHERE username= '%u' AND active = 1 user_query = \ SELECT CONCAT('/var/mail/virtual/', maildir) AS home, 101 AS uid, 103 AS gid, concat('dict:storage=', CAST(ROUND(quota / 1024) AS CHAR), '::proxy::quotadict') AS quota, CONCAT('*:storage=', CAST(quota AS CHAR), 'B') AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' ============================================= cat /etc/dovecot/dovecot-dict-quota.conf connect = host=localhost dbname=eximdb user=eximdbadm password=********* map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } ============================================== cat /etc/dovecot/dovecot-trash.conf 1 Trash 2 Spam 3 Sent 4 Draft From jbates at brightok.net Tue Oct 16 16:48:17 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 08:48:17 -0500 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D6381.2030703@gmail.com> References: <507D6381.2030703@gmail.com> Message-ID: <507D65A1.2060809@brightok.net> On 10/16/2012 8:39 AM, tower wrote: > Hi > > A have a problem with quota2 table, working under postfixadmin. When I > login into imap server with thunderbird client everything works fine, > quota is update when i receive a message, but when I move message to > trash or any other folder value for messages field in table quota2 > increases +1 instead decreases or have still this same value. Only if > I close my imap client value descending -2. I wish to table quota2 > will be updated immediately after any operation on inbox. Is there any > mistake in my configuration? > > Thanks for any advice. > Are you sure the client isn't just copying it to the other folder. When you exit, you may be expunging inbox (deleting the marked message) and emptying trash (deleting the copied message). Jack From gdelafond+dovecot at aquaray.com Tue Oct 16 16:56:00 2012 From: gdelafond+dovecot at aquaray.com (de Lafond Guillaume) Date: Tue, 16 Oct 2012 15:56:00 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> Message-ID: Hi, > Hi.I'm very sorry for the repost but I forgot the subject. > So, I'm > using dovecot 2.0.18 and I'm trying to authenticate through a CAS server > (until now authentication was through MS Active Directory). I could not > find anywhere some examples, so here is what i have done so far. > -install phpcas and pam_cas > -edit /etc/pam.d/dovecot > auth sufficient /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf > -edit > /etc.pam_cas.conf > host mycas.mydomain.com > port 443 > uriValidate /cas/proxyValidate > ssl on > proxy ??????????????????????? > trusted_ca /etc/cert/certificate.pem > debug > on > > - and finally dovecot.conf which I'm sure is complety wrong > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > passdb { > driver = pam > args = cache_key=%u dovecot > } Could you try with "failure_show_msg=yes" on passdb args ? You can try without the "proxy" line in pam_cas.conf and a static userdb with allow_all_users=yes (in place of ldap configuration). Hope this can help. You have some logs in /var/log/auth.log (depends of your distrib). -- Guillaume de Lafond Aqua Ray From towern at gmail.com Tue Oct 16 17:00:46 2012 From: towern at gmail.com (tower) Date: Tue, 16 Oct 2012 16:00:46 +0200 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D65A1.2060809@brightok.net> References: <507D6381.2030703@gmail.com> <507D65A1.2060809@brightok.net> Message-ID: <507D688E.6000903@gmail.com> On 10/16/2012 03:48 PM, Jack Bates wrote: > On 10/16/2012 8:39 AM, tower wrote: >> Hi >> >> A have a problem with quota2 table, working under postfixadmin. When >> I login into imap server with thunderbird client everything works >> fine, quota is update when i receive a message, but when I move >> message to trash or any other folder value for messages field in >> table quota2 increases +1 instead decreases or have still this same >> value. Only if I close my imap client value descending -2. I wish to >> table quota2 will be updated immediately after any operation on >> inbox. Is there any mistake in my configuration? >> >> Thanks for any advice. >> > Are you sure the client isn't just copying it to the other folder. > When you exit, you may be expunging inbox (deleting the marked > message) and emptying trash (deleting the copied message). > > > Jack Yes, now I see, that messages still is in folder from which I move it. In thunderbird I do drag and drop (FYI). Is there any switch in dovecot configuration which treats message as deleted from folder, from which I move that message? From s.lazzaris at interactive.eu Tue Oct 16 17:40:57 2012 From: s.lazzaris at interactive.eu (Simone Lazzaris) Date: Tue, 16 Oct 2012 16:40:57 +0200 Subject: [Dovecot] Plugin hooks in login process In-Reply-To: <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> References: <1947528.35zxeZD9k1@orion> <65FBA611-F6D8-4D0F-BC8A-A9F06E983CFA@iki.fi> Message-ID: <2673453.lfcgdkNUMt@orion> In data marted? 16 ottobre 2012 03:41:14, Timo Sirainen ha scritto: > On 15.10.2012, at 16.13, Simone Lazzaris wrote: > > Problem is, if I execute the update on the backend, I miss the information > > regarding the original IP, as I only see the IP of the proxies. > > This is easy to solve: Set login_trusted_networks setting to point to your > proxies, and you'll see the original IP. Thanks, it works as a charm. Just one more thing. I wasn't be able to find this option in the wiki, and also now that I know the meaning, I can't successfully google for it. Maybe the docs needs some love ? I hate tamper the developers for something I should have found by myself on the first place. > > Looking in the dovecot source code, I noticed that there aren't any hooks > > in the execution path used by the proxies; I am missing something ? I am > > the only one missing the presence of this hooks in the auth/proxy process > > ? > The login processes aren't really meant to have any plugins. I see. Well, thanks anyway. -- Simone Lazzaris | Responsabile aree datacenter e VoIP Interactive Network srl | via Roggia Vignola 9, 24047 Treviglio (BG) Tel. 0363 1970352 | Fax 0363.1971971 | www.interactive.eu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From david.travers at cohenschemist.co.uk Tue Oct 16 19:39:54 2012 From: david.travers at cohenschemist.co.uk (David Travers) Date: Tue, 16 Oct 2012 16:39:54 +0000 Subject: [Dovecot] Per user quotas Message-ID: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> Hi All, I keep going round in circles with this. I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. I believe I need to add to a passwd file, but I have added to that and nothing has changed. # dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.5 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:~/mail:LAYOUT=fs:INBOX=~/mail/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 lda: mail_plugins: quota auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: quota: maildir:user quota quota_rule: *:storage=200M quota_rule2: Trash:storage=10%% quota_rule2: SPAM:ignore quota_warning: storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2: storage=80%% /usr/local/bin/quota-warning.sh 80 quota_exceeded_message: Quota exceeded, Please delete some emails Any help gratefully received Dave _________________________________________ This email has been scanned for malicious content. _________________________________________ From jbates at brightok.net Tue Oct 16 19:54:21 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 11:54:21 -0500 Subject: [Dovecot] Problem with quota update in dovecot 1.2 In-Reply-To: <507D688E.6000903@gmail.com> References: <507D6381.2030703@gmail.com> <507D65A1.2060809@brightok.net> <507D688E.6000903@gmail.com> Message-ID: <507D913D.8070004@brightok.net> On 10/16/2012 9:00 AM, tower wrote: > Yes, now I see, that messages still is in folder from which I move it. > In thunderbird I do drag and drop (FYI). Is there any switch in > dovecot configuration which treats message as deleted from folder, > from which I move that message? Not to my knowledge, but someone else may know more. I setup the Trash folder with a +100MB quota per the wiki so that it could handle deleting emails when quota was near full. Jack From stocton12 at yahoo.com Tue Oct 16 21:12:37 2012 From: stocton12 at yahoo.com (b m) Date: Tue, 16 Oct 2012 11:12:37 -0700 (PDT) Subject: [Dovecot] CAS Authentication In-Reply-To: References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> Message-ID: <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login? with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. ________________________________ From: de Lafond Guillaume To: b m ; Dovecot Mailing List Sent: Tuesday, October 16, 2012 4:56 PM Subject: Re: [Dovecot] CAS Authentication Hi, > Hi.I'm very sorry for the repost but I forgot the subject. > So,? I'm > using dovecot 2.0.18 and I'm trying to authenticate through a CAS server > (until now authentication was through MS Active Directory). I could not > find anywhere some examples, so here is what i have done so far. > -install phpcas and pam_cas > -edit /etc/pam.d/dovecot >? ? ? ? ? ? ? ? ? auth? ? sufficient? ? ? /lib/security/pam_cas.so -simap://webmail.mydomain.com -f /etc/pam_cas.conf > -edit > /etc.pam_cas.conf >? ? ? ? ? ? ? ? ? host mycas.mydomain.com >? ? ? ? ? ? ? ? ? port 443 >? ? ? ? ? ? ? ? ? uriValidate /cas/proxyValidate >? ? ? ? ? ? ? ? ? ssl on >? ? ? ? ? ? ? ? ? proxy ??????????????????????? >? ? ? ? ? ? ? ? ? trusted_ca /etc/cert/certificate.pem >? ? ? ? ? ? ? ? ? debug > on > > - and finally dovecot.conf which I'm sure is complety wrong >? ? ? ? ? ? ? userdb { >? ? ? ? ? ? ? args = /etc/dovecot/dovecot-ldap.conf >? ? ? ? ? ? ? driver = ldap >? ? ? ? ? ? ? } >? ? ? ? ? ? ? passdb { >? ? ? ? ? ? ? driver = pam >? ? ? ? ? ? ? args = cache_key=%u dovecot >? ? ? ? ? ? ? } Could you try with "failure_show_msg=yes" on passdb args ? You can try without the "proxy" line in pam_cas.conf and a static userdb with allow_all_users=yes (in place of ldap configuration). Hope this can help. You have some logs in /var/log/auth.log (depends of your distrib). -- Guillaume de Lafond Aqua Ray From jbates at brightok.net Tue Oct 16 23:43:00 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 15:43:00 -0500 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 Message-ID: <507DC6D4.7090902@brightok.net> Timo, Please check the code. I didn't add it, but a real_lip might be useful for some people as well. Also, I notice that pop3-proxy is doing a different xsession than the imap proxy. Is there an xsession standard that is different between the two, or just an oversight in the code? Both send the remote address/port, but only imap proxy sends the local address/port. This patch declares long variable %{real_rip} so that a backend server can declare a different login_log_format_elements login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c This is primarily useful for backend servers to log both the rip, lip, and in case of xsession, the real rip. I haven't done extensive testing yet, but as long as nothing does anything weird elsewhere in the code, it should be good. http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch dovecot: pop3-login: Login: user=, method=PLAIN, rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured Jack Bates From list at airstreamcomm.net Tue Oct 16 23:57:45 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 16 Oct 2012 15:57:45 -0500 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> Message-ID: <507DCA49.9010702@airstreamcomm.net> On 10/4/12 9:58 AM, Timo Sirainen wrote: > On 4.10.2012, at 17.41, list at airstreamcomm.net wrote: > >>> protocol lmtp { >>> userdb { >>> .. >>> } >>> } >>> protocol !lmtp { >>> userdb { >>> .. >>> } >>> } >>> >>> >> Forgot to mention I am running 2.0.17. > The above works in v2.1. > >> And I am getting the following error: >> >> auth: Fatal: No passdbs specified in configuration file. PLAIN >> mechanism needs one >> >> >> From a previous post it appears that Dovecot cannot run without a global lookups specified: >> >> http://www.dovecot.org/list/dovecot/2012-March/064407.html >> >> Per the suggestion in the old post I created an empty passwdfile and included it in the auth-passwdfile which seems to have alleviated the issue, however this seems like a sub-optimal solution. Is this still the case, or is there a way to tell Dovecot that there is no global lookups? > The !lmtp version avoids that fatal problem. So the solution is: upgrade. Timo, I upgraded to 2.1 and configured as recommended, however I am still getting an error: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one Doveconf -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_debug = yes auth_verbose = yes disable_plaintext_auth = no mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service pop3-login { inet_listener pop3 { port = 110 } } ssl_cert = References: <507DC6D4.7090902@brightok.net> Message-ID: <507DE508.8070705@brightok.net> Still have some problems with the patch. will have to research it more. Worked fine on localhost, but isn't logging at all for foreign hosts in a production environment. Jack On 10/16/2012 3:43 PM, Jack Bates wrote: > Timo, > > Please check the code. I didn't add it, but a real_lip might be useful > for some people as well. Also, I notice that pop3-proxy is doing a > different xsession than the imap proxy. Is there an xsession standard > that is different between the two, or just an oversight in the code? > Both send the remote address/port, but only imap proxy sends the local > address/port. > > This patch declares long variable %{real_rip} so that a backend server > can declare a different login_log_format_elements > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l > pip=%{real_rip} mpid=%e %c > > This is primarily useful for backend servers to log both the rip, lip, > and in case of xsession, the real rip. I haven't done extensive > testing yet, but as long as nothing does anything weird elsewhere in > the code, it should be good. > > http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch > > dovecot: pop3-login: Login: user=, method=PLAIN, > rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured > > > Jack Bates > From fred at fredk.com Wed Oct 17 02:14:04 2012 From: fred at fredk.com (Fred Kilbourn) Date: Tue, 16 Oct 2012 18:14:04 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS Message-ID: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> I've had squat running on dovecot 2.0 and have been updating all users mailbox indexes nighty via cron with this command: doveadm -v search -A text xyzzyx I've just updated to 2.1 and I'm migrating to lucene indexes, but reading the documentation I'm having a hard time understanding the semantic differences between indexing and rescanning. If I were to continue to run an all user all mailbox index every night, would that be appropriate? Would running this every night avoid the need to ever rescan? Should I run rescan instead of index? Should I run both rescan and index? In which order? Best Regards, FredK From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 17 02:18:56 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 17 Oct 2012 01:18:56 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users Message-ID: <20121016231856.GA10851@daniel.localdomain> # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-44-server x86_64 Ubuntu 10.04.4 LTS doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A shows the following message in the log when iterating the 49th user: Oct 17 00:47:17 10.129.3.233 dovecot: doveadm: Error: purge: invalid option -- 'e' Oct 17 00:47:17 10.129.3.233 dovecot: doveadm(someuser at example-ll.org): Error: doveadm purge: Client sent unknown parameter: ? Any ideas on how this error gets triggered? Regards Daniel -- https://plus.google.com/103021802792276734820 From calestyo at scientia.net Wed Oct 17 02:21:14 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 01:21:14 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? Message-ID: <1350429674.3360.27.camel@fermat.scientia.net> Hi. I'm migrating all my mail archive (some 60 GB) from Evolution (which is really a broken piece of software) into dovecot. Now I face the problem how to do this best... Evolution (which is still a old 2.32.x version) itself uses mbox files, in a special hierarchical structure to allow subfolders and that like. It also stores it's own status info in X-Evolution and X-Evolution-Source mail headers. Unfortunately,... much of the mail was earlier from a Thunderbird installation, which uses it's own status headers (X-Mozilla*) that were not recognised by Evolution. I have no idea which mbox subformat was always used throughout the different programs and versions... 1) Any way to check for that? To make things worse... Thunder(burden) seems to have used a modified From_ line syntax... "^From -
$" Ideally I'd like migrate all mail into dovecot (for storage reasons again, mbox) retaining the different status flags (read, forwarded, etc.) and getting rid of the proprietary headers (of course only when they were. First thing I tried was to simply copy mail within Evolution (i.e. dragging&dropping it from the local folders to the IMAP folders from dovecot). - that preserves the status from Evolution, but doesn't restore that from Thunderbird - it clutters up the information of all From_ lines... "
" becomes "
" - neither does it handle the special Thunderbird From_ lines - neither does it remove the Thunderbird or the X-Evolution-Source headers - if Evolution has already had corrupted index files (and this is extremely likely... as it happens even immediately during recreating all of them)... so I may loose mail So my idea was that I need a program that: - can parse all the different mbox formats (those that use the quoted ">From" style and those that use Conent-Length) - can differentiate message headers from body (so that I can drop the proprietary headers and replace them by what dovecot uses as headers) - must of course understand multiline message headers 2) Any idea for a tool like that? The meaning of the X-Evolution and X-Mozilla* headers are easy to find on the web.... so I can convert them. So I basically "just" need a tool that parses all kinds of mbox formats... allow me to drop/add headers... and spits out the rest unmodified. 3) dovecot uses some special headers like X-UID and X-IMAPD... will it create these on it's own, the first time it processes the new mbox file? I mean these headers won't be there after creation. 4) Should I drop out (during) conversion... any other mail headers.. that dovecot uses as it's own? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 17 04:21:46 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 20:21:46 -0500 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 In-Reply-To: <507DE508.8070705@brightok.net> References: <507DC6D4.7090902@brightok.net> <507DE508.8070705@brightok.net> Message-ID: <507E082A.9050508@brightok.net> Umm, yeah. Setting your rsyslog to pipe certain IP matches to another file really screws things up when you change the log to include that IP. Hours of work before I figured out that it was logging just fine but to another file. lol Jack On 10/16/2012 5:51 PM, Jack Bates wrote: > Still have some problems with the patch. will have to research it > more. Worked fine on localhost, but isn't logging at all for foreign > hosts in a production environment. > > > Jack > > On 10/16/2012 3:43 PM, Jack Bates wrote: >> Timo, >> >> Please check the code. I didn't add it, but a real_lip might be >> useful for some people as well. Also, I notice that pop3-proxy is >> doing a different xsession than the imap proxy. Is there an xsession >> standard that is different between the two, or just an oversight in >> the code? Both send the remote address/port, but only imap proxy >> sends the local address/port. >> >> This patch declares long variable %{real_rip} so that a backend >> server can declare a different login_log_format_elements >> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l >> pip=%{real_rip} mpid=%e %c >> >> This is primarily useful for backend servers to log both the rip, >> lip, and in case of xsession, the real rip. I haven't done extensive >> testing yet, but as long as nothing does anything weird elsewhere in >> the code, it should be good. >> >> http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch >> >> dovecot: pop3-login: Login: user=, method=PLAIN, >> rip=192.168.1.5, lip=::1, pip=::1, mpid=8665, secured >> >> >> Jack Bates >> > From calestyo at scientia.net Wed Oct 17 05:00:36 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 04:00:36 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350429674.3360.27.camel@fermat.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> Message-ID: <1350439236.18957.13.camel@fermat.scientia.net> Hi again... Things are even much much worse... (oh how I hate Evolution right now). I found a bug in Evolution, where it apparently corrupts all mail by incorrectly (not) quoting From_ lines in headers/bodies... It quotes lines matching "^From (.*)$" as ">From \1" but it does not quote at all already quoted From_ lines, i.e. "^>+From .*$". Now that means AFAICS, that it's not possible to repair that corruption (you'll see my "happiness" about this, when reading the offensive bug report): Details here: https://bugzilla.gnome.org/show_bug.cgi?id=686258 I'm not sure how this affects any of my migration/conversion plans... any ideas? Thanks, a desperate Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jbates at brightok.net Wed Oct 17 05:44:19 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 21:44:19 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: <507E1B83.7070104@brightok.net> On 10/16/2012 6:14 PM, Fred Kilbourn wrote: > I've had squat running on dovecot 2.0 and have been updating all users > mailbox indexes nighty via cron with this command: > > doveadm -v search -A text xyzzyx > > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > the documentation I'm having a hard time understanding the semantic > differences between indexing and rescanning. > > > > If I were to continue to run an all user all mailbox index every night, > would that be appropriate? > > > > Would running this every night avoid the need to ever rescan? > > 2 sets of indexes. dovecot indexes FTS indexes Performing the cron search will update the FTS indexes, although you should read up on 2.1's doveadm index command. The dovecot indexes should stay in sync. However, if they do lose track of the FTS indexes, you can do a rescan to sync them back up. Except for corruption or index changes made outside of dovecot, the two sets of indexes should stay in sync. Jack From jbates at brightok.net Wed Oct 17 06:01:17 2012 From: jbates at brightok.net (Jack Bates) Date: Tue, 16 Oct 2012 22:01:17 -0500 Subject: [Dovecot] Per user quotas In-Reply-To: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> Message-ID: <507E1F7D.4080107@brightok.net> On 10/16/2012 11:39 AM, David Travers wrote: > Hi All, > > I keep going round in circles with this. > I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! > > I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. > > I believe I need to add to a passwd file, but I have added to that and nothing has changed. > > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields Note that the extra fields are prefixed with userdb_ when placed in a passwd file. Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). Jack From fred at fredk.com Wed Oct 17 06:09:20 2012 From: fred at fredk.com (Fred Kilbourn) Date: Tue, 16 Oct 2012 22:09:20 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <507E1B83.7070104@brightok.net> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> <507E1B83.7070104@brightok.net> Message-ID: <00b301cdac14$cda1f880$68e5e980$@fredk.com> > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] > On Behalf Of Jack Bates > Sent: Tuesday, October 16, 2012 9:44 PM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Difference between Indexing and Rescan in FTS > > On 10/16/2012 6:14 PM, Fred Kilbourn wrote: > > I've had squat running on dovecot 2.0 and have been updating all users > > mailbox indexes nighty via cron with this command: > > > > doveadm -v search -A text xyzzyx > > > > > > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > > the documentation I'm having a hard time understanding the semantic > > differences between indexing and rescanning. > > > > > > > > If I were to continue to run an all user all mailbox index every night, > > would that be appropriate? > > > > > > > > Would running this every night avoid the need to ever rescan? > > > > > 2 sets of indexes. > > dovecot indexes > FTS indexes > > Performing the cron search will update the FTS indexes, although you > should read up on 2.1's doveadm index command. The dovecot indexes > should stay in sync. However, if they do lose track of the FTS indexes, > you can do a rescan to sync them back up. Except for corruption or index > changes made outside of dovecot, the two sets of indexes should stay in > sync. > > > Jack Thanks Jack, So here are my takeaways, let me know if I'm wrong: - The FTS index is the actual search data - The dovecot index holds, among other information, which messages are indexed by FTS - The FTS index still doesn't update automatically, so my nightly cronjob should keep it in order - The dovecot index should stay in order under normal circumstances, and issuing a resync command shouldn't be needed unless something bad happens Assuming my understanding above is correct, how about these questions, to further clarify my original questions: - As a system administrator, what signs should I look for that a resync is needed? (aside from user complaints) - What exact impact does running the resync command have? - Is it worthwhile to resync periodically as a maintenance task? - Or, does resyncing reset all FTS indexing that has been done, causing it to have to be done again from scratch? And, I did catch the revision in the user docs for updating indexes. I plan on updating my maintenance script accordingly. Thanks, Fred From tss at iki.fi Wed Oct 17 06:15:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 17 Oct 2012 06:15:54 +0300 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: On 17.10.2012, at 2.14, Fred Kilbourn wrote: > I've had squat running on dovecot 2.0 and have been updating all users > mailbox indexes nighty via cron with this command: > > doveadm -v search -A text xyzzyx doveadm index is a bit more efficient. > I've just updated to 2.1 and I'm migrating to lucene indexes, but reading > the documentation I'm having a hard time understanding the semantic > differences between indexing and rescanning. doveadm fts rescan makes sure that 1) all of the old messages are indexed and 2) there are no extra (already deleted) messages indexed. So it's basically repairing fts index. You probably shouldn't run it automatically, or at least not very often. From dmalolepszy at optusnet.com.au Wed Oct 17 09:11:13 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 17 Oct 2012 17:11:13 +1100 Subject: [Dovecot] Dovecot failed logins delay all logins Message-ID: <507E4C01.6010303@optusnet.com.au> Hi all, I have observed with my Dovecot setup that unique failed logins cause legitimate correct logins to be slowed. I am running two servers, each with two Dovecot instances, a Proxy with Director, and a backend Dovecot. I suspect that the backend instance is throttling connections from the same IP, and because I am running a Proxy, the backend will only see either of the two server IPs. I confirmed this by directly connecting to the backend, to bypass the proxy and rule it. I initiated dozens of unique failed logins from one IP and separately attempted to login from the same IP, and experienced an extended delay during login. At the same time a login from a different IP suceeded imediately. I see nothing in the logs suggesting some sort of process limits were exceeded, however I do see the following proc title for the backend auth process: "dovecot/auth [7 wait, 0 passdb, 0 userdb]" I have increased the mail_max_userip_connections to a very large value however I believe that setting is a per username/ip limit. Is there any sort of setting in Dovecot that I can configure that stops this authentication throttling per IP? Below is the configuration of the backend Dovecot instance. # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.3 (Santiago) auth_cache_negative_ttl = 3 secs auth_cache_size = 100 M auth_cache_ttl = 10 mins auth_default_realm = example.com auth_failure_delay = 5 secs auth_mechanisms = plain login auth_verbose_passwords = sha1 auth_worker_max_count = 25 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 last_valid_gid = 12 last_valid_uid = 8 login_greeting = Hello there. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_fsync = always mail_gid = mail mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " stats" mail_uid = mail mmap_disable = yes namespace { inbox = yes location = maildir:%h/Maildir prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocols = pop3 imap service auth { unix_listener auth-userdb { group = mail mode = 0660 user = mail } } service imap-login { inet_listener imap { address = 0.0.0.0 port = 9143 } process_min_avail = 5 service_count = 0 vsz_limit = 256 M } service imap { process_limit = 1000 vsz_limit = 256 M } service pop3-login { inet_listener pop3 { address = 0.0.0.0 port = 9110 } process_min_avail = 5 service_count = 0 vsz_limit = 256 M } service pop3 { process_limit = 1000 vsz_limit = 256 M } service stats { fifo_listener stats-mail { mode = 0600 user = mail } inet_listener { address = 127.0.0.1 port = 24242 } } ssl = no stats_memory_limit = 64 M userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol imap { imap_logout_format = bytes_read=%i bytes_send=%o mail_max_userip_connections = 1000 mail_plugins = " stats " } protocol pop3 { mail_max_userip_connections = 1000 } Dominic From pipefab at mweb.co.za Wed Oct 17 09:26:30 2012 From: pipefab at mweb.co.za (Hendrik) Date: Wed, 17 Oct 2012 08:26:30 +0200 Subject: [Dovecot] dovecot died Message-ID: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Hi All I have been trying to get this website running for months now. I get this emails from cpanel and don't know how to fix it. If anyone can help me please contact me at pipefab at mweb.co.za. Kind regards Hendrik imap failed @ Fri Oct 12 05:51:18 2012. A restart was attempted automagically. Service Check Method: [check command] Number of Restart Attempts: 1 Cmd Service Check Raw Output: dovecot is not running Startup Log: /etc/init.d/dovecot: line 15: 6043 Alarm clock /usr/sbin/dovecot > /dev/null 2>&1 Syslog Messages: Oct 12 04:01:01 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3 yj43fuh...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:01:02 scene dovecot: IMAP(__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3y j43fuhjgeiqomc3dhlkyjwdq): Disconnected: Logged out bytes=11/340 Oct 12 04:06:06 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0 jaovpdr...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:06:06 scene dovecot: IMAP(__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0j aovpdrgqjnf0_rxyi0wncetn): Disconnected: Logged out bytes=11/313 Oct 12 04:11:01 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4z k8h05nx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:11:01 scene dovecot: IMAP(__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4zk 8h05nx2p9n4yfxhrp3a2gjhl): Disconnected: Logged out bytes=11/313 Oct 12 04:16:08 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigs c75cnov...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:16:09 scene dovecot: IMAP(__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigsc 75cnovslbll4702ue2veu2n3): Disconnected: Logged out bytes=11/318 Oct 12 04:21:10 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsf ou8d5ky...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:21:11 scene dovecot: IMAP(__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsfo u8d5kysrfeaqvlmjgx6afvnb): Disconnected: Logged out bytes=11/340 Oct 12 04:26:35 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoe oga08za...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:26:36 scene dovecot: IMAP(__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoeo ga08zaq_nh6yzqsmveqpvnxi): Disconnected: Logged out bytes=11/313 Oct 12 04:31:05 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaus tpvzl1q...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:31:05 scene dovecot: IMAP(__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaust pvzl1qjjei5uuoi1c4g8tpea): Disconnected: Logged out bytes=11/313 Oct 12 04:36:02 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjj fzonng9...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:36:03 scene dovecot: IMAP(__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjjf zonng9eqrsw5l5hg7xoejer2): Disconnected: Logged out bytes=11/340 Oct 12 04:41:03 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gw eibvov1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:41:04 scene dovecot: IMAP(__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gwe ibvov1_minfh7j4_4ejaz7v2): Disconnected: Logged out bytes=11/340 Oct 12 04:46:04 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhd rhafjur...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:46:05 scene dovecot: IMAP(__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhdr hafjureydiuxbtbk2jkpkvlo): Disconnected: Logged out bytes=11/340 Oct 12 04:51:06 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrq h1iztqo...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:51:06 scene dovecot: IMAP(__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrqh 1iztqosnzfwt28kkzv4riyd9): Disconnected: Logged out bytes=11/340 Oct 12 04:56:07 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan 2pdmdvg...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 04:56:07 scene dovecot: IMAP(__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan2 pdmdvg6jofzylncdb3ytjaaz): Disconnected: Logged out bytes=11/340 Oct 12 05:01:08 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvir quwcv1z...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:01:08 scene dovecot: IMAP(__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvirq uwcv1zhloqt12rqni_o2pqcj): Disconnected: Logged out bytes=11/340 Oct 12 05:06:09 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgr cpi3usb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:06:09 scene dovecot: IMAP(__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgrc pi3usbdjkiy2n8zy2bbvhhny): Disconnected: Logged out bytes=11/340 Oct 12 05:11:10 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg 4vew3df...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:11:10 scene dovecot: IMAP(__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg4 vew3dfbyrglsasuldldaspck): Disconnected: Logged out bytes=11/340 Oct 12 05:16:11 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0 e2bwtas...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:16:11 scene dovecot: IMAP(__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0e 2bwtastu0kton3azlhmmuhwi): Disconnected: Logged out bytes=11/340 Oct 12 05:21:12 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszq mf4h8z1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:21:12 scene dovecot: IMAP(__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszqm f4h8z1shjl34q9zpid3g4gsp): Disconnected: Logged out bytes=11/340 Oct 12 05:26:13 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaog ha3yc0w...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:26:13 scene dovecot: IMAP(__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaogh a3yc0w6t7vvgyf1snz8vechf): Disconnected: Logged out bytes=11/340 Oct 12 05:31:14 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62 mucudsm...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:31:15 scene dovecot: IMAP(__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62m ucudsmjfmyolzcpm9shakkiw): Disconnected: Logged out bytes=11/340 Oct 12 05:36:15 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxc kmn7fkd...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:36:16 scene dovecot: IMAP(__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxck mn7fkdpsvbrjptqanfuljfv2): Disconnected: Logged out bytes=11/340 Oct 12 05:41:15 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqun f1neeqb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:41:16 scene dovecot: IMAP(__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqunf 1neeqbmzas00uqbzmmjsxiab): Disconnected: Logged out bytes=11/340 Oct 12 05:46:17 scene dovecot: imap-login: Login: user=<__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfre yewqvxx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 05:46:17 scene dovecot: IMAP(__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfrey ewqvxxeoo7yhyiki7ghukxss): Disconnected: Logged out bytes=11/340 Oct 12 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 seconds. This might cause a lot of problems, so I'll just kill myself now. http://wiki.dovecot.org/TimeMovedBackwards From fred at fredk.com Wed Oct 17 09:26:32 2012 From: fred at fredk.com (Fred Kilbourn) Date: Wed, 17 Oct 2012 01:26:32 -0500 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> Message-ID: <00e101cdac30$5ab63270$10229750$@fredk.com> > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] > On Behalf Of Timo Sirainen > Sent: Tuesday, October 16, 2012 10:16 PM > To: Fred Kilbourn > Cc: dovecot at dovecot.org > Subject: Re: [Dovecot] Difference between Indexing and Rescan in FTS > > On 17.10.2012, at 2.14, Fred Kilbourn wrote: > > > I've had squat running on dovecot 2.0 and have been updating all users > > mailbox indexes nighty via cron with this command: > > > > doveadm -v search -A text xyzzyx > > doveadm index is a bit more efficient. > > > I've just updated to 2.1 and I'm migrating to lucene indexes, but > reading > > the documentation I'm having a hard time understanding the semantic > > differences between indexing and rescanning. > > doveadm fts rescan makes sure that 1) all of the old messages are > indexed and 2) there are no extra (already deleted) messages indexed. So > it's basically repairing fts index. You probably shouldn't run it > automatically, or at least not very often. Okay, you've clarified that for me. I understand that rescan isn't a nightly task, but could be run every now and then periodically. How often might be appropriate if I wanted to do this as a maintenance task? Once a month? Lastly, I'm trying to use the index command instead of the search command, but I can't figure out how to make it index every mailbox for every user. Is there a wildcard that can be used for the mailbox? Or do I need to iterate all the mailboxes with one command and run index however many times for each inbox? Thanks for your help From dmalolepszy at optusnet.com.au Wed Oct 17 09:44:10 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 17 Oct 2012 17:44:10 +1100 Subject: [Dovecot] Dovecot failed logins delay all logins In-Reply-To: <507E4C01.6010303@optusnet.com.au> References: <507E4C01.6010303@optusnet.com.au> Message-ID: <507E53BA.7030001@optusnet.com.au> I think I found a solution to this thanks to a post by Timo here: http://dovecot.org/list/dovecot/2011-December/062631.html service anvil { unix_listener anvil-auth-penalty { mode = 0 } } On 17/10/12 17:11, Dominic Malolepszy wrote: > Hi all, > > I have observed with my Dovecot setup that unique failed logins cause > legitimate correct logins to be slowed. I am running two servers, each > with two Dovecot instances, a Proxy with Director, and a backend > Dovecot. I suspect that the backend instance is throttling > connections from the same IP, and because I am running a Proxy, the > backend will only see either of the two server IPs. I confirmed this > by directly connecting to the backend, to bypass the proxy and rule > it. I initiated dozens of unique failed logins from one IP and > separately attempted to login from the same IP, and experienced an > extended delay during login. At the same time a login from a different > IP suceeded imediately. I see nothing in the logs suggesting some sort > of process limits were exceeded, however I do see the following proc > title for the backend auth process: > "dovecot/auth [7 wait, 0 passdb, 0 userdb]" > > I have increased the mail_max_userip_connections to a very large value > however I believe that setting is a per username/ip limit. Is there > any sort of setting in Dovecot that I can configure that stops this > authentication throttling per IP? Below is the configuration of the > backend Dovecot instance. > > > # 2.1.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Red Hat Enterprise Linux > Server release 6.3 (Santiago) > auth_cache_negative_ttl = 3 secs > auth_cache_size = 100 M > auth_cache_ttl = 10 mins > auth_default_realm = example.com > auth_failure_delay = 5 secs > auth_mechanisms = plain login > auth_verbose_passwords = sha1 > auth_worker_max_count = 25 > base_dir = /var/run/dovecot/ > disable_plaintext_auth = no > first_valid_gid = 12 > first_valid_uid = 8 > last_valid_gid = 12 > last_valid_uid = 8 > login_greeting = Hello there. > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > mail_fsync = always > mail_gid = mail > mail_location = maildir:%h/Maildir > mail_nfs_index = yes > mail_nfs_storage = yes > mail_plugins = " stats" > mail_uid = mail > mmap_disable = yes > namespace { > inbox = yes > location = maildir:%h/Maildir > prefix = INBOX. > separator = . > } > passdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > protocols = pop3 imap > service auth { > unix_listener auth-userdb { > group = mail > mode = 0660 > user = mail > } > } > service imap-login { > inet_listener imap { > address = 0.0.0.0 > port = 9143 > } > process_min_avail = 5 > service_count = 0 > vsz_limit = 256 M > } > service imap { > process_limit = 1000 > vsz_limit = 256 M > } > service pop3-login { > inet_listener pop3 { > address = 0.0.0.0 > port = 9110 > } > process_min_avail = 5 > service_count = 0 > vsz_limit = 256 M > } > service pop3 { > process_limit = 1000 > vsz_limit = 256 M > } > service stats { > fifo_listener stats-mail { > mode = 0600 > user = mail > } > inet_listener { > address = 127.0.0.1 > port = 24242 > } > } > ssl = no > stats_memory_limit = 64 M > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-ldap.conf > driver = ldap > } > verbose_proctitle = yes > protocol imap { > imap_logout_format = bytes_read=%i bytes_send=%o > mail_max_userip_connections = 1000 > mail_plugins = " stats " > } > protocol pop3 { > mail_max_userip_connections = 1000 > } > > > Dominic From amateo at um.es Wed Oct 17 10:24:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 17 Oct 2012 09:24:42 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> Message-ID: <507E5D3A.5030900@um.es> El 16/10/12 20:12, b m escribi?: > Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. > This is my config. In /etc/pam.d/dovecot I have: auth sufficient pam_cas_ssh.so -simap://localhost -f/etc/pam_cas.conf account sufficient pam_permit.so session sufficient pam_permit.so and /etc/pam_cas.conf host port 443 uriValidate /cas/proxyValidate ssl on debug off proxy trusted_ca in dovecot, I have these users dbs: userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } With this, it works fine. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From david.travers at cohenschemist.co.uk Wed Oct 17 10:42:06 2012 From: david.travers at cohenschemist.co.uk (David Travers) Date: Wed, 17 Oct 2012 07:42:06 +0000 Subject: [Dovecot] Per user quotas In-Reply-To: <507E1F7D.4080107@brightok.net> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> Message-ID: <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> Hi Jake, Yep, similar to what I had been trying, but it doesn't seem to be working. In my /etc/passwd file I had the line:- dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash I have changed it to show:- dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange Also yes, I noticed the numbering and have corrected. Thanks for your quick reply. Dave -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates Sent: 17 October 2012 04:02 To: dovecot at dovecot.org Subject: Re: [Dovecot] Per user quotas On 10/16/2012 11:39 AM, David Travers wrote: > Hi All, > > I keep going round in circles with this. > I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! > > I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. > > I believe I need to add to a passwd file, but I have added to that and nothing has changed. > > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields Note that the extra fields are prefixed with userdb_ when placed in a passwd file. Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). Jack _________________________________________ This email has been scanned for malicious content. _________________________________________ From CMarcus at Media-Brokers.com Wed Oct 17 12:59:30 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 17 Oct 2012 05:59:30 -0400 Subject: [Dovecot] LMTP userdb lookup In-Reply-To: <507DCA49.9010702@airstreamcomm.net> References: <506C915C.2070709@airstreamcomm.net> <506DA023.5030609@airstreamcomm.net> <507DCA49.9010702@airstreamcomm.net> Message-ID: <507E8182.3040904@Media-Brokers.com> On 2012-10-16 4:57 PM, list at airstreamcomm.net wrote: > > Doveconf -n: > > # 2.1.1: /etc/dovecot/dovecot.conf Ummm... latest is 2.1.10... try upgrading again... ;) From CMarcus at Media-Brokers.com Wed Oct 17 13:13:34 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 17 Oct 2012 06:13:34 -0400 Subject: [Dovecot] dovecot died In-Reply-To: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> References: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Message-ID: <507E84CE.60708@Media-Brokers.com> On 2012-10-17 2:26 AM, Hendrik wrote: > 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 > seconds. This might cause a lot of problems, so I'll just kill myself now. > http://wiki.dovecot.org/TimeMovedBackwards Presumably this is on a shared hosting service, and they haven't gotten the VM to sync time properly. It is absolutely critical that time be kept in sync on servers, especially mail servers, so dovecot assumes a serious problem exists when time leaps like this happen and kills itself. Fix the time sync problem on your server (you may have to work with the service/hosting provider) and dovecot will stop committing seppuku... Charles From raabe at froglogic.com Wed Oct 17 15:44:12 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Wed, 17 Oct 2012 14:44:12 +0200 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read Message-ID: <507EA81C.5060806@froglogic.com> Hi, I'm running Dovecot 1.2.17 on FreeBSD 9 to serve an archive of a few internal mailinglists. The archive is implemented using a public namespace: namespace private { separator = / prefix = inbox = yes } namespace public { separator = / prefix = Lists/ location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists subscriptions = no } As you can see, the CONTROL/INDEX files are stored per-user to allow private \Seen flags. The different mailinglists are all sent to the 'lists' user which has a Sieve script to file them into different folders, so I have directories on my harddisk like /home/vmail/lists/Maildir/.somelist /home/vmail/lists/Maildir/.anotherlist Now, I'd like to mark the mail in *one* of those folders as \Seen by default. If the INDEX files weren't per-user, it would simply be a matter of using 'addflag "\Seen";' in the Sieve script of the lists user. Alas, this has no effect. Hence my question - how can I have the mail of just one mailinglist get marked as "read" for all users? So far, the only option I see is to add a second public namespace, with a different prefix - and this namespace doesn't use private CONTROL/INDEX files. However, I'd like to keep using the "Lists" prefix if possible to avoid too many changes to the clients. -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From jbates at brightok.net Wed Oct 17 16:02:47 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:02:47 -0500 Subject: [Dovecot] Dovecot failed logins delay all logins In-Reply-To: <507E53BA.7030001@optusnet.com.au> References: <507E4C01.6010303@optusnet.com.au> <507E53BA.7030001@optusnet.com.au> Message-ID: <507EAC77.6060401@brightok.net> On 10/17/2012 1:44 AM, Dominic Malolepszy wrote: > I think I found a solution to this thanks to a post by Timo here: > http://dovecot.org/list/dovecot/2011-December/062631.html > > service anvil { unix_listener anvil-auth-penalty { mode = 0 } } > You can also leave IP based penalties and set your other servers such as proxy and webmail as trusted. Jack From jbates at brightok.net Wed Oct 17 16:07:06 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:07:06 -0500 Subject: [Dovecot] dovecot died In-Reply-To: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> References: <000001cdac30$5a517d30$0ef47790$@mweb.co.za> Message-ID: <507EAD7A.2080100@brightok.net> Read the wiki that was linked. It could be anything from ntpd/OS/hardware bug. It isn't uncommon for there to be TSC timing issues as well. I have a cutting edge server that has a bug that breaks TSC and causes timing issues. Luckily, my OS is relatively good at not stepping backwards in time. Jack On 10/17/2012 1:26 AM, Hendrik wrote: > Hi All > > I have been trying to get this website running for months now. I get this > emails from cpanel and don't know how to fix it. If anyone can help me > please contact me at pipefab at mweb.co.za. > > > > Kind regards > > Hendrik > > imap failed @ Fri Oct 12 05:51:18 2012. A restart was attempted > automagically. > > > > Service Check Method: [check command] > > > > Number of Restart Attempts: 1 > > > > Cmd Service Check Raw Output: dovecot is not running > > > > > > Startup Log: /etc/init.d/dovecot: line 15: 6043 Alarm clock > /usr/sbin/dovecot > /dev/null 2>&1 > > > > > > Syslog Messages: Oct 12 04:01:01 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3 > yj43fuh...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:01:02 scene dovecot: > IMAP(__cpanel__service__auth__imap__lvgws_iymiqndfmflick2pa3yjzc56ukpa2t6x3y > j43fuhjgeiqomc3dhlkyjwdq): Disconnected: Logged out bytes=11/340 Oct 12 > 04:06:06 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0 > jaovpdr...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:06:06 scene dovecot: > IMAP(__cpanel__service__auth__imap__jxy1xcsu0koedgkhexhexpu3_idp4ynukxpaou0j > aovpdrgqjnf0_rxyi0wncetn): Disconnected: Logged out bytes=11/313 Oct 12 > 04:11:01 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4z > k8h05nx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:11:01 scene dovecot: > IMAP(__cpanel__service__auth__imap__6oxwycgffp_5xkysaitw4eifev2nffi_dqlhj4zk > 8h05nx2p9n4yfxhrp3a2gjhl): Disconnected: Logged out bytes=11/313 Oct 12 > 04:16:08 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigs > c75cnov...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:16:09 scene dovecot: > IMAP(__cpanel__service__auth__imap__7zurxql5qf5whp4rupxen3viduh5kucjqtrzigsc > 75cnovslbll4702ue2veu2n3): Disconnected: Logged out bytes=11/318 Oct 12 > 04:21:10 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsf > ou8d5ky...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:21:11 scene dovecot: > IMAP(__cpanel__service__auth__imap__cq4wbk6o7svgbljnmw1hx2iiaunvzp3w1cywwsfo > u8d5kysrfeaqvlmjgx6afvnb): Disconnected: Logged out bytes=11/340 Oct 12 > 04:26:35 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoe > oga08za...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:26:36 scene dovecot: > IMAP(__cpanel__service__auth__imap__uhyykmmdnf31il4pn_kfci9y2gw2o9skyz7zuoeo > ga08zaq_nh6yzqsmveqpvnxi): Disconnected: Logged out bytes=11/313 Oct 12 > 04:31:05 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaus > tpvzl1q...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:31:05 scene dovecot: > IMAP(__cpanel__service__auth__imap__hka0onlsdbqugjyirdyygk_d9wtw_xtkl7jgaust > pvzl1qjjei5uuoi1c4g8tpea): Disconnected: Logged out bytes=11/313 Oct 12 > 04:36:02 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjj > fzonng9...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:36:03 scene dovecot: > IMAP(__cpanel__service__auth__imap__ouonjmdpliwgyj8ij6gucv6y7fxq6ojdk9hsxjjf > zonng9eqrsw5l5hg7xoejer2): Disconnected: Logged out bytes=11/340 Oct 12 > 04:41:03 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gw > eibvov1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:41:04 scene dovecot: > IMAP(__cpanel__service__auth__imap__wpwfzoyvnb2rsz2desu10swelok4cdwrqqw70gwe > ibvov1_minfh7j4_4ejaz7v2): Disconnected: Logged out bytes=11/340 Oct 12 > 04:46:04 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhd > rhafjur...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:46:05 scene dovecot: > IMAP(__cpanel__service__auth__imap__razxoe9ffiqhzj6rahuftwwqprhj2blovjvsbhdr > hafjureydiuxbtbk2jkpkvlo): Disconnected: Logged out bytes=11/340 Oct 12 > 04:51:06 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrq > h1iztqo...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:51:06 scene dovecot: > IMAP(__cpanel__service__auth__imap__as64dk7mx4gfxupigti8wwrbqpqhetm9zyhzlrqh > 1iztqosnzfwt28kkzv4riyd9): Disconnected: Logged out bytes=11/340 Oct 12 > 04:56:07 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan > 2pdmdvg...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 04:56:07 scene dovecot: > IMAP(__cpanel__service__auth__imap__x0do00ujo_2w89kbu0kfk6s8evtdz9u3davldan2 > pdmdvg6jofzylncdb3ytjaaz): Disconnected: Logged out bytes=11/340 Oct 12 > 05:01:08 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvir > quwcv1z...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:01:08 scene dovecot: > IMAP(__cpanel__service__auth__imap__lkup_seowtz4xhnmf5fkihlw6hpacvnfyeyzvirq > uwcv1zhloqt12rqni_o2pqcj): Disconnected: Logged out bytes=11/340 Oct 12 > 05:06:09 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgr > cpi3usb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:06:09 scene dovecot: > IMAP(__cpanel__service__auth__imap__yl1ivh4akxdj7vph6nz2w8jdmeuzukqkvezokgrc > pi3usbdjkiy2n8zy2bbvhhny): Disconnected: Logged out bytes=11/340 Oct 12 > 05:11:10 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg > 4vew3df...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:11:10 scene dovecot: > IMAP(__cpanel__service__auth__imap__z0jltexjylqusc5j7qrtw5t7m_tqzcjyheljxrg4 > vew3dfbyrglsasuldldaspck): Disconnected: Logged out bytes=11/340 Oct 12 > 05:16:11 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0 > e2bwtas...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:16:11 scene dovecot: > IMAP(__cpanel__service__auth__imap__0ycn3mqrthjv1aqo7w45zy1ndeodd2xxh92y3v0e > 2bwtastu0kton3azlhmmuhwi): Disconnected: Logged out bytes=11/340 Oct 12 > 05:21:12 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszq > mf4h8z1...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:21:12 scene dovecot: > IMAP(__cpanel__service__auth__imap__ofmf6ptqgyhrzgrlsx3p33fz52q7j1afid0uszqm > f4h8z1shjl34q9zpid3g4gsp): Disconnected: Logged out bytes=11/340 Oct 12 > 05:26:13 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaog > ha3yc0w...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:26:13 scene dovecot: > IMAP(__cpanel__service__auth__imap__awz7xewlgj6pjfdz1dogl28vm9ld5fqfwviyaogh > a3yc0w6t7vvgyf1snz8vechf): Disconnected: Logged out bytes=11/340 Oct 12 > 05:31:14 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62 > mucudsm...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:31:15 scene dovecot: > IMAP(__cpanel__service__auth__imap__uyfeustvewbyjgjyyrqgrflkxhuxfloywsash62m > ucudsmjfmyolzcpm9shakkiw): Disconnected: Logged out bytes=11/340 Oct 12 > 05:36:15 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxc > kmn7fkd...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:36:16 scene dovecot: > IMAP(__cpanel__service__auth__imap__g6g0w1ikkl7h2dx3uw8hdpin8uhjml0lgk08zxck > mn7fkdpsvbrjptqanfuljfv2): Disconnected: Logged out bytes=11/340 Oct 12 > 05:41:15 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqun > f1neeqb...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:41:16 scene dovecot: > IMAP(__cpanel__service__auth__imap__hucgkkpt7ggbzuudcxecprlhdf_c1qenb56hqunf > 1neeqbmzas00uqbzmmjsxiab): Disconnected: Logged out bytes=11/340 Oct 12 > 05:46:17 scene dovecot: imap-login: Login: > user=<__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfre > yewqvxx...>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 12 > 05:46:17 scene dovecot: > IMAP(__cpanel__service__auth__imap__09irfuxyirdwosv6_f9prfvxpxhozr8qfauvfrey > ewqvxxeoo7yhyiki7ghukxss): Disconnected: Logged out bytes=11/340 Oct 12 > 05:47:27 scene dovecot: dovecot: Fatal: Time just moved backwards by 35 > seconds. This might cause a lot of problems, so I'll just kill myself now. > http://wiki.dovecot.org/TimeMovedBackwards > > > > From jbates at brightok.net Wed Oct 17 16:44:38 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 08:44:38 -0500 Subject: [Dovecot] Per user quotas In-Reply-To: <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> Message-ID: <507EB646.5090702@brightok.net> My recommendation is that you use Passwd-file instead of Passwd and specify /etc/passwd. I mention this, as Passwd can use NSS and may not give you the results you want. Passwd-file will guarantee you use the /etc/passwd file. Also, I'm not as familiar with v1.x, but I know in v2.1.10, a userdb lookup doesn't use the userdb_ prefix. So you can try it with and without that prefix. userdb_ prefix on v2 is for cases where you do a prefetch on the passdb. I hope this helps. I've been using Passwd-file only for proxy settings and ldap for my backends to handle quota. Jack On 10/17/2012 2:42 AM, David Travers wrote: > Hi Jake, > > Yep, similar to what I had been trying, but it doesn't seem to be working. > > In my /etc/passwd file I had the line:- > dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash > > I have changed it to show:- > dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M > > Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? > > Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange > > Also yes, I noticed the numbering and have corrected. > > Thanks for your quick reply. > > Dave > > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates > Sent: 17 October 2012 04:02 > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Per user quotas > > On 10/16/2012 11:39 AM, David Travers wrote: >> Hi All, >> >> I keep going round in circles with this. >> I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! >> >> I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. >> >> I believe I need to add to a passwd file, but I have added to that and nothing has changed. >> >> > user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M > > Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields > > Note that the extra fields are prefixed with userdb_ when placed in a passwd file. > > Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). > > > Jack > > > > _________________________________________ > This email has been scanned for malicious content. > _________________________________________ > From dg at dguhl.org Wed Oct 17 17:51:44 2012 From: dg at dguhl.org (Dennis Guhl) Date: Wed, 17 Oct 2012 16:51:44 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350429674.3360.27.camel@fermat.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> Message-ID: <20121017145144.GA777@PC211.ikt.de> On Wed, Oct 17, 2012 at 01:21:14AM +0200, Christoph Anton Mitterer wrote: > Hi. [..] > First thing I tried was to simply copy mail within Evolution (i.e. > dragging&dropping it from the local folders to the IMAP folders from > dovecot). This seems to be the smartest idea. > - that preserves the status from Evolution, but doesn't restore that > from Thunderbird Why not use TB to copy the emails from your 'TB mboxes' to Dovecot? This way I moved around 25 GiB of emails from >> 50 mbox files, created with TB 3.6 way down to some 0.x beta, to Dovecot -- without any issues I could recall. [..] > - neither does it remove the Thunderbird or the X-Evolution-Source > headers If they bug you remove them with sed or awk or perl or python or ... Dennis [..] From calestyo at scientia.net Wed Oct 17 20:57:38 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 19:57:38 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121017145144.GA777@PC211.ikt.de> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1350496658.27003.6.camel@heisenberg.scientia.net> On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: > > First thing I tried was to simply copy mail within Evolution (i.e. > > dragging&dropping it from the local folders to the IMAP folders from > > dovecot). > This seems to be the smartest idea. Well as I've mentioned... on looses the info in the From_ lines (that is the RCPT TO address and the date of arrival) because Evolution does not correctly migrated them (actually I'm not sure whether IMAP would allow that). > > - that preserves the status from Evolution, but doesn't restore that > > from Thunderbird > Why not use TB to copy the emails from your 'TB mboxes' to Dovecot? > This way I moved around 25 GiB of emails from >> 50 mbox files, > created with TB 3.6 way down to some 0.x beta, to Dovecot -- without > any issues I could recall. Sorry... too late for that... cause back in the "old" days when I went away from TB I didn't notice that the used other mail headers for their statuses... so now everthing is already mixed together. > If they bug you remove them with sed or awk or perl or python or ... Yeah... but sed alone is not enough... cause such lines may also appear in the body... and I mustn't remove them... So in principle I'm looking for a smart parser of mbox which already gives me headers and body and I can modify either. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From rob0 at gmx.co.uk Wed Oct 17 21:12:36 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 17 Oct 2012 13:12:36 -0500 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350496658.27003.6.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> Message-ID: <20121017181236.GN3672@harrier.slackbuilds.org> On Wed, Oct 17, 2012 at 07:57:38PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: > > > First thing I tried was to simply copy mail within Evolution > > > (i.e. dragging&dropping it from the local folders to the IMAP > > > folders from dovecot). > > This seems to be the smartest idea. > Well as I've mentioned... on looses the info in the From_ lines > (that is the RCPT TO address and the date of arrival) because > Evolution does not correctly migrated them (actually I'm not sure > whether IMAP would allow that). Perhaps you mean the "^From " mbox delimiter line. You do not need mbox delimiters in maildir files. Did you mention whether or not you're using maildir? -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From calestyo at scientia.net Wed Oct 17 21:21:47 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 17 Oct 2012 20:21:47 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121017181236.GN3672@harrier.slackbuilds.org> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> Message-ID: <1350498107.27003.10.camel@heisenberg.scientia.net> On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: > > Well as I've mentioned... on looses the info in the From_ lines > > (that is the RCPT TO address and the date of arrival) because > > Evolution does not correctly migrated them (actually I'm not sure > > whether IMAP would allow that). > Perhaps you mean the "^From " mbox delimiter line. Yes I meant them (the _ should have denoted the space) > You do not need > mbox delimiters in maildir files. I know.. > Did you mention whether or not > you're using maildir? The reason is mainly that I have gazillions of mail in a ~ 60 GB archive... even with an fs optimised for small files I'd loose far too much space per mail than I want to afford. Also, AFAIK full text search becomes much solver in maildir (as you need to open/close endless files). On the longterm view I want to have a look into things like dbmail/archiveopteryx... for the giant local archive... and keep dovecot "only" as the internet mail server. Ideally dovecot would have such an SQL backend...or incorporate that part from Archiveopteryx. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From rob0 at gmx.co.uk Wed Oct 17 21:36:47 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 17 Oct 2012 13:36:47 -0500 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350498107.27003.10.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> <1350498107.27003.10.camel@heisenberg.scientia.net> Message-ID: <20121017183647.GO3672@harrier.slackbuilds.org> On Wed, Oct 17, 2012 at 08:21:47PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: > > Did you mention whether or not you're using maildir? > The reason is mainly that I have gazillions of mail in a ~ 60 GB > archive... even with an fs optimised for small files I'd loose far > too much space per mail than I want to afford. Fine, maildir is not the perfect solution for everyone. But I'm confused about why Evolution/Thunderbird local folders to IMAP folders does not work. That should be the best approach. If it does not work, you're going to have some perl/python/ruby scripting to do. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From list at airstreamcomm.net Wed Oct 17 22:27:04 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 17 Oct 2012 14:27:04 -0500 Subject: [Dovecot] Dsync clustering Message-ID: <507F0688.6000707@airstreamcomm.net> I have not seen mention of using dsync for clustering Dovecot in some time on the mailing list, but I believe Timo was going to write a wiki page when v2.2 became more mature. Does this documentation exist yet, or are there any resources on what dsync replication is capable of at this point (looking on the wiki and google didn't reveal much)? Thank in advance. From roundcube222 at alaadin.org Wed Oct 17 21:33:42 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 17 Oct 2012 21:33:42 +0300 Subject: [Dovecot] Disconnected for inactivity time. Message-ID: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> Hello, After 10 mins of unactivity of pop3 , dovecot disconnect the user (-ERR Disconnected for inactivity.) What is the option in the config file which control the unactivity logout time ? becuase i want to decrease the inactivity time To 5 mins instead of 10 mins Please advise Robert JR From slusarz at curecanti.org Wed Oct 17 22:47:43 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Wed, 17 Oct 2012 13:47:43 -0600 Subject: [Dovecot] Disconnected for inactivity time. In-Reply-To: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> References: <763dcf2f1e07f6443bac14d46fc207f2@Coptics.org> Message-ID: <20121017134743.Horde.af5lO4F5lbhQfwtfwboGUbA@bigworm.curecanti.org> Quoting Robert JR : > After 10 mins of unactivity of pop3 , dovecot disconnect the user > (-ERR Disconnected for inactivity.) > > What is the option in the config file which control the unactivity > logout time ? becuase i want to decrease the inactivity time > > To 5 mins instead of 10 mins You can't (at least without hacking the code). The POP3 specification **requires** a minimum of 10 minutes before an autologout occurs (RFC 1939 [3]): http://tools.ietf.org/html/rfc1939#section-3 michael From stephan at rename-it.nl Wed Oct 17 23:12:00 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 17 Oct 2012 22:12:00 +0200 Subject: [Dovecot] Dovecot sieve and duplicate email subjects In-Reply-To: References: Message-ID: <507F1110.3020701@rename-it.nl> On 8/27/2012 12:40 PM, Benjamin Thomas wrote: > I was wondering if it's possible to gracefully handle duplicate subjects > lines (within a given time frame) with sieve filters ? > > Ideally, I would like the first email of the day with subject ""AutoAlert > Type1..." to get moved to the subfolder given above. Subsequent "duplicate" > emails would then go into a "duplicate" subfolder. > > Is this possible with sieve filters ? I could not find anything striking me > as obvious while reading the page http://wiki.dovecot.org/LDA/Sieve. Although the above is still not possible with the standard Sieve features, I updated the Dovecot-specific vnd.dovecot.duplicate extension with support for the scenario you describe. Basically, I made it much more flexible for checking all kinds of string value duplicates and not only strictly based on the Message-ID header. Also, the user can now directly control the time frame within which duplicates are detected (within configurable limits). http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt It will be included in the next Pigeonhole v0.3 release. Regards, Stephan. From limon at koli.be Thu Oct 18 01:40:45 2012 From: limon at koli.be (Levent Dane) Date: Wed, 17 Oct 2012 17:40:45 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c Message-ID: <456733b1b04e92265fbd9ba8e005132c@koli.be> First, I don't know why but dovecot gots this error. Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) Then, when I tried to run SELECT "INBOX/Code" (UNSEEN) virtual plugin got a segfault. Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] I'm running gentoo with kernel 2.6.32.12. My dovecot version is 2.1.10. I applied http://hg.dovecot.org/dovecot-2.1/raw-diff/0306792cc843/src/lib-storage/mail-search.c But still, i'm getting this problem. Thanks, Levent Dane From mailadmin at cubixys.com Thu Oct 18 02:16:55 2012 From: mailadmin at cubixys.com (Fasil) Date: Thu, 18 Oct 2012 02:16:55 +0300 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> Message-ID: <507F3C67.5020900@cubixys.com> Thanks Frank. Followed the URL and could not find any luck. Is there a way to change the value of 'max open files' of dovecot. I have tried to set the value in /etc/default/dovecot by setting ulimit. But the value is not getting applied. Could anyone help on this regard. Fasil. On 08/16/2012 09:17 AM, Frank Bonnet wrote: > hello > > here some useful informations > > http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ > > Envoy? de mon iPhone. > > > Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : > >> Dear all, >> >> Thank you all for such a wonderful support. Hats off to all :) >> >> Few times I came across imap login issues where new users will not be allowed to login. >> /var/log/mail.err shows the error below >> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in creating login processes, slowing down for now >> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open files >> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open files >> >> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >> >> # ulimit -Hn >> 1024 >> >> # ulimit -Sn >> 1024 >> >> # cat /proc/sys/fs/file-max >> 1238548 >> >> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >> Max open files 1024 1024 files >> >> Please advice how to get rid off this. >> >> Fasil. From jtam.home at gmail.com Thu Oct 18 02:46:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 17 Oct 2012 16:46:30 -0700 (PDT) Subject: [Dovecot] Disconnected for inactivity time. In-Reply-To: References: Message-ID: Robert JR writes: > After 10 mins of unactivity of pop3 , dovecot disconnect the user (-ERR > Disconnected for inactivity.) > > What is the option in the config file which control the unactivity > logout time ? becuase i want to decrease the inactivity time > > To 5 mins instead of 10 mins Looks like it's set in the source code pop3-client.c:#define CLIENT_IDLE_TIMEOUT_MSECS (10*60*1000) but the output of # doveconf -a ... service pop3 { ... idle_kill = 0 ... } maybe points to the config that overrides this. Easy enough to test. It's non-RFC compliant as one poster points out, so unless you got a good reason to do this like lots of zombie pop3 processes, leave it alone. Joseph Tam From web at guzman.com.ar Thu Oct 18 02:47:09 2012 From: web at guzman.com.ar (Ricardo) Date: Wed, 17 Oct 2012 20:47:09 -0300 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze Message-ID: Hello list apologize in advance for my bad English, this is the first time I write to a list if I mistake Excuse me I want to implement mail server with MySQL database, Postfix and Postfixadmin, dovecot-core, dovecot-mysql dovecot-imapd dovecot-pop3d for multiple domains. I have problems installing the daemon dovecot-core, dovecot-mysql dovecot-imapd dovecot-pop3d Debian squeeze, I'm using the repositories: deb http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze main deb-src http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze main deb http://security.debian.org/ squeeze/updates main deb-src http://security.debian.org/ squeeze/updates main # squeeze-updates, previously known as 'volatile' deb http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze-updates main deb-src http://ftp.ccc.uba.ar/pub/linux/debian/debian/ squeeze-updates main deb http://packages.dotdeb.org squeeze all deb-src http://packages.dotdeb.org squeeze all deb http://backports.debian.org/debian-backports squeeze-backports main to install dovecot-core, dovecot-mysql, install it without problems is the version (2.1.7-2 ~ bpo60 +1) of both packages, now wanting to install dovecot-imapd dovecot-pop3d (version 1.2.15-7) breaks the dovecot-core, dovecot-mysql, apparently must be the same version all packages. Debian Wheezy, installs without problems but installs the version (dovecot-core_2.1.7-2 ~ ppa12.04 +1 _i386.deb) all packets are the same version. What is the correct version for Debian squeeze? Look for San Google but eh had success. Can anybody help? Ricardo From rfs9999 at earthlink.net Wed Oct 17 18:53:24 2012 From: rfs9999 at earthlink.net (Rick Sanders) Date: Wed, 17 Oct 2012 15:53:24 +0000 (UTC) Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: Your best bet for a clean migration is to use an IMAP migration tool (assuming both of your servers support IMAP). It avoids all of the issues surrounding the underlying databases used to store the mailboxes and messages since everything is done through IMAP commands. There are lots of different IMAP tools out there, some free some not. Using an IMAP migration tool is usually straight-forward and simple. Here is a list of some of them. Full disclosure, imap_tools is mine. imapsync: http://imapsync.lamiral.info imap_tools: http://www.athensfbc.com/imap_tools offlineimap: https://github.com/nicolas33/offlineimap mbsync: http://isync.sourceforge.net/ mailsync: http://mailsync.sourceforge.net/ mailutil: http://www.washington.edu/imap/ part of the UW IMAP tookit. imaprepl: http://www.bl0rg.net/software/ http://freecode.com/projects/imap-repl/ imapcopy: http://home.arcor.de/armin.diehl/imapcopy/imapcopy.html migrationtool: http://sourceforge.net/projects/migrationtool/ imapmigrate: http://sourceforge.net/projects/cyrus-utils/ larch: https://github.com/rgrove/larch (derived from wonko_imapsync) wonko_imapsync: http://wonko.com/article/554 pop2imap: http://www.linux-france.org/prj/pop2imap/ exchange-away: http://exchange-away.sourceforge.net/ To copy all of a user's mailboxes from one IMAP server to another using my imapcopy tool is as simple as executing the following command: imapcopy.pl -S source/username/password -D destination/user/password Regards, Rick From jbates at brightok.net Thu Oct 18 05:13:25 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 21:13:25 -0500 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <507F3C67.5020900@cubixys.com> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> <507F3C67.5020900@cubixys.com> Message-ID: <507F65C5.3090803@brightok.net> I'm using RHEL6 instead of ubuntu, but check the startup scripts. In RHEL's case, the following file is sourced, so I updated it instead of the startup scripts. cat /etc/sysconfig/dovecot # Here you can specify your dovecot command line options. # #OPTIONS="" ulimit -n 4096 ulimit -u 5120 In addition, I had to also up the max allowed processes in the dovecot config. 2.x and 1.x are different on this. http://wiki1.dovecot.org/LoginProcess <-1.x method Jack On 10/17/2012 6:16 PM, Fasil wrote: > Thanks Frank. > Followed the URL and could not find any luck. > > Is there a way to change the value of 'max open files' of dovecot. > I have tried to set the value in /etc/default/dovecot by setting > ulimit. But the value is not getting applied. > Could anyone help on this regard. > > Fasil. > > On 08/16/2012 09:17 AM, Frank Bonnet wrote: >> hello >> >> here some useful informations >> >> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ >> >> Envoy? de mon iPhone. >> >> >> Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : >> >>> Dear all, >>> >>> Thank you all for such a wonderful support. Hats off to all :) >>> >>> Few times I came across imap login issues where new users will not >>> be allowed to login. >>> /var/log/mail.err shows the error below >>> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in creating >>> login processes, slowing down for now >>> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open >>> files >>> >>> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >>> >>> # ulimit -Hn >>> 1024 >>> >>> # ulimit -Sn >>> 1024 >>> >>> # cat /proc/sys/fs/file-max >>> 1238548 >>> >>> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >>> Max open files 1024 1024 files >>> >>> Please advice how to get rid off this. >>> >>> Fasil. > > From jbates at brightok.net Thu Oct 18 05:30:58 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 21:30:58 -0500 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze In-Reply-To: References: Message-ID: <507F69E2.7050801@brightok.net> On 10/17/2012 6:47 PM, Ricardo wrote: > > to install dovecot-core, dovecot-mysql, install it without problems is > the version (2.1.7-2 ~ bpo60 +1) of both packages, > now wanting to install dovecot-imapd dovecot-pop3d (version 1.2.15-7) > breaks the dovecot-core, dovecot-mysql, > apparently must be the same version all packages. > > I think something is wrong with your debian setup. 1.2 is normal version. 2.1 is the backports version. You should be getting this: Package: dovecot-imapd Priority: optional Section: mail Installed-Size: 559 Maintainer: Dovecot Maintainers Architecture: i386 Source: dovecot Version: 1:2.1.7-2~bpo60+1 Provides: imap-server Depends: dovecot-core (= 1:2.1.7-2~bpo60+1), libc6 (>= 2.4), ucf (>= 2.0020) That's from the Packages list http://backports.debian.org/debian-backports/dists/squeeze-backports/main/binary-i386/ Jack From tss at iki.fi Thu Oct 18 06:06:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:06:27 +0300 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <456733b1b04e92265fbd9ba8e005132c@koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> Message-ID: <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> On 18.10.2012, at 1.40, Levent Dane wrote: > First, I don't know why but dovecot gots this error. > Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) > Then, when I tried to run > SELECT "INBOX/Code" (UNSEEN) > virtual plugin got a segfault. > Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html From tss at iki.fi Thu Oct 18 06:07:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:07:39 +0300 Subject: [Dovecot] Dsync clustering In-Reply-To: <507F0688.6000707@airstreamcomm.net> References: <507F0688.6000707@airstreamcomm.net> Message-ID: On 17.10.2012, at 22.27, list at airstreamcomm.net wrote: > I have not seen mention of using dsync for clustering Dovecot in some time on the mailing list, but I believe Timo was going to write a wiki page when v2.2 became more mature. Does this documentation exist yet, or are there any resources on what dsync replication is capable of at this point (looking on the wiki and google didn't reveal much)? Thank in advance. You can probably find some mails from this mailing list. Try searching for "dsync replication". From tss at iki.fi Thu Oct 18 06:10:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:10:03 +0300 Subject: [Dovecot] Per user quotas In-Reply-To: <507EB646.5090702@brightok.net> References: <9E98F3EACD0C344685A2A32DE106873649A53985@LYNEX02.cohenschemist.co.uk> <507E1F7D.4080107@brightok.net> <9E98F3EACD0C344685A2A32DE106873649A54D40@LYNEX02.cohenschemist.co.uk> <507EB646.5090702@brightok.net> Message-ID: Correct, except I wouldn't go modifying /etc/passwd directly in any case. Other software besides Dovecot might not like it. Better to create a whole new /etc/dovecot/passwd or something. On 17.10.2012, at 16.44, Jack Bates wrote: > My recommendation is that you use Passwd-file instead of Passwd and specify /etc/passwd. I mention this, as Passwd can use NSS and may not give you the results you want. Passwd-file will guarantee you use the /etc/passwd file. Also, I'm not as familiar with v1.x, but I know in v2.1.10, a userdb lookup doesn't use the userdb_ prefix. So you can try it with and without that prefix. userdb_ prefix on v2 is for cases where you do a prefetch on the passdb. > > I hope this helps. I've been using Passwd-file only for proxy settings and ldap for my backends to handle quota. > > Jack > > On 10/17/2012 2:42 AM, David Travers wrote: >> Hi Jake, >> >> Yep, similar to what I had been trying, but it doesn't seem to be working. >> >> In my /etc/passwd file I had the line:- >> dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash >> >> I have changed it to show:- >> dave:x:1000:1000:David Travers,,,:/home/dave:/bin/bash:userdb_quota_rule=*:storage=100M >> >> Do I have to put the " userdb_mail=mbox:~/mail " in as well as that is specified already in Dovecot? >> >> Is there anything I need to do once specifying this in the passwd file as the quota limit is not being shown as changed in Open Xchange >> >> Also yes, I noticed the numbering and have corrected. >> >> Thanks for your quick reply. >> >> Dave >> >> -----Original Message----- >> From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Jack Bates >> Sent: 17 October 2012 04:02 >> To: dovecot at dovecot.org >> Subject: Re: [Dovecot] Per user quotas >> >> On 10/16/2012 11:39 AM, David Travers wrote: >>> Hi All, >>> >>> I keep going round in circles with this. >>> I have quotas up and running but would like to add a couple of per user exceptions but can't figure out how to do it! >>> >>> I am using Open Xchange community edition on top of dovecot and tha tis showing the 200MB limit, if I change it in the dovecot.conf the change is reflected in open xchange, but can't figure out how to change for 1 user. >>> >>> I believe I need to add to a passwd file, but I have added to that and nothing has changed. >>> >>> >> user:{plain}pass:1000:1000::/home/user::userdb_mail=mbox:~/mail userdb_quota_rule=*:storage=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_mail=maildir:~/Maildir userdb_quota_rule=*:storage=200M >> >> Example given on http://wiki.dovecot.org/UserDatabase/ExtraFields >> >> Note that the extra fields are prefixed with userdb_ when placed in a passwd file. >> >> Also, watch your quota_rule numbering. You have 2 rules with the same number (quota_rule2 for Trash and SPAM). In the passwd file, you are replacing the rule specified by number (no number technically being the first). >> >> >> Jack >> >> >> >> _________________________________________ >> This email has been scanned for malicious content. >> _________________________________________ >> > From tss at iki.fi Thu Oct 18 06:14:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:14:19 +0300 Subject: [Dovecot] Difference between Indexing and Rescan in FTS In-Reply-To: <00e101cdac30$5ab63270$10229750$@fredk.com> References: <007f01cdabf3$efc6fad0$cf54f070$@fredk.com> <00e101cdac30$5ab63270$10229750$@fredk.com> Message-ID: On 17.10.2012, at 9.26, Fred Kilbourn wrote: >> doveadm fts rescan makes sure that 1) all of the old messages are >> indexed and 2) there are no extra (already deleted) messages indexed. So >> it's basically repairing fts index. You probably shouldn't run it >> automatically, or at least not very often. > > Okay, you've clarified that for me. > > I understand that rescan isn't a nightly task, but could be run every now > and then periodically. How often might be appropriate if I wanted to do > this as a maintenance task? Once a month? I don't know, depends on if you have problems related to it. I think the most common answer would be "never". > Lastly, I'm trying to use the index command instead of the search command, > but I can't figure out how to make it index every mailbox for every user. > Is there a wildcard that can be used for the mailbox? Or do I need to > iterate all the mailboxes with one command and run index however many times > for each inbox? doveadm index '*' works in new versions. I don't remember from which version. From tss at iki.fi Thu Oct 18 06:18:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:18:49 +0300 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <20121016091153.15601eysq5n040qh@webmail.unipa.it> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> <20121016091153.15601eysq5n040qh@webmail.unipa.it> Message-ID: <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> On 16.10.2012, at 10.11, Benedetto Vassallo wrote: >> What are the permissions of the MailDir directory for user1/user2? >> >> ls -ld /home/user1/MailDir >> ls -ld /home/user2/MailDir >> >> > > Thank you for your reply. > They are different groups: > > drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ > drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ > drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ Not very secure permissions.. Maybe would be easiest to just have one vmail user for everyone? > I tryed to issue: > chgrp -R mail /home/user1/MailDir > chgrp -R mail /home/user2/MailDir > chgrp -R mail /home/user3/MailDir Dovecot doesn't do hard linking when it looks like the permissions aren't compatible. The current code checks that if the owner UIDs are different, then the group needs to be writable. From tss at iki.fi Thu Oct 18 06:23:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:23:58 +0300 Subject: [Dovecot] real_rip variable addition for dovecot 2.1.10 In-Reply-To: <507DC6D4.7090902@brightok.net> References: <507DC6D4.7090902@brightok.net> Message-ID: <23D747EC-045A-44B0-8A85-29B76B66B969@iki.fi> On 16.10.2012, at 23.43, Jack Bates wrote: > Please check the code. I didn't add it, but a real_lip might be useful for some people as well. Also, I notice that pop3-proxy is doing a different xsession than the imap proxy. Is there an xsession standard that is different between the two, or just an oversight in the code? Both send the remote address/port, but only imap proxy sends the local address/port. > > This patch declares long variable %{real_rip} so that a backend server can declare a different login_log_format_elements > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c > > This is primarily useful for backend servers to log both the rip, lip, and in case of xsession, the real rip. I haven't done extensive testing yet, but as long as nothing does anything weird elsewhere in the code, it should be good. > > http://www.brightok.net/jbates/dovecot-2.1.10-real-ip.patch Added: http://hg.dovecot.org/dovecot-2.1/rev/92364817f4ba From tss at iki.fi Thu Oct 18 06:31:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 06:31:19 +0300 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read In-Reply-To: <507EA81C.5060806@froglogic.com> References: <507EA81C.5060806@froglogic.com> Message-ID: On 17.10.2012, at 15.44, Frerich Raabe wrote: > I'm running Dovecot 1.2.17 on FreeBSD 9 to serve an archive of a few internal mailinglists. The archive is implemented using a public namespace: > > namespace private { > separator = / > prefix = > inbox = yes > } > > namespace public { > separator = / > prefix = Lists/ > location = maildir:/home/vmail/lists/Maildir:CONTROL=~/Maildir/lists:INDEX=~/Maildir/lists > subscriptions = no > } > > As you can see, the CONTROL/INDEX files are stored per-user to allow private \Seen flags. The different mailinglists are all sent to the 'lists' user which has a Sieve script to file them into different folders, so I have directories on my harddisk like > > /home/vmail/lists/Maildir/.somelist > /home/vmail/lists/Maildir/.anotherlist > > Now, I'd like to mark the mail in *one* of those folders as \Seen by default. If the INDEX files weren't per-user, it would simply be a matter of using 'addflag "\Seen";' in the Sieve script of the lists user. Alas, this has no effect. > > Hence my question - how can I have the mail of just one mailinglist get marked as "read" for all users? You can't with the above settings. It would require writing the seen flag to all users' index files. Not easy to do and definitely not efficient to do. Maybe if there was some kind of a mixed hybrid of accepting seen flag changes from the shared index, but no such code exists (also private/shared index separation doesn't exist before v2.2). > So far, the only option I see is to add a second public namespace, with a different prefix - and this namespace doesn't use private CONTROL/INDEX files. However, I'd like to keep using the "Lists" prefix if possible to avoid too many changes to the clients. Use: prefix=Lists/anotherlist/ location = maildir:/home/vmail/lists/sharedseen/Maildir Then deliver the mails to /home/vmail/lists/sharedseen/Maildir root directly. Of course this means that you need to create a namespace for each such list. Alternative would be to use prefix=Lists/sharedseen/ and create lists under it. From jbates at brightok.net Thu Oct 18 06:48:42 2012 From: jbates at brightok.net (Jack Bates) Date: Wed, 17 Oct 2012 22:48:42 -0500 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507C6DD3.2000309@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> Message-ID: <507F7C1A.9030301@brightok.net> Timo, How do you feel about parent pointers in child structures? I'm curious as the proxy structure is passed the input channel, but it doesn't know much else about the input client. Rather than pass additional information in the structure, I think it'd be better to just place a pointer back to the input client so we can access all it's details. I ask, as that might solve the problem of lack of information in logging from some of the various functions in the proxy code. I know I was limited in the quick patch I did below for my own use. I'm afraid to change it too much. You have already started xclient work in v2.2 which would necessitate a lot of changes to the lmtp/proxy code. I've actually debated backporting it to 2.1 for my own use. :) Jack On 10/15/2012 3:10 PM, Jack Bates wrote: > On 10/15/2012 2:07 PM, Jack Bates wrote: >> On 10/12/2012 2:40 AM, Timo Sirainen wrote: >>> would probably complicate the code. >>> I don't think this would be difficult to implement. Probably just a >>> few lines of code. Yeah, could be useful. >>> >>> >> > > If there's no argument over the last email, confirm and check this > patch. It's not the overall logging I would like, but the lmtp code > isn't as mature as pop3/imap and the proxy is a quick and dirty on the > lmtp code. Both need a good revamp, preferably with x-session support > and perhaps logging rip/lip similar to how we do pop3/imap logins. > > I think we should also work on adjusting all logging for services > using x-session to also log the proxy ip. rip,lip,pip. As I get time > I'll look at it. > > This patch is just to keep us from having no useful logging in lmtp > proxy. Based on lmtp pid, one can at least follow the connect, the > proxy replies, and the disconnect of a session. > > --- dovecot-2.1.10/src/lmtp/lmtp-proxy.c 2012-10-12 > 19:46:49.688952484 +0000 > +++ dovecot-2.1.10/src/lmtp/lmtp-proxy.c-new 2012-10-12 > 19:48:51.751932325 +0000 > @@ -160,6 +160,8 @@ static bool lmtp_proxy_send_data_replies > break; > o_stream_send_str(proxy->client_output, > t_strconcat(rcpt[i]->reply, "\r\n", > NULL)); > + i_info("proxy(%s): proxy host=%s: > status=%s",rcpt[i]->address, > + rcpt[i]->conn->set.host,rcpt[i]->reply); > } > o_stream_uncork(proxy->client_output); > proxy->next_data_reply_idx = i; > > From tss at iki.fi Thu Oct 18 07:07:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 07:07:03 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507F7C1A.9030301@brightok.net> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> Message-ID: On 18.10.2012, at 6.48, Jack Bates wrote: > How do you feel about parent pointers in child structures? I'm curious as the proxy structure is passed the input channel, but it doesn't know much else about the input client. Rather than pass additional information in the structure, I think it'd be better to just place a pointer back to the input client so we can access all it's details. Generally speaking it's cleaner to keep things as separate as possible. Maybe instead of proxy getting lmtp_client pointer both of them could contain a shared struct lmtp_client_info or something like that. But in any case I'll probably more or less rewrite the whole LMTP code at some point, because I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) From limon at koli.be Thu Oct 18 09:03:55 2012 From: limon at koli.be (Levent Dane) Date: Thu, 18 Oct 2012 01:03:55 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> Message-ID: <20121018060354.GA2528@leningrad.koli.be> On 10/18, Timo Sirainen wrote: >On 18.10.2012, at 1.40, Levent Dane wrote: >> First, I don't know why but dovecot gots this error. >> Oct 15 13:24:02 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) >> Then, when I tried to run >> SELECT "INBOX/Code" (UNSEEN) >> virtual plugin got a segfault. >> Oct 15 13:24:03 widder kernel: imap[22749]: segfault at 2c ip b757f8ec sp bfa3a160 error 4 in lib20_virtual_plugin.so[b7579000+d000][b74b0000+d000] > >I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html in Code/dovecot-virtual: Archive inthread refs keyword code not deleted dovecot -n http://pastebin.com/6CQd7tJK My mail client is Mutt-hg with sidebar patch I tried to take coredump but i didn't compile with debug flags. http://pastebin.com/CMbiYJeK If you can't reproduce this error. Tomorrow, I'll compile with debug flags. Thanks, -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 From janfrode at tanso.net Thu Oct 18 11:05:44 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 18 Oct 2012 10:05:44 +0200 Subject: [Dovecot] trash plugin not doing it's job Message-ID: I enabled the trash plugin yesterday, adding "trash" to mail_plugins, and configuring the plugin setting "trash = /etc/dovecot/dovecot-trash.conf.ext". But I still see users with lots of files in INBOX.Trash getting bounced because of quota exceeded: postfix/lmtp[26273]:: C89F490061: to=, relay=loadbalancers.example.net[192.168.42.15]:24, delay=1.2, delays=0.61/0.02/0/0.54, dsn=5.2.2, status=bounced (host loadbalancers.example.net[192.168.42.15] said: 552 5.2.2 Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) dovecot:: lmtp(19730, XXXXXXX at example.no): Error: BErxFCyrf1ASTQAAWNPRnw: sieve: msgid=: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) $ sudo doveadm quota get -u XXXXXXXX at example.no Quota name Type Value Limit % UserQuota STORAGE 1048559 1048576 99 UserQuota MESSAGE 4487 - 0 Postfix if delivering via LMTP trough dovecot director. Anybody see anything obvious in my config: ------------------------------------------------------------ # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) auth_cache_size = 100 M auth_verbose = yes auth_verbose_passwords = sha1 disable_plaintext_auth = no login_trusted_networks = 192.168.0.0/16 109.247.114.192/27 mail_gid = 3000 mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u mail_max_userip_connections = 20 c = quota zlib trash mail_uid = 3000 maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = dict:UserQuota::file:%h/dovecot-quota sieve = /sieve/%1Lu/%1.1Lu/%Lu/.dovecot.sieve sieve_before = /etc/dovecot/sieve/dovecot.sieve sieve_dir = /sieve/%1Lu/%1.1Lu/%Lu sieve_max_script_size = 1M trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at example.net protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 4521 unix_listener auth-userdb { group = mode = 0600 user = atmail } } service imap-login { inet_listener imap { address = * port = 143 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login /usr/local/sbin/imap-postlogin.sh } service imap { executable = imap imap-postlogin process_limit = 2048 } service lmtp { client_limit = 1 inet_listener lmtp { address = * port = 24 } process_limit = 25 process_min_avail = 10 } service managesieve-login { inet_listener sieve { address = * port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service pop3-postlogin { executable = script-login /usr/local/sbin/pop3-postlogin.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 2048 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota zlib trash sieve } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota zlib trash imap_quota } protocol pop3 { mail_plugins = quota zlib trash pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v } protocol sieve { managesieve_logout_format = bytes=%i/%o } ------------------------------------------------------------ and my trash config: $ cat /etc/dovecot/dovecot-trash.conf.ext # Spam mailbox is emptied before Trash 1 INBOX.Spam # Trash mailbox is emptied before Sent 2 INBOX.Trash Global sieve script: $ cat /etc/dovecot/sieve/dovecot.sieve require ["comparator-i;ascii-numeric","relational","fileinto","mailbox"]; if allof ( not header :matches "x-spam-score" "-*", header :value "ge" :comparator "i;ascii-numeric" "x-spam-score" "10" ) { discard; stop; } elsif allof ( not header :matches "x-spam-score" "-*", header :value "ge" :comparator "i;ascii-numeric" "x-spam-score" "6" ) { fileinto :create "INBOX.Spam"; } -jf From stocton12 at yahoo.com Thu Oct 18 11:33:25 2012 From: stocton12 at yahoo.com (b m) Date: Thu, 18 Oct 2012 01:33:25 -0700 (PDT) Subject: [Dovecot] CAS Authentication In-Reply-To: <507E5D3A.5030900@um.es> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> <507E5D3A.5030900@um.es> Message-ID: <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> Thanks for the configuration files. I have a question. In pam_cas.conf I don't know what to put in "proxy ". In some examples I have seen something like http:///proxy.php Do I need a php file in my webmail to handle the cas tickets and if so where can I find it? ________________________________ From: Angel L. Mateo To: dovecot at dovecot.org Sent: Wednesday, October 17, 2012 10:24 AM Subject: Re: [Dovecot] CAS Authentication El 16/10/12 20:12, b m escribi?: > Thanks for the reply. I have already tried successfully the setup without proxing the cas tickets and setting dovecot to login? with a master password. The problem is that I need a password file with all the users and also I need the proxy feature for other applications. > ??? This is my config. In /etc/pam.d/dovecot I have: auth? ? sufficient??? pam_cas_ssh.so -simap://localhost -f/etc/pam_cas.conf account sufficient??? pam_permit.so session sufficient??? pam_permit.so ??? and /etc/pam_cas.conf host port 443 uriValidate /cas/proxyValidate ssl on debug off proxy??? trusted_ca ??? in dovecot, I have these users dbs: userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } passdb { ? args = session=yes cache_key=%n dovecot ? driver = pam } ??? With this, it works fine. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From rs at sys4.de Thu Oct 18 11:42:56 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 18 Oct 2012 10:42:56 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350498107.27003.10.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121017181236.GN3672@harrier.slackbuilds.org> <1350498107.27003.10.camel@heisenberg.scientia.net> Message-ID: <507FC110.1040809@sys4.de> Am 17.10.2012 20:21, schrieb Christoph Anton Mitterer: > On Wed, 2012-10-17 at 13:12 -0500, /dev/rob0 wrote: >>> Well as I've mentioned... on looses the info in the From_ lines >>> (that is the RCPT TO address and the date of arrival) because >>> Evolution does not correctly migrated them (actually I'm not sure >>> whether IMAP would allow that). >> Perhaps you mean the "^From " mbox delimiter line. > Yes I meant them (the _ should have denoted the space) > > >> You do not need >> mbox delimiters in maildir files. > I know.. > > >> Did you mention whether or not >> you're using maildir? > The reason is mainly that I have gazillions of mail in a ~ 60 GB > archive... even with an fs optimised for small files I'd loose far too > much space per mail than I want to afford. > > Also, AFAIK full text search becomes much solver in maildir (as you need > to open/close endless files). On the longterm view I want to have a look > into things like dbmail/archiveopteryx... for the giant local archive... > and keep dovecot "only" as the internet mail server. > > Ideally dovecot would have such an SQL backend...or incorporate that > part from Archiveopteryx. > > > Cheers, > Chris. > this may help too http://www.stchman.com/export_evolution.html http://www.ubuntugeek.com/how-to-export-your-mails-from-evolution-to-thunderbird.html http://ubuntuforums.org/showthread.php?t=1760469 http://ubuntuforums.org/showthread.php?t=1870445 http://jaisejames.wordpress.com/2012/03/15/to-activate-maildir-in-thunderbird/ http://realtechtalk.com/ThunderbirdMBOX_to_IMAPMaildir_migration_done_easy_with_mb2md-1134-articles -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From fabio.ferrari at unimore.it Thu Oct 18 11:51:37 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Thu, 18 Oct 2012 10:51:37 +0200 Subject: [Dovecot] Problem with process_limit In-Reply-To: References: <19748af43b2e64680c728f5af50da879.squirrel@webmail2.unimore.it> Message-ID: <370893b18f6c82ba13f4cb31d19ea259.squirrel@webmail2.unimore.it> Yes, thanks, it seems that this configuration changed something, but I think there is something else. Now this particular warning in the dovecot.log disappeared, but it shows these lines instead: Oct 17 10:55:57 imap-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 17 10:55:57 imap-login: Fatal: Couldn't connect to anvil Oct 17 10:56:12 pop3-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 17 10:56:12 pop3-login: Fatal: Couldn't connect to anvil the result is quite the same, I have to reload the dovecot because it does'n accept connections. I tried to add these lines in /etc/dovecot/conf.d/10-master.conf: service anvil { client_limit = 5000 } but without good results. Any ideas? thanks in advance Fabio Ferrari > On 1.10.2012, at 12.15, FABIO FERRARI wrote: > >> Occasionally, it happens that the dovecot.log shows this line: >> master: Warning: service(imap): process_limit reached, client >> connections >> are being dropped > .. >> Then, i edited the file /etc/dovecot/conf.d/10-master.conf and set the >> line >> process_limit = 1500 > > But did you set it inside service imap {}? All of the services have > process_limit parameter. > > From CMarcus at Media-Brokers.com Thu Oct 18 14:22:35 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 18 Oct 2012 07:22:35 -0400 Subject: [Dovecot] lmtp proxy logging In-Reply-To: References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> Message-ID: <507FE67B.4030705@Media-Brokers.com> On 2012-10-18 12:07 AM, Timo Sirainen wrote: > I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) Hey Timo, I hope this means what it sounds like it means... Can you confirm that this 'submission server' would support the ability to automatically add a copy of all emails sent using it to the designated 'Sent' folder, so that email clients could simply disable the 'Save a copy to Sent folder' feature (that causes the client to upload the message to the server twice, once to send the message, and again to save the Sent copy)? This is one feature of gmail that I simply love... Thanks as always, -- Best regards, Charles From amateo at um.es Thu Oct 18 14:23:47 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 18 Oct 2012 13:23:47 +0200 Subject: [Dovecot] CAS Authentication In-Reply-To: <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1350317302.27204.YahooMailNeo@web125701.mail.ne1.yahoo.com> <1350411157.29084.YahooMailNeo@web125705.mail.ne1.yahoo.com> <507E5D3A.5030900@um.es> <1350549205.48116.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <507FE6C3.80702@um.es> El 18/10/12 10:33, b m escribi?: > Thanks for the configuration files. I have a question. In pam_cas.conf I don't know what to put in "proxy ". In some examples I have seen something like http:///proxy.php > Do I need a php file in my webmail to handle the cas tickets and if so where can I find it? > You need this script at your webmail server. This script depend on the webmail you are using. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From tss at iki.fi Thu Oct 18 14:30:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 18 Oct 2012 14:30:00 +0300 Subject: [Dovecot] lmtp proxy logging In-Reply-To: <507FE67B.4030705@Media-Brokers.com> References: <5075881C.4060905@brightok.net> <5D1B8D4E-58B7-4AF0-8346-C6E82A75655A@iki.fi> <507C5EDB.7050401@brightok.net> <507C6DD3.2000309@brightok.net> <507F7C1A.9030301@brightok.net> <507FE67B.4030705@Media-Brokers.com> Message-ID: <141971D9-6DC7-4BEF-B4AE-B1EBD5314499@iki.fi> On 18.10.2012, at 14.22, Charles Marcus wrote: > On 2012-10-18 12:07 AM, Timo Sirainen wrote: >> I'm planning to implement SMTP submission server and it should share the code with LMTP. (Also I've already written a completely separate tiny SMTP server implementation, which should be merged with both of those. So I guess it needs to become a bit more generic lib-smtp-server.) > > Hey Timo, > > I hope this means what it sounds like it means... > > Can you confirm that this 'submission server' would support the ability to automatically add a copy of all emails sent using it to the designated 'Sent' folder, so that email clients could simply disable the 'Save a copy to Sent folder' feature (that causes the client to upload the message to the server twice, once to send the message, and again to save the Sent copy)? That's not the intended reason for creating it, but easy enough to add as an option, assuming \Sent SPECIAL-USE mailbox is defined. Anyway, I don't know when I'll actually start implementing it. Mainly just a "would be nice to have some day" thing to support LEMONADE SMTP extensions. From dg at dguhl.org Thu Oct 18 15:34:41 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 18 Oct 2012 14:34:41 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <1350496658.27003.6.camel@heisenberg.scientia.net> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> Message-ID: <20121018123440.GA29330@laptop-dg.leere.eu> On Wed, Oct 17, 2012 at 07:57:38PM +0200, Christoph Anton Mitterer wrote: > On Wed, 2012-10-17 at 16:51 +0200, Dennis Guhl wrote: [move through Evolution to IMAP] > Well as I've mentioned... on looses the info in the From_ lines (that is > the RCPT TO address and the date of arrival) because Evolution does not The date and time of arrival can be concluded from the last Received: header. The RCPT TO need to be converted to a X-Original-To: header. [..] > > If they bug you remove them with sed or awk or perl or python or ... > Yeah... but sed alone is not enough... cause such lines may also appear > in the body... and I mustn't remove them... > So in principle I'm looking for a smart parser of mbox which already > gives me headers and body and I can modify either. I think, like Rob suggested, you are in need of some serious scripting. Dennis From dg at dguhl.org Thu Oct 18 17:24:02 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 18 Oct 2012 16:24:02 +0200 Subject: [Dovecot] dovecot-core, dovecot-mysql for Debian squeeze In-Reply-To: References: Message-ID: <20121018142400.GA1261@PC211.ikt.de> On Wed, Oct 17, 2012 at 08:47:09PM -0300, Ricardo wrote: > Hello list [..] > I have problems installing the daemon dovecot-core, dovecot-mysql > dovecot-imapd dovecot-pop3d [..] > to install dovecot-core, dovecot-mysql, install it without problems > is the version (2.1.7-2 ~ bpo60 +1) of both packages, now wanting to > install dovecot-imapd dovecot-pop3d (version 1.2.15-7) breaks the > dovecot-core, dovecot-mysql, apparently must be the same version all > packages. Yea, of course you must use the same version for all packages. Upstream there is only one package for dovecot. It's part of Debian's philosophy to split monolithic packages into a bunch separate packages. > Debian Wheezy, installs without problems but installs the version > (dovecot-core_2.1.7-2 ~ ppa12.04 +1 _i386.deb) all packets are the > same version. This no Debian version schema but from Ubuntu 12.04. I don't know if they work correct on Debian. > What is the correct version for Debian squeeze? Stock Squeeze ships Dovecot in Debian version 1.2.15-7. Squeeze Backports offers version 2.1.7-2~bpo60+1. To install Dovecot 2.1 for Debian Squeeze: % sudo apt-get update % apt-get -s -t squeeze-backports install dovecot-imapd dovecot-pop3d dovecot-mysql The '-s' switch simulates the installation and works without root privileges. If you get no error and apt shows to install version 2.1.7-2~bpo60+1 repeat the command with a preceding 'sudo ' and no '-s'. Dennis From alessio at skye.it Thu Oct 18 17:29:50 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 18 Oct 2012 16:29:50 +0200 Subject: [Dovecot] Add S= to maildirfile Message-ID: <5080125E.5020904@skye.it> Hi, in some old Maildir/ I have file without the S= in file name. Is possibile to add the size to the file name with some tools like doveadm? Are there other methods to update these file? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From rs at sys4.de Thu Oct 18 19:22:39 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 18 Oct 2012 18:22:39 +0200 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <5080125E.5020904@skye.it> References: <5080125E.5020904@skye.it> Message-ID: <50802CCF.3000200@sys4.de> Am 18.10.2012 16:29, schrieb Alessio Cecchi: > Hi, > > in some old Maildir/ I have file without the S= in file name. > > Is possibile to add the size to the file name with some tools like doveadm? > > Are there other methods to update these file? > > Thanks > perhaps this helps for ideas http://wiki2.dovecot.org/HowTo/RefilterMail perhaps you can use dsync also , but i am really not sure if this works http://wiki2.dovecot.org/Tools/Dsync however its easy to test -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From jbates at brightok.net Thu Oct 18 19:49:05 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 18 Oct 2012 11:49:05 -0500 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50802CCF.3000200@sys4.de> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> Message-ID: <50803301.4060508@brightok.net> On 10/18/2012 11:22 AM, Robert Schetterer wrote: > Am 18.10.2012 16:29, schrieb Alessio Cecchi: >> Hi, >> >> in some old Maildir/ I have file without the S= in file name. >> >> Is possibile to add the size to the file name with some tools like doveadm? >> >> Are there other methods to update these file? >> >> Thanks >> > perhaps this helps for ideas > > http://wiki2.dovecot.org/HowTo/RefilterMail > > perhaps you can use dsync also , but i am really not sure > if this works > > http://wiki2.dovecot.org/Tools/Dsync > > however its easy to test Dsync would be the best option, I believe. It should work moving from maildir to maildir, but if necessary, you could also convert it to another format and then put it back to maildir. Jack From nanovox at gmail.com Thu Oct 18 23:32:15 2012 From: nanovox at gmail.com (Steven Kiehl) Date: Thu, 18 Oct 2012 16:32:15 -0400 Subject: [Dovecot] Emails from invalid local accounts Message-ID: Hi, I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues with receiving spam where the "from" header contains an address like accounting at mydomain.com. Is there some way I can filter out these emails coming from outside our network with an account associated with our network which doesn't exist? Do I just need to configure some custom process to evaluate these addresses, or is there some way either in dovecot or spamassassin to do this? Thanks, Steve K From noeldude at gmail.com Fri Oct 19 00:00:21 2012 From: noeldude at gmail.com (Noel) Date: Thu, 18 Oct 2012 16:00:21 -0500 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: Message-ID: <50806DE5.6050904@gmail.com> On 10/18/2012 3:32 PM, Steven Kiehl wrote: > Hi, > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > with receiving spam where the "from" header contains an address like > accounting at mydomain.com. Is there some way I can filter out these emails > coming from outside our network with an account associated with our network > which doesn't exist? Do I just need to configure some custom process to > evaluate these addresses, or is there some way either in dovecot or > spamassassin to do this? > > Thanks, > > Steve K > This should be dealt with in postfix or SpamAssassin, not dovecot, and there are likely other, better ways to detect this particular spam rather than mucking with the From: header. Sometimes mail arrives with a header something like From: accounting and postfix appends @$myorigin to the unqualified address while passing the mail through your content_filter. The fix for that is to set in your postfix main.cf remote_header_rewrite_domain = domain.invalid so that unqualified addresses will be rewritten with a known domain. Don't be tempted to reject such mail outright since you'll reject a significant amount of non-spam mail. Another thing to consider setting in postfix main.cf is: smtpd_reject_unlisted_sender = yes which will reject invalid envelope senders in your domain. (Note the difference between envelope sender and the From: header.) -- Noel Jones From ben at morrow.me.uk Fri Oct 19 00:00:27 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 18 Oct 2012 22:00:27 +0100 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: Message-ID: <20121018210026.GB5388@anubis.morrow.me.uk> At 4PM -0400 on 18/10/12 you (Steven Kiehl) wrote: > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > with receiving spam where the "from" header contains an address like > accounting at mydomain.com. Is there some way I can filter out these emails > coming from outside our network with an account associated with our network > which doesn't exist? Do I just need to configure some custom process to > evaluate these addresses, or is there some way either in dovecot or > spamassassin to do this? You want to do this in Postfix, with either the smtpd_reject_unlisted_sender parameter or the reject_unlisted_sender policy in smtpd_sender_restrictions. You will need to make sure Postfix has access to the list of valid mailboxes at your domain, which it should have already for recipient checking. Ben From stephan at rename-it.nl Fri Oct 19 02:01:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 19 Oct 2012 01:01:43 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <507BBE00.9010007@dada.eu> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> <507BBE00.9010007@dada.eu> Message-ID: <50808A57.8040201@rename-it.nl> On 10/15/2012 9:40 AM, Sandro Tosi wrote: > Hi Stephan, > thanks a lot for your reply. > > On 10/11/2012 10:35 PM, Stephan Bosch wrote: >> On 10/10/2012 11:23 AM, Sandro Tosi wrote: >>> Hello, >>> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >>> as this will be provided with an upcoming Debian package) with MySQL >>> dict lookups with the mail setup we're designing. >>> >>> Our (main) goals are: >>> >>> 1. store the filters on the database >> That is possible with some limitations. > > Are the ones below the only limitatios (ie one script per user) or are > there any other worth knowing? You cannot currently use ManageSieve when the active script is located in a dict database. And 'one script per user' is not an fully accurate description. It is technically possible to access multiple different scripts from the dict database. It is however not possible to use dict support combination with multiscript support ( http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Executing_Multiple_Scripts_Sequentially) to execute multiple scripts in a sequence. Multiscript currently only works for Sieve scripts that are located in the filesystem. > In our situation, what would you suggest? We're now thinking of > keeping the scripts list on a separate table, and merge the "user > selected ones" in a single script to write in the filters table. Is > that what would you suggest? Is there a better solution? You can use the include extension (https://tools.ietf.org/html/draft-ietf-sieve-include-05) to access scripts in a dict database from a main active script to combine them. I believe you could even dynamically construct that main script in SQL using some string manipulation in the query, but that is a bit ugly. Could you send me an overview of your configuration, including your database layout? Provided that I have some time in the next week, I could investigate building a simple working configuration for the sake of example. Regards, Stephan. From calestyo at scientia.net Fri Oct 19 02:32:59 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Fri, 19 Oct 2012 01:32:59 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1350603179.3391.21.camel@fermat.scientia.net> Hi Rick and Robert. Thanks for the tools... I'll have a look over them. :) On Wed, 2012-10-17 at 15:53 +0000, Rick Sanders wrote: > Your best bet for a clean migration is to use an IMAP migration tool (assuming > both of your servers support IMAP). It avoids all of the issues surrounding the > underlying databases used to store the mailboxes and messages since everything > is done through IMAP commands. Well the problem is that a) the mboxes are already mixed up (with respect to different formats), which was basically my fault. b) Evolution is severely broken, amongst others for this https://bugzilla.gnome.org/show_bug.cgi?id=686258 reason. So I cannot really trust that automatic migration will work. > imapsync: http://imapsync.lamiral.info > imap_tools: http://www.athensfbc.com/imap_tools > offlineimap: https://github.com/nicolas33/offlineimap > mbsync: http://isync.sourceforge.net/ > mailsync: http://mailsync.sourceforge.net/ > mailutil: http://www.washington.edu/imap/ part of the UW IMAP tookit. > imaprepl: http://www.bl0rg.net/software/ http://freecode.com/projects/imap-repl/ > imapcopy: http://home.arcor.de/armin.diehl/imapcopy/imapcopy.html > migrationtool: http://sourceforge.net/projects/migrationtool/ > imapmigrate: http://sourceforge.net/projects/cyrus-utils/ > larch: https://github.com/rgrove/larch (derived from wonko_imapsync) > wonko_imapsync: http://wonko.com/article/554 > pop2imap: http://www.linux-france.org/prj/pop2imap/ > exchange-away: http://exchange-away.sourceforge.net/ For most of them, I unfortunately didn't found information on whether they support the different subformats of mbox... what about your MboxtoIMAP.pl ? Right now I tent to create my own converter based on mb2md... just that I don't write out maildir but again mbox. Timo, when you're reading this: I'm not sure though, on which headers I must/should stripe for dovecot? From http://wiki.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata I'd guess that I have to drop all X-IMAPbase, X-IMAP and X-UID. (Will dovcote recreate them, when it indexes the mbox file the first time?) And I have to manually create/calculate, Status, X-Status, X-Keyword (based on what either Evolution or Thunderbird set) and also Content-Length... the "From_" lines in the mails need then to be _not_ quoted. Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Fri Oct 19 02:38:38 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Fri, 19 Oct 2012 01:38:38 +0200 Subject: [Dovecot] how to best import Evolution/Thunderbird mail into dovecot? In-Reply-To: <20121018123440.GA29330@laptop-dg.leere.eu> References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> <1350496658.27003.6.camel@heisenberg.scientia.net> <20121018123440.GA29330@laptop-dg.leere.eu> Message-ID: <1350603518.3391.23.camel@fermat.scientia.net> On Thu, 2012-10-18 at 14:34 +0200, Dennis Guhl wrote: > [move through Evolution to IMAP] Seriously... I can just suggest anyone to never trust this piece of crap ;) Don't know which daemons led me to using it... > I think, like Rob suggested, you are in need of some serious > scripting. Yeah... guess that's what it will end up with. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From nanovox at gmail.com Fri Oct 19 04:59:40 2012 From: nanovox at gmail.com (Steven Kiehl) Date: Thu, 18 Oct 2012 21:59:40 -0400 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: <50806DE5.6050904@gmail.com> References: <50806DE5.6050904@gmail.com> Message-ID: This is great information on some options I should look into further, however adding the "smtpd_reject_unlisted_sender" option doesn't seem to eliminate the problem. What these spammers are doing is forging the "from" header to be a full address like "accounting at mydomain.com" and they are sending to a real address like "webmaster at mydomain.com". So even if the envelope sender is valid or coming from an outside domain, the visible originating from address is invalid and is in my own domain. And I'm absolutely positive any mail received from these forged from addresses are spam that shouldn't even be delivered. This is also complicated further by the use of virtual domains and virtual alias mapping (all sql based) in the Postfix configuration. Some of my problem may be that Postfix might not be able to get a comprehensive list of valid mailboxes and aliases to deliver to the virtual transport. I've tried to define the virtual mailbox maps, but every time I do that the aliases stop working. On Thu, Oct 18, 2012 at 5:00 PM, Noel wrote: > On 10/18/2012 3:32 PM, Steven Kiehl wrote: > > Hi, > > > > I'm using dovecot 1.2.9 in a postfix/dovecot setup and I'm having issues > > with receiving spam where the "from" header contains an address like > > accounting at mydomain.com. Is there some way I can filter out these > emails > > coming from outside our network with an account associated with our > network > > which doesn't exist? Do I just need to configure some custom process to > > evaluate these addresses, or is there some way either in dovecot or > > spamassassin to do this? > > > > Thanks, > > > > Steve K > > > > This should be dealt with in postfix or SpamAssassin, not dovecot, > and there are likely other, better ways to detect this particular > spam rather than mucking with the From: header. > > Sometimes mail arrives with a header something like > From: accounting > and postfix appends @$myorigin to the unqualified address while > passing the mail through your content_filter. > > The fix for that is to set in your postfix main.cf > remote_header_rewrite_domain = domain.invalid > so that unqualified addresses will be rewritten with a known > domain. Don't be tempted to reject such mail outright since you'll > reject a significant amount of non-spam mail. > > Another thing to consider setting in postfix main.cf is: > smtpd_reject_unlisted_sender = yes > which will reject invalid envelope senders in your domain. (Note > the difference between envelope sender and the From: header.) > > > > -- Noel Jones > From noeldude at gmail.com Fri Oct 19 06:50:30 2012 From: noeldude at gmail.com (Noel) Date: Thu, 18 Oct 2012 22:50:30 -0500 Subject: [Dovecot] Emails from invalid local accounts In-Reply-To: References: <50806DE5.6050904@gmail.com> Message-ID: <5080CE06.1080706@gmail.com> On 10/18/2012 8:59 PM, Steven Kiehl wrote: > This is great information on some options I should look into > further, however adding the "smtpd_reject_unlisted_sender" > option doesn't seem to eliminate the problem. [This is OT for the dovecot list, and my last post in this thread. Please send all followups to the appropriate postfix, amavisd-new, or spamassassin list in consideration of other list members. Thank you.] smtpd_reject_unlisted_sender works with the envelope address; this option has no effect on headers. > What these spammers are doing is forging the "from" header to be > a full address like "accounting at mydomain.com > " Possible, but I doubt it. The only way you'll ever see the more likely original "From: accounting" header is by running postfix in debug mode (which is not recommended) or by using a tcp sniffer in front of postfix. That's why I recommend setting "remote_header_rewrite_domain = domain.invalid". Also, this setting requires a non-ancient postfix, but I don't remember which version; if it shows up in "postconf -n" output, you're OK. > and they are sending to a real address like > "webmaster at mydomain.com ". So even > if the envelope sender is valid or coming from an outside domain, > the visible originating from address is invalid and is in my own > domain. And I'm absolutely positive any mail received from these > forged from addresses are spam that shouldn't even be delivered. If there are a few frequently-abused addresses, you can add them to a header_checks rule. But don't get too tied up in wack-a-mole header_checks; that's a great time waster for limited benefit. > This is also complicated further by the use of virtual domains and > virtual alias mapping (all sql based) in the Postfix > configuration. Some of my problem may be that Postfix might not > be able to get a comprehensive list of valid mailboxes and aliases > to deliver to the virtual transport. I've tried to define the > virtual mailbox maps, but every time I do that the aliases stop > working. If your postfix is not able to properly validate recipients, you should ask about that on the postfix list. That is a serious problem. http://www.postfix.org/DEBUG_README.html#mail The point you're missing is that there is no way to validate the From: header. Look at other features of the unwanted mail for ways to reject it. -- Noel Jones From tomislav.mihalicek at gmail.com Fri Oct 19 10:40:50 2012 From: tomislav.mihalicek at gmail.com (tmihalicek) Date: Fri, 19 Oct 2012 00:40:50 -0700 (PDT) Subject: [Dovecot] Dovecot quota postgres dictionary problems Message-ID: <1350632450161-38234.post@n4.nabble.com> I have a strange errors in .err log file, but the postgres seem to be filling with quota changes, i will also put configs in Oct 19 09:23:52 mailstore-node-01 dovecot: imap(test at example.net): Error: read(/var/run/dovecot/dict) failed: Timeout after 30 seconds Oct 19 09:24:22 mailstore-node-01 dovecot: imap(test at example.net): Error: read(/var/run/dovecot/dict) failed: Timeout after 30 seconds Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Panic: file dict-client.c: line 270 (client_dict_finish_transaction): assertion failed: (dict->async_commits > 0) Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x484ea) [0x7fbed405d4ea] -> /usr/lib/dovecot/libdovecot.so.0(+0x48536) [0x7fbed405d536] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fbed4030eaf] -> /usr/lib/dovecot/libdovecot.so.0(+0x22337) [0x7fbed4037337] -> /usr/lib/dovecot/libdovecot.so.0(+0x2236b) [0x7fbed403736b] -> /usr/lib/dovecot/libdovecot.so.0(+0x22e78) [0x7fbed4037e78] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0x8a3f) [0x7fbed2c76a3f] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7fbed2c73262] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(quota_transaction_commit+0x1e7) [0x7fbed2c738d7] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xb39f) [0x7fbed2c7939f] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xb4f4) [0x7fbed2c794f4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x2a) [0x7fbed432396a] -> dovecot/imap [test at example.net 10.84.34.2 expunge](imap_sync_deinit+0x4d) [0x418edd] -> dovecot/imap [test at example.net 10.84.34.2 expunge]() [0x41918c] -> dovecot/imap [test at example.net 10.84.34.2 expunge](cmd_sync_delayed+0x1f5) [0x4195b5] -> dovecot/imap [test at example.net 10.84.34.2 expunge](client_handle_input+0x1fd) [0x41127d] -> dovecot/imap [test at example.net 10.84.34.2 expunge](client_input+0x5f) [0x411adf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7fbed40696c6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7fbed406a6ff] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7fbed4069668] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fbed4055043] -> dovecot/imap [test at example.net 10.84.34.2 expunge](main+0x2a4) [0x419d24] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fbed3cd1c8d] -> dovecot/imap [test at example.net 10.84.34.2 expunge]() [0x409059] doveconf.txt dovecot-dict-sql.conf.ext -- View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-quota-postgres-dictionary-problems-tp38234.html Sent from the Dovecot mailing list archive at Nabble.com. From amateo at um.es Fri Oct 19 15:38:36 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 19 Oct 2012 14:38:36 +0200 Subject: [Dovecot] Auth caching and password changes Message-ID: <508149CC.9070004@um.es> Hello, In my system I have configured auth caching. The problem I have is that whenever a user changes his password, he/she can't login to dovecot after a while and the scenarios described at http://wiki2.dovecot.org/Authentication/Caching are not applied. I have tried also with "doveadm auth cache flush ", but it didn't work. He also could to login again if he waits for a time or if I run "doveadm auth cache flush" in the server, flushing all auth information from cache. I have attached the log I had when I changed my password (and suffered the problem). I have attached my doveconf -n too. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 -------------- next part -------------- A non-text attachment was scrubbed... Name: cambioclave.log Type: text/x-log Size: 1349 bytes Desc: not available URL: -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota zlib mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes cache_key=%n dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 3075 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From alessio at skye.it Fri Oct 19 19:12:26 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 19 Oct 2012 18:12:26 +0200 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50803301.4060508@brightok.net> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> <50803301.4060508@brightok.net> Message-ID: <50817BEA.6090201@skye.it> Il 18/10/2012 18:49, Jack Bates ha scritto: > On 10/18/2012 11:22 AM, Robert Schetterer wrote: >> Am 18.10.2012 16:29, schrieb Alessio Cecchi: >>> Hi, >>> >>> in some old Maildir/ I have file without the S= in file name. >>> >>> Is possibile to add the size to the file name with some tools like >>> doveadm? >>> >>> Are there other methods to update these file? >>> >>> Thanks >>> >> perhaps this helps for ideas >> >> http://wiki2.dovecot.org/HowTo/RefilterMail >> >> perhaps you can use dsync also , but i am really not sure >> if this works >> >> http://wiki2.dovecot.org/Tools/Dsync >> >> however its easy to test > > Dsync would be the best option, I believe. It should work moving from > maildir to maildir, but if necessary, you could also convert it to > another format and then put it back to maildir. > > Jack > Dsync could be a good idea but I need to "sync" maildir to the same maildir and I don't think can be done this. My situation is a Maildir/cur/ with old email files without S=size and, in the same Maildir/cur/, some files with S=size. Can doveadm do this? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From wamp at promax.media.pl Fri Oct 19 15:32:20 2012 From: wamp at promax.media.pl (wamp) Date: Fri, 19 Oct 2012 05:32:20 -0700 (PDT) Subject: [Dovecot] Dovecot 2 quota limit and actual size (mysql) Message-ID: <1350649940026-38235.post@n4.nabble.com> Hello, Can You explain to me how dovecot-lda knows actual size of virtual user directory? I want to keep max size of user directory in mysql - should I also use some kind of script to upgrade actual size information in mysql ? I read docs from wiki but still dont know it. thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-2-quota-limit-and-actual-size-mysql-tp38235.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Fri Oct 19 19:17:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 19 Oct 2012 19:17:04 +0300 Subject: [Dovecot] Add S= to maildirfile In-Reply-To: <50817BEA.6090201@skye.it> References: <5080125E.5020904@skye.it> <50802CCF.3000200@sys4.de> <50803301.4060508@brightok.net> <50817BEA.6090201@skye.it> Message-ID: <9BE333EF-2120-4581-9A51-79C08EAF9085@iki.fi> On 19.10.2012, at 19.12, Alessio Cecchi wrote: >>>> in some old Maildir/ I have file without the S= in file name. >>>> >>>> Is possibile to add the size to the file name with some tools like doveadm? Not directly. >>>> Are there other methods to update these file? A script that renames the files and updates dovecot-uidlist. No such script exists as far as I know. You could also switch from Maildir++ quota to dict-file quota and this wouldn't be a problem. > Dsync could be a good idea but I need to "sync" maildir to the same maildir and I don't think can be done this. You could sync to another maildir, rm -rf the original, sync back to original. > My situation is a Maildir/cur/ with old email files without S=size and, in the same Maildir/cur/, some files with S=size. > > Can doveadm do this? No. From tobias at maffert.net Fri Oct 19 21:47:20 2012 From: tobias at maffert.net (Tobias Maffert) Date: Fri, 19 Oct 2012 20:47:20 +0200 Subject: [Dovecot] Question about salted hashes Message-ID: <5081A038.8070908@maffert.net> Hello. I'm switching from b1gmail to my own setup which consists of Postfix+Dovecot+MySQL (and maybe VBoxAdm). There are two problems: - b1gmail is using unsalted MD5 hashes. Is there any good way to make my new setup backward compatible? So I don't have to force all of my 50k users to change their password. - How do I change my setup to salted SHA256 (or an even better algorithm). And how do I make the hashes compatible between Postfix, Dovecot, MySQL (and maybe VBoxAdm)? My system: Debian Squeeze Postfix version: 2.7.1 Dovecot version: 1.2.15 ----------------------------------------------- dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.6 protocols: imap pop3 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_access_groups: vmail mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): UID%u-%v lda: postmaster_address: auth_socket_path: /var/run/dovecot/auth-master mail_plugins: quota sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain login user: vmail passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir ----------------------------------------------- driver = mysql connect = host=127.0.0.1 dbname=smail user=smail password=mypw default_pass_scheme = CRYPT password_query = SELECT username AS user, password_enc AS password, CONCAT(homedir, maildir) AS userdb_home, uid AS userdb_uid, gid AS userdb_gid, CONCAT('maildir:', homedir, maildir) AS userdb_mail, CONCAT('maildir:storage=', (quota*1024)) as userdb_quota FROM mail_users WHERE (username = '%u' OR email = '%u') AND ((imap = 1 AND '%Ls' = 'imap') OR (pop3 = 1 AND '%Ls' = 'pop3') OR '%Ls' = 'smtp') user_query = SELECT CONCAT(homedir, maildir) AS home, CONCAT('maildir:', homedir, maildir) AS mail, uid, gid, CONCAT('*:storage=', (quota*1024)) as quota_rule FROM mail_users WHERE (username = '%u' OR email = '%u') ----------------------------------------------- Regards, Tobi M. From h.reindl at thelounge.net Fri Oct 19 21:50:28 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 19 Oct 2012 20:50:28 +0200 Subject: [Dovecot] Question about salted hashes In-Reply-To: <5081A038.8070908@maffert.net> References: <5081A038.8070908@maffert.net> Message-ID: <5081A0F4.9090704@thelounge.net> Am 19.10.2012 20:47, schrieb Tobias Maffert: > Hello. > > I'm switching from b1gmail to my own setup which consists of > Postfix+Dovecot+MySQL (and maybe VBoxAdm). There are two problems: - > b1gmail is using unsalted MD5 hashes. Is there any good way to make my > new setup backward compatible? So I don't have to force all of my 50k > users to change their password. - How do I change my setup to salted > SHA256 (or an even better algorithm). And how do I make the hashes > compatible between Postfix, Dovecot, MySQL (and maybe VBoxAdm)? > > My system: > > Debian Squeeze > Postfix version: 2.7.1 > Dovecot version: 1.2.15 besides your question you REALLY do not want to START with 1.2.15 while dovecot-2.1.10 is the recent version postfix is somehow OK but recent is postfix-2.9.4 these outdated versions usually results in many questions on mailing-lists because several howtos do not work or bugs fixed since years are still there -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From dave.mehler at gmail.com Fri Oct 19 23:11:07 2012 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 19 Oct 2012 16:11:07 -0400 Subject: [Dovecot] still having difficulties with per-user quotas Message-ID: Hello, I am trying to get per-user quotas working. My thanks to all who have helped so far. To recap I am running Dovecot 2.1 and Mysql where I've got my virtual users. All virtual users are under the system user vmail with a UID and GID of 5000. Looking over the wiki docs I've added a quota table and got the dict service working, I am not having problems with permissions or the login username and password, all that is working fine. Here's my current doveconf -n output it is producing the following debug error related to the userdb sql query: # 2.1.10: /etc/dovecot/dovecot.conf # XXX dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 hostname = XXX last_valid_gid = 5000 last_valid_uid = 5000 mail_location = maildir:/home/vmail/%d/%n:LAYOUT=fs mail_plugins = " quota" namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = xxx xxxx ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_cert = , method=PLAIN, rip=::1, lip=::1, mpid=29282, secured, session= Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User is missing UID (see mail_uid setting) Oct 19 15:23:52 imap(xxx): Error: Internal error occurred. Refer to server log for more information. I am wanting a majority of my users to have the global 1GB quota, but the users in the quota table to have given quotas. Here's what the virtual_users and quota tables look like: mysql> describe virtual_users; +-----------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | domain_id | int(11) | NO | MUL | NULL | | | user | varchar(40) | NO | | NULL | | | password | varchar(128) | NO | | NULL | | +-----------+--------------+------+-----+---------+----------------+ 4 rows in set (0.00 sec) mysql> describe quota; +----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ | username | varchar(100) | NO | PRI | NULL | | | bytes | bigint(20) | NO | | 0 | | | messages | int(11) | NO | | 0 | | +----------+--------------+------+-----+---------+-------+ 3 rows in set (0.00 sec) I'd appreciate any help. Thanks. Dave. From emailbuilder88 at yahoo.com Fri Oct 19 23:43:29 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Fri, 19 Oct 2012 13:43:29 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? Message-ID: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Hello, I'm having some problems getting LDA to work without userdb lookups and have a few related questions. This system has all users in MySQL, each user with unique UID/GID, no local users at all.? Installation is from apt-get. 1) If LDA is invoked without lookups, is it correct to assume that the "service auth" and "service auth-worker" can be completely removed from dovecot master configuration? (I have tried commenting them out and logging into IMAP, which seems to work, not sure if anyone else needs the auth service) 2) If LDA is invoked without lookups, will I be unable to use Dovecot quota plugin? Does it need to have a user lookup to get quota info? (haven't added quota support, need to take this one step at a time) 3) The interesting part -- I am invoking LDA from Maildrop. See: http://thread.gmane.org/gmane.mail.imap.dovecot/65473 So when invoked, Maildrop has already dropped to the destination UID/GID and the needed paths are available in the environment.? However, using as many permutations of calling LDA as I can think of (based on ??? http://wiki2.dovecot.org/LDA ), I always get this: (command line usage error. Command output: lda: Fatal: Couldn't lookup our username (uid=2500) ) The UID is correct for the target user. If I add "-d $LOGNAME" to my LDA callout, I get permission denied on the userdb lookup, which I guess is another issue to work out if I want to go with lookups. But right now I am trying not to. Why does LDA seem to try for a lookup even when I follow the wiki instructions how to call it without a lookup? 3.5) Related question, my users have separate homedir and maildir, both paths are looked up by Maildrop. I think I need to call LDA with "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 14:45:20 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 13:45:20 +0200 Subject: [Dovecot] Dovecot 2 quota limit and actual size (mysql) In-Reply-To: <1350649940026-38235.post@n4.nabble.com> References: <1350649940026-38235.post@n4.nabble.com> Message-ID: <20121020114520.GA26196@daniel.localdomain> Use LMTP instead of lda. The dovecot lmtp service automatically cares about updating quota values in mysql database when mail arrives through the lmtp socket. Regards Daniel wamp wrote: > Hello, > Can You explain to me how dovecot-lda knows actual size of virtual user > directory? I want to keep > max size of user directory in mysql - should I also use some kind of script > to upgrade actual size information in mysql ? > > I read docs from wiki but still dont know it. > > > thanks > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/Dovecot-2-quota-limit-and-actual-size-mysql-tp38235.html > Sent from the Dovecot mailing list archive at Nabble.com. > -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 15:47:49 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 14:47:49 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: Message-ID: <20121020124749.GA26942@daniel.localdomain> David Mehler wrote: > Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User is missing UID (see mail_uid setting) Set the following options in your dovecot.conf: mail_uid = vmail mail_gid = vmail Also see section "Mail users" at http://wiki2.dovecot.org/UserIds Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 16:51:44 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 15:51:44 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: References: Message-ID: <20121020135144.GA28609@daniel.localdomain> Jan-Frode Myklebust wrote: > $ cat /etc/dovecot/dovecot-trash.conf.ext > # Spam mailbox is emptied before Trash > 1 INBOX.Spam > # Trash mailbox is emptied before Sent > 2 INBOX.Trash Are you sure the Trash Folder of the affected users is located below "INBOX"? doveadm mailbox list -u user at domain | grep -iE "trash|spam" Example at http://wiki2.dovecot.org/Plugins/Trash omits "INBOX." Have you tried INBOX/Trash as mailbox name? Regards Daniel -- https://plus.google.com/103021802792276734820 From sven at svenhartge.de Sat Oct 20 19:39:22 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sat, 20 Oct 2012 18:39:22 +0200 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive Message-ID: <09718hdveev8@mids.svenhartge.de> Hi! I am about to migrate a perdition-based IMAP/POP3 proxy to Dovecot. Unfortunately some users are behind a firewall/NAT setup which throws away seemingly idle TCP connections sooner than the established default of 24 hours (more likely after 30 minutes ...) resulting in all kinds of weird client behavior. And unfortunately? this firewall/NAT setup is outside of my control and I have no means of correcting this (in my opinion) flawed configuration. Now, with perdition I was able to use the --tcp_keepalive option which totally solved the mentioned weird client behavior. My question is: does Dovecot2 use TCP-Keepalive on its sockets per default or do I need to enable it some way I have not yet discovered? The manual and wiki only talk about "keepalive" in connection with the IMAP protocol and IDLE and my C-fu is too weak to understand the source code. Gr??e, Sven. -- Sigmentation fault. Core dumped. From tss at iki.fi Sat Oct 20 20:02:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 20 Oct 2012 20:02:36 +0300 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive In-Reply-To: <09718hdveev8@mids.svenhartge.de> References: <09718hdveev8@mids.svenhartge.de> Message-ID: <9881D0A3-70EB-454B-A71D-9C7A253AE21E@iki.fi> On 20.10.2012, at 19.39, Sven Hartge wrote: > I am about to migrate a perdition-based IMAP/POP3 proxy to Dovecot. > > Unfortunately some users are behind a firewall/NAT setup which throws > away seemingly idle TCP connections sooner than the established default > of 24 hours (more likely after 30 minutes ...) resulting in all kinds of > weird client behavior. > > And unfortunately? this firewall/NAT setup is outside of my control and > I have no means of correcting this (in my opinion) flawed configuration. > > Now, with perdition I was able to use the --tcp_keepalive option which > totally solved the mentioned weird client behavior. > > My question is: does Dovecot2 use TCP-Keepalive on its sockets per > default or do I need to enable it some way I have not yet discovered? It's the default yes. Of course Linux's default keepalive interval is something like 90 minutes, so have you changed that already?.. > The manual and wiki only talk about "keepalive" in connection with the > IMAP protocol and IDLE and my C-fu is too weak to understand the source > code. imap_idle_notify_interval (default 2 min) causes Dovecot to send data to IDLEing connections, which pretty much makes the TCP keepalive irrelevant. For non-IDLE connections Dovecot has a disconnect timeout of 30 minutes. From dave.mehler at gmail.com Sat Oct 20 20:06:59 2012 From: dave.mehler at gmail.com (David Mehler) Date: Sat, 20 Oct 2012 13:06:59 -0400 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: <20121020124749.GA26942@daniel.localdomain> References: <20121020124749.GA26942@daniel.localdomain> Message-ID: Hello, Thank you for your reply. Adding mail_uid and mail_gid fixed it. I now have quotas going but I don't know if I have them right or just don't like my setup. My first issue is from what it is looking like I have to define all my users in the quota database not just the ones whose values I want to override the global quota declaration in 90-quota.conf. If I just add the user at domain to the database the bytes and messages columns have zero as default, this means those values override global quota in 90-quota.conf and they effectively have unlimited access. My second issue is I have entered a quota of 250 megabytes for a test user. This works but he seems to get more space everytime he logs in, started out at 250, on the next login it was 255, then 269 on the third, and so forth. I've checked the quota table and yes the value in the bytes column is increasing. Thanks for any help. Dave. On 10/20/12, Daniel Parthey wrote: > David Mehler wrote: >> Oct 19 15:23:52 imap(xxx): Error: user xxx: Couldn't drop privileges: User >> is missing UID (see mail_uid setting) > > Set the following options in your dovecot.conf: > > mail_uid = vmail > mail_gid = vmail > > Also see section "Mail users" at > http://wiki2.dovecot.org/UserIds > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 > From sven at svenhartge.de Sat Oct 20 20:15:25 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sat, 20 Oct 2012 19:15:25 +0200 Subject: [Dovecot] Dovecot 2 and TCP-Keepalive References: <09718hdveev8@mids.svenhartge.de> <9881D0A3-70EB-454B-A71D-9C7A253AE21E@iki.fi> Message-ID: <1971afdveev8@mids.svenhartge.de> Timo Sirainen wrote: > On 20.10.2012, at 19.39, Sven Hartge wrote: >> My question is: does Dovecot2 use TCP-Keepalive on its sockets per >> default or do I need to enable it some way I have not yet discovered? > It's the default yes. Of course Linux's default keepalive interval is > something like 90 minutes, so have you changed that already?.. Yes, I did. For those systems it is set to 15 minutes right now. >> The manual and wiki only talk about "keepalive" in connection with >> the IMAP protocol and IDLE and my C-fu is too weak to understand the >> source code. > imap_idle_notify_interval (default 2 min) causes Dovecot to send data > to IDLEing connections, which pretty much makes the TCP keepalive > irrelevant. For non-IDLE connections Dovecot has a disconnect timeout > of 30 minutes. This is fine. As long as the client notices the termination of the connection, everything should be OK. Before I switched keepalive on for Perdition, the firewall/NAT would internally throw away a connection but neither the client or the server would notice this. Then if the client tried to do something with this connection, like select or save a message, the firewall/NAT would send a RST and the client would then bug the user with a meaningless message like "folder does not exist" which caused a lot of confusion for the end-user and created quite the bit of trouble tickets. This problem mostly happend with an IMAP connection to the "Sent Messages" folder which normally does not see much changes until the users writes and sends a mail. Then after the mail was sent via SMTP the client tries to save the message, gets sent an RST from the firewall/NAT and presents the user with a wrong and confusing error message. The user then thinks his mail was not sent and sends it again. This time the client opens a new connection to select the "Sent Messages" folder and everything works. But the recipient gets the mail twice. Again resulting in confusion and trouble tickets to be dealt with. By switching to TCP keepalive (and reducing the keepalive time to 15 minutes) all those problems were solved and my users (and support staff) were happy again ;) Gr??e, Sven. -- Sigmentation fault. Core dumped. From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 21:51:24 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 20:51:24 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> Message-ID: <20121020185124.GA2609@daniel.localdomain> David Mehler wrote: > My first issue is from what it is looking like I have to define all my > users in the quota database not just the ones whose values I want to > override the global quota declaration in 90-quota.conf. If I just add > the user at domain to the database the bytes and messages columns have > zero as default, this means those values override global quota in > 90-quota.conf and they effectively have unlimited access. This is expected behavior. If the userdb returns a quota rule, it overrides the global quota rule. Extend your SQL query to return a default quota_rule for rows without quota entry. > My second issue is I have entered a quota of 250 megabytes for a test > user. This works but he seems to get more space everytime he logs in, > started out at 250, on the next login it was 255, then 269 on the > third, and so forth. I've checked the quota table and yes the value in > the bytes column is increasing. Please show output of doveconf -n and any external (sql/dict) includes related to quota or quota_rules. Regards Daniel -- https://plus.google.com/103021802792276734820 From emailbuilder88 at yahoo.com Sat Oct 20 22:17:12 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Sat, 20 Oct 2012 12:17:12 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Message-ID: <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> > 1) If LDA is invoked without > lookups, is it correct to assume that the "service auth" and > "service > auth-worker" can be completely removed from dovecot master > configuration? (I have tried commenting them out and logging into IMAP, > which seems to work, not sure if anyone else needs the auth service) Any confirmation on this? > 2) > If LDA is invoked without lookups, will I be unable to use Dovecot > quota plugin? Does it need to have a user lookup to get quota info? > (haven't added quota support, need to take this one step at a time) I'm especially interested if someone can comment on this, since maybe it makes my efforts here wasted > 3) The interesting part -- I am invoking LDA from Maildrop. See: > http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So > when invoked, Maildrop has already dropped to the destination UID/GID > and the needed paths are available in the environment.? However, using > as many permutations of calling LDA as I can think of (based on ??? > http://wiki2.dovecot.org/LDA ), I always get this: > > (command line usage error. Command output: lda: Fatal: Couldn't lookup our > username (uid=2500) ) I could not find anything in the mailing list archives to help me, but I googled and found a link to a source file: http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 I read the source, it looks like it's not exactly a userdb lookup - LDA is trying to get the unix username for the given UID. In my case, UIDs are "virtual" so there isn't a unix username. The source doesn't really use the username that it looks up except in a call "open_logfile." Is it possible to avoid this problem? It looks like the answer is no, I have to use -d which also forces a userdb lookup. Maybe this limitation can be removed in the future? Now I suppose I have to go understand the problems of userdb lookup permissions, but I think there are solutions for that. Am I on the right understanding?? ? > The > UID is correct for the target user. If I add "-d $LOGNAME" to my LDA > callout, I get permission denied on the userdb lookup, which I guess is > another issue to work out if I want to go with lookups. But right now I > am trying not to. Why does LDA seem to try for a lookup even when I > follow the wiki instructions how to call it without a lookup? > > 3.5) > Related question, my users have separate homedir and maildir, both > paths are looked up by Maildrop. I think I need to call LDA with > "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? > From daniel.parthey at informatik.tu-chemnitz.de Sat Oct 20 23:04:32 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 20 Oct 2012 22:04:32 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> <20121020185124.GA2609@daniel.localdomain> Message-ID: <20121020200432.GA3161@daniel.localdomain> David Mehler wrote: > Thanks for your reply. So with the extending of the query to return a > default quota rule, do you have an example of that by the way, does > that mean I only have to put the overrided users in the quota table? Assuming that quota values are in the dovecot_users table... # passdb with userdb prefetch and default quota of 1024M for quota=0 rows # The userdb_ prefix is for prefetch userdb entries in password_query password_query = SELECT username AS user, \ password AS password, \ home AS userdb_home, \ uid AS userdb_uid, \ gid AS userdb_gid, \ CASE quota \ WHEN 0 \ THEN '*:bytes=1024M:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota AS CHAR), 'M:messages=', CAST(quota_message AS CHAR)) \ END AS `userdb_quota_rule` \ FROM dovecot_users \ WHERE username='%u'; # user_query with default quota of 1024M for quota=0 rows user_query = SELECT username AS user, \ home AS home, \ uid AS uid, \ gid as gid, \ CASE quota \ WHEN 0 \ THEN '*:bytes=1024M:messages=0' \ ELSE \ CONCAT('*:bytes=', CAST(quota AS CHAR), 'M:messages=', CAST(quota_message AS CHAR)) \ END AS `quota_rule` \ FROM dovecot_users \ WHERE username='%u'; Your user_query needs to return a row if the user exists, otherwise dovecot will assume that the user does not exist and the mail or user will be rejected. Regards Daniel -- https://plus.google.com/103021802792276734820 From emailbuilder88 at yahoo.com Sat Oct 20 23:34:12 2012 From: emailbuilder88 at yahoo.com (E.B.) Date: Sat, 20 Oct 2012 13:34:12 -0700 (PDT) Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> <1350760632.64676.YahooMailNeo@web39305.mail.mud.yahoo.com> Message-ID: <1350765252.74118.YahooMailNeo@web39301.mail.mud.yahoo.com> >> 3) The interesting part -- I am invoking LDA from Maildrop. See: >> http://thread.gmane.org/gmane.mail.imap.dovecot/65473 >> So >> when invoked, Maildrop has already dropped to the destination UID/GID >> and the needed paths are available in the environment.? However, using >> as many permutations of calling LDA as I can think of (based on ??? >> http://wiki2.dovecot.org/LDA ), I always get this: >> >> (command line usage error. Command output: lda: Fatal: Couldn't lookup > our >> username (uid=2500) ) > > I could not find anything in the mailing list archives to help me, but I googled > and found a link to a source file: > > http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 > > I read the source, it looks like it's not exactly a userdb lookup - LDA is > trying to get the unix username for the given UID. In my case, UIDs are > "virtual" so there isn't a unix username. The source doesn't > really use the username that it looks up except in a call > "open_logfile." > > Is it possible to avoid this problem? It looks like the answer is no, I have to > use -d which also forces a userdb lookup. Maybe this limitation can be removed > in the future? Now I suppose I have to go understand the problems of userdb > lookup permissions, but I think there are solutions for that. FWIW, in this scenario, "service auth" in master config has to have its mode relaxed to 0606 to make userdb lookups work.? So ANYONE on the machine can see all userdb lookups.? I don't have local users here, so it's probably safe anyway(?). Can anyone explain if there are other security risks of running the auth service at 0606? From jeff at bubble.org Sun Oct 21 04:52:01 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Sat, 20 Oct 2012 21:52:01 -0400 Subject: [Dovecot] Configuring Dovecot & Snarf plugin for the first time Message-ID: <50835541.8000808@bubble.org> I've been using uw-imap for some time on my linux system and have been running into issues with it so I've decided to move to Dovecote, so far it seems to have solved the issues I've been having however I need/want to move the incoming emails out of /var/spool/mail/{user} in the same (or similar fashion) that uw-imap did, and I found the snarf plugin. However whenever I enable the snarf plugin using the example on the wiki page my email is not loaded and when I remove my configuration for snarf my email re-appears. Based upon what I can tell the snarf plugin is either not loading (but I see it listed in the logs) or simply not working (which is probably because its not configured properly). The system is Fedora 16 (x86_64), the MTA is Exim, and Dovecot is 2.0.21 (version supplied with Fedora). I know I'm making a newbie mistake. Any guidance would be appreciated. Thanks, Jeff dovecot -n provides the following: [root at xyzzy conf.d]# more /tmp/dovecot.changes # 2.0.21: /etc/dovecot/dovecot.conf # OS: Linux 3.4.11-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mbox mail_plugins = " zlib" mbox_write_locks = fcntl namespace default { inbox = yes location = prefix = separator = / } namespace snarf { hidden = yes list = no location = mbox:/run/dovecot/empty:INBOX=/var/spool/mail/%u prefix = /snarf separator = / } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename snarf = = /snarf/INBOX } service imap-login { inet_listener imap { address = localhost } } service pop3-login { inet_listener pop3 { address = localhost } } ssl_cert = , method=PLAIN, rip=98.109.156.118, lip=132.238.254.34, mpid=19627, TLS Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff Oct 20 21:29:45 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= Oct 20 21:29:46 xyzzy dovecot: imap-login: Login: user=, method=PLAIN, rip=98.109.156.118, lip=132.238.254.34, mpid=19629, TLS Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff Oct 20 21:29:46 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= From nicolas at devels.es Sun Oct 21 14:15:37 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 12:15:37 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log Message-ID: <5083D959.5090407@devels.es> Hi list! I'm using Dovecot along with Postfix (with MySQL) and I'm having some curious error messages in the mail log. Everything runs normally and suddenly I start viewing messages like these: Oct 12 14:24:23 dovecot: last message repeated 5 times Oct 12 14:25:23 dovecot: last message repeated 6 times Oct 12 14:26:23 dovecot: last message repeated 6 times Oct 12 14:27:23 dovecot: last message repeated 6 times Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory Oct 12 14:30:23 dovecot: last message repeated 5 times Oct 12 14:31:23 dovecot: last message repeated 6 times Oct 12 14:33:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory Oct 12 14:34:23 dovecot: last message repeated 5 times Oct 12 14:35:23 dovecot: last message repeated 6 times Oct 12 14:36:01 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory This file which Dovecot references to indeed exists, is accesible, etc. Once I get the first of these messages, the user stops receiving e-mails (the mail server keeps receiving them normally, though) until he restarts the mail client. Then the error msg is gone and he receives all their unreceived messages. This error appears in 6-12h. intervals once the user starts their client. Version is 2.0.19, and dovecot -n is: root at mail:~# dovecot -n # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-32-generic-pae i686 Ubuntu 12.04.1 LTS ext4 disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir mailbox_idle_check_interval = 2 mins namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = root } ssl_cert = Hi everyone, short version: Is there no built in failover mechanism for the director service to handle a backend failure? Long version: I have a frontend server running the director service and two backends. Due to maintenance I had to shut down one of the backends which caused connection errors for the users being directed to this backend. I was very surprised as I expected the director to redirect these users to the remaining backend. Am I wrong or is the director not working as expected? Regards Patrick # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain login director_mail_servers = 172.17.1.1 172.17.1.2 director_servers = 172.17.1.3 172.17.1.4 lmtp_proxy = yes log_path = /var/log/dovecot.log managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 172.17.1.3 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 172.17.1.3 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = References: <5083D959.5090407@devels.es> Message-ID: On 21.10.2012, at 14.15, Nicol?s wrote: > Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? From nicolas at devels.es Sun Oct 21 14:29:39 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 12:29:39 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: References: <5083D959.5090407@devels.es> Message-ID: <5083DCA3.8000808@devels.es> El 21/10/2012 12:26, Timo Sirainen escribi?: > On 21.10.2012, at 14.15, Nicol?s wrote: > >> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory > fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? > No, on that machine I'm just using local filesystem, no NFS. From tss at iki.fi Sun Oct 21 16:58:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 21 Oct 2012 16:58:36 +0300 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <5083DCA3.8000808@devels.es> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> Message-ID: <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> On 21.10.2012, at 14.29, Nicol?s wrote: > El 21/10/2012 12:26, Timo Sirainen escribi?: >> On 21.10.2012, at 14.15, Nicol?s wrote: >> >>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >> > > No, on that machine I'm just using local filesystem, no NFS. Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. From janfrode at tanso.net Sun Oct 21 19:45:18 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 21 Oct 2012 18:45:18 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: <20121020135144.GA28609@daniel.localdomain> References: <20121020135144.GA28609@daniel.localdomain> Message-ID: On Sat, Oct 20, 2012 at 3:51 PM, Daniel Parthey wrote: > Jan-Frode Myklebust wrote: >> $ cat /etc/dovecot/dovecot-trash.conf.ext >> # Spam mailbox is emptied before Trash >> 1 INBOX.Spam >> # Trash mailbox is emptied before Sent >> 2 INBOX.Trash > > Are you sure the Trash Folder of the affected users is located below "INBOX"? > doveadm mailbox list -u user at domain | grep -iE "trash|spam" $ sudo doveadm mailbox list -u XXXXX at example.no INBOX INBOX.Drafts INBOX.Sent INBOX.Spam INBOX.Trash > Example at http://wiki2.dovecot.org/Plugins/Trash omits "INBOX." > Have you tried INBOX/Trash as mailbox name? No, should I, when my prefix is "INBOX." and separator is "." ? namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } BTW: I think it's mostly working.. as the number or quota exceeded messages has clearly dropped since implementing it, but I do find a few users that get quota exceeded and has lots of messages in INBOX.Trash og INBOX.Spam.. -jf From nicolas at devels.es Sun Oct 21 20:58:46 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Sun, 21 Oct 2012 18:58:46 +0100 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> Message-ID: <508437D6.8020305@devels.es> El 21/10/2012 14:58, Timo Sirainen escribi?: > On 21.10.2012, at 14.29, Nicol?s wrote: > >> El 21/10/2012 12:26, Timo Sirainen escribi?: >>> On 21.10.2012, at 14.15, Nicol?s wrote: >>> >>>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >>> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >>> >> No, on that machine I'm just using local filesystem, no NFS. > Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. > Well, at least it's good to know it's not a misconfiguration :-) If there's any additional info I can provide feel free to ask. Regards, Nicol?s From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 21 22:43:20 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 21 Oct 2012 21:43:20 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <5083D963.3000700@wk-serv.de> References: <5083D963.3000700@wk-serv.de> Message-ID: <20121021194320.GA7977@daniel.localdomain> Patrick Westenberg wrote: > Is there no built in failover mechanism for the director service to > handle a backend failure? No, the director's job is to keep a hash table and direct the connection for each user to its associated backend. Currently, there is no built-in backend monitoring. In order to handle maintenance of backends, you will need the poolmon daemon, which enables/disables backends in the director depending on their availability: https://github.com/brandond/poolmon Regards Daniel -- https://plus.google.com/103021802792276734820 From tss at iki.fi Sun Oct 21 22:45:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 21 Oct 2012 22:45:46 +0300 Subject: [Dovecot] fstat() failed with file */dovecot.index.log In-Reply-To: <508437D6.8020305@devels.es> References: <5083D959.5090407@devels.es> <5083DCA3.8000808@devels.es> <0086379B-D116-4765-8F66-B128FB4A4958@iki.fi> <508437D6.8020305@devels.es> Message-ID: On 21.10.2012, at 20.58, Nicol?s wrote: > El 21/10/2012 14:58, Timo Sirainen escribi?: >> On 21.10.2012, at 14.29, Nicol?s wrote: >> >>> El 21/10/2012 12:26, Timo Sirainen escribi?: >>>> On 21.10.2012, at 14.15, Nicol?s wrote: >>>> >>>>> Oct 12 14:29:22 mail dovecot: imap(nicolas at devels.es): Error: fstat() failed with file /home/vmail/devels.es/nicolas/Maildir/dovecot.index.log: No such file or directory >>>> fstat() can't normally fail with ENOENT. Are you using NFS or some other non-local filesystem? >>>> >>> No, on that machine I'm just using local filesystem, no NFS. >> Then it's a kernel bug. Although I guess there are some workarounds that could be done on Dovecot code. >> > > Well, at least it's good to know it's not a misconfiguration :-) If there's any additional info I can provide feel free to ask. Any way you can reproduce this somewhat easily? For example with http://imapwiki.org/ImapTest ? From daniel.parthey at informatik.tu-chemnitz.de Sun Oct 21 23:49:19 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 21 Oct 2012 22:49:19 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020124749.GA26942@daniel.localdomain> <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> Message-ID: <20121021204919.GA9191@daniel.localdomain> Hi Dave, David Mehler wrote: > Thanks for all your help so far. I have per-user quotas now working, I > had to also alter my dict config file as well. I am having two > outstanding issues, actually one outstanding issue and one question. > > Here's the question, given that the userdb sql query returns a default > quota entry for rows of zero in quota and quota_messages is the > default quota section needed in 90-quota.conf? > > plugin { > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > } You need quota_rule2 to give the user some additional space in the Trash folder if he/she wants to delete messages when over quota. The dict is also needed for quota lookup from the database. The only thing which might be omitted is the global quota_rule since it is returned by the userdb/passwd in any case, but I'm not sure what happens if you only configure a "quota_rule2" without configuring a "quota_rule". > My outstanding issue is whenever I as the root mysql user update a > user's quota the other user also gets an update, I noticed with one > the messages column on the other user went from 0 to 2, another time > the quota value went up from 0 to 3500 it seems random. You should not be accounting the actual mailbox usage in the same virtual_users table as the quota is read from. Use *different* column or table name in your dict file where dovecot may write the current storage/message count. > dovecot-dict-sql.conf.ext > > map { > pattern = priv/quota/storage > table = virtual_users > username_field = user > value_field = quota value_field should be current_quota_storage (writable column) > } > map { > pattern = priv/quota/messages > table = virtual_users > username_field = user > value_field = quota_messages value_field should be current_quota_messages (writable column) > I'd like to know why these columns are updating. Dovecot stores the current storage and mailcount in there. These columns should be different from the columns defining the maximum limit. Regards Daniel From daniel.parthey at informatik.tu-chemnitz.de Mon Oct 22 01:22:07 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 22 Oct 2012 00:22:07 +0200 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: References: <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> <20121021204919.GA9191@daniel.localdomain> Message-ID: <20121021222207.GA10903@daniel.localdomain> David Mehler wrote: > Thanks, so if I understand what your saying the reason I'm getting the > column update issues is Dovecot is reading from and writing to the > quota and quota_messages columns in my virtual_users table? > > My database user I believe only has select permissions on that table. > > So, I either need another table and to adjust my dovecot-dict.sql file > for that table, that's where Dovecot will write to, or two more > columns in the virtual_users table? > > Which way do you recommend? I would recommend to create a new table for dovecot_usage where dovecot is granted write permission. Regards Daniel From dmalolepszy at optusnet.com.au Mon Oct 22 02:08:01 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Mon, 22 Oct 2012 10:08:01 +1100 Subject: [Dovecot] Dovecot LDA message save logging Message-ID: <50848051.6050308@optusnet.com.au> Hi, Is there any option in Dovecot that enables logging the full path of where a message is saved in the backend? Dominic From dave.mehler at gmail.com Mon Oct 22 02:14:56 2012 From: dave.mehler at gmail.com (David Mehler) Date: Sun, 21 Oct 2012 19:14:56 -0400 Subject: [Dovecot] still having difficulties with per-user quotas In-Reply-To: <20121021222207.GA10903@daniel.localdomain> References: <20121020185124.GA2609@daniel.localdomain> <20121020200432.GA3161@daniel.localdomain> <20121020214900.GA5887@daniel.localdomain> <20121021204919.GA9191@daniel.localdomain> <20121021222207.GA10903@daniel.localdomain> Message-ID: Hello, Thanks. I've created a quota table as described in dovecot-dict sql configuration file and granted the mail user select, insert, update, and delete rights to that table, while the virtual_users table select rights only. I configured for the new table. Here's my config, have I got it? Are the columns now going to stay where I put them? mysql> show create table virtual_users; +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | virtual_users | CREATE TABLE `virtual_users` ( `id` int(11) NOT NULL AUTO_INCREMENT, `domain_id` int(11) NOT NULL, `user` varchar(40) NOT NULL, `password` varchar(128) NOT NULL, `quota` bigint(20) NOT NULL DEFAULT '0', `quota_messages` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`id`), UNIQUE KEY `UNIQUE_EMAIL` (`domain_id`,`user`), CONSTRAINT `virtual_users_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `virtual_domains` (`id`) ON DELETE CASCADE ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 | +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> show create table quota; +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | quota | CREATE TABLE `quota` ( `username` varchar(100) NOT NULL, `bytes` bigint(20) NOT NULL DEFAULT '0', `messages` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`username`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) dovecot dictionary configuration map { pattern = priv/quota/storage table = quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = username value_field = messages } Thanks. Dave. On 10/21/12, Daniel Parthey wrote: > David Mehler wrote: >> Thanks, so if I understand what your saying the reason I'm getting the >> column update issues is Dovecot is reading from and writing to the >> quota and quota_messages columns in my virtual_users table? >> >> My database user I believe only has select permissions on that table. >> >> So, I either need another table and to adjust my dovecot-dict.sql file >> for that table, that's where Dovecot will write to, or two more >> columns in the virtual_users table? >> >> Which way do you recommend? > > I would recommend to create a new table for dovecot_usage > where dovecot is granted write permission. > > Regards > Daniel > From dovecot at knutejohnson.com Mon Oct 22 04:14:56 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 21 Oct 2012 18:14:56 -0700 Subject: [Dovecot] Anybody recognize these log lines? Message-ID: <50849E10.1080404@knutejohnson.com> WARN: Duplicate profile 'Dovecot POP3', using last found WARN: Duplicate profile 'Dovecot Secure POP3', using last found WARN: Duplicate profile 'Dovecot IMAP', using last found WARN: Duplicate profile 'Dovecot Secure IMAP', using last found Anybody know if these are dovecot generated? Thanks, -- Knute Johnson From mcguire at neurotica.com Mon Oct 22 04:17:07 2012 From: mcguire at neurotica.com (Dave McGuire) Date: Sun, 21 Oct 2012 21:17:07 -0400 Subject: [Dovecot] Anybody recognize these log lines? In-Reply-To: <50849E10.1080404@knutejohnson.com> References: <50849E10.1080404@knutejohnson.com> Message-ID: <50849E93.8040706@neurotica.com> On 10/21/2012 09:14 PM, Knute Johnson wrote: > WARN: Duplicate profile 'Dovecot POP3', using last found > WARN: Duplicate profile 'Dovecot Secure POP3', using last found > WARN: Duplicate profile 'Dovecot IMAP', using last found > WARN: Duplicate profile 'Dovecot Secure IMAP', using last found > > Anybody know if these are dovecot generated? Looks like output from the "ufw" firewall package. -Dave -- Dave McGuire, AK4HZ New Kensington, PA From alessio at skye.it Mon Oct 22 08:51:54 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 22 Oct 2012 07:51:54 +0200 Subject: [Dovecot] Segmentation fault in doveadm with lib01_acl_plugin.so In-Reply-To: <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> References: <8f93cb3c40e68e7628392a3f113bc83e@skye.it> <6F094CBD-8642-49DA-9C3E-D5CEF0334F53@iki.fi> Message-ID: <5084DEFA.6060404@skye.it> Il 13/10/2012 10:42, Timo Sirainen ha scritto: > On 13.10.2012, at 11.16, Alessio Cecchi wrote: > >> I'm running dovecot 2.1.10 on Debian 6. >> >> When I run "doveadm expunge -A mailbox Trash savedbefore 30d" it crash with "Segmentation fault" >> >> [15022673.496902] doveadm[13072]: segfault at 8 ip 00007f4b7041f551 sp 00007fffdab4f8c0 error 4 in lib01_acl_plugin.so[7f4b70415000+10000] > The most helpful way to get this fixed is to get a gdb backtrace: http://dovecot.org/bugreport.html > Hi, after run "dovecot quota recalc -A" no error from "doveadm expunge -A mailbox Trash savedbefore 30d" is appeared and so have not been able to generate the "dump". -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From raabe at froglogic.com Mon Oct 22 10:21:51 2012 From: raabe at froglogic.com (Frerich Raabe) Date: Mon, 22 Oct 2012 09:21:51 +0200 Subject: [Dovecot] Marking all mail in one folder of public mailbox as read In-Reply-To: References: <507EA81C.5060806@froglogic.com> Message-ID: <5084F40F.7070601@froglogic.com> Am 10/18/2012 5:31 AM, schrieb Timo Sirainen: > Use: > > prefix=Lists/anotherlist/ > location = maildir:/home/vmail/lists/sharedseen/Maildir > > Then deliver the mails to /home/vmail/lists/sharedseen/Maildir root directly. Of course this means that you need to create a namespace for each such list. > > Alternative would be to use prefix=Lists/sharedseen/ and create lists under it. Thanks, the second version is basically what I did! I added a new namespace namespace public { separator = / prefix = Lists/Archive/ location = maildir:/home/vmail/lists/archive/Maildir subscriptions = no } ...and then had my Sieve script fileinto that. Works fine! Thanks for your help! -- Frerich Raabe - raabe at froglogic.com www.froglogic.com - Multi-Platform GUI Testing From amateo at um.es Mon Oct 22 13:58:10 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 22 Oct 2012 12:58:10 +0200 Subject: [Dovecot] Auth caching and password changes In-Reply-To: <508149CC.9070004@um.es> References: <508149CC.9070004@um.es> Message-ID: <508526C2.8030403@um.es> El 19/10/12 14:38, Angel L. Mateo escribi?: > Hello, > > In my system I have configured auth caching. The problem I have is > that whenever a user changes his password, he/she can't login to dovecot > after a while and the scenarios described at > http://wiki2.dovecot.org/Authentication/Caching are not applied. > > I have tried also with "doveadm auth cache flush ", but it > didn't work. He also could to login again if he waits for a time or if I > run "doveadm auth cache flush" in the server, flushing all auth > information from cache. > > I have attached the log I had when I changed my password (and > suffered the problem). I have attached my doveconf -n too. > I think I have found part of the problem. My problem is that my authentication chain is first try by ldap (for normal clients authentication), and if it failed, then try with pam_cas (for webmail accesses with SSO). My change password application forms part of webmail, which also uses an imapproxy, so when I change the password, automatically seems to enter in the "Early change scenario" (I still haven't found the concrete reason for this). But I have tried to manually change the password in my ldap servers, and it works fine. So my point is that something related with this authentication chain provokes this scenario. My question now is there any way to configure authentication so a mechanism is only use when connections coming from a set of IPs? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337 From tss at iki.fi Mon Oct 22 15:39:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 22 Oct 2012 15:39:34 +0300 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <20120926180633.GE80443@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: On 26.9.2012, at 21.06, Kelsey Cummings wrote: > 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) > 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) > 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) Looks like I had broken this in v2.1.8. http://hg.dovecot.org/dovecot-2.1/rev/e4c337f38ed6 fixes this. I also added a bunch of other things to give better error messages and to try to fix any unexpected problems. From berni at birkenwald.de Mon Oct 22 17:23:18 2012 From: berni at birkenwald.de (Bernhard Schmidt) Date: Mon, 22 Oct 2012 14:23:18 +0000 (UTC) Subject: [Dovecot] auth timeout state=2, bad? Message-ID: Hello, we run Dovecot 2.1.7 as SASL backend for our Postfix SMTP-AUTH farm and see this error message occasionally Oct 22 16:15:32 lxmhs52 dovecot: auth: PLAIN(?,xx.xx.xx.xx): Request 0.21 timeouted after 150 secs, state=2 Since it is mostly the same IP repeating I'm assuming it's a client issue. Is that correct? What could cause this? Can we safely ignore it? Thanks, Bernhard From busseniu at in.tum.de Mon Oct 22 17:33:33 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 22 Oct 2012 16:33:33 +0200 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 Message-ID: <5085593D.3080403@in.tum.de> Hi, Thunderbird uses the following IMAP command to list shared folders: . list "" INBOX.shared.%.% Dovecot 2.1.10 does not list any folders in response to this command. Dovecot 2.0.21 does list them: * LIST (\HasNoChildren) "." "INBOX.shared.user1.folder" . OK List completed. Both versions list the folders if "*" is used instead of "INBOX.shared.%.%". Because of this issue, shared folders are not shown in Thunderbird. The ACL of the folder in question is ACL "keilrwtscd" for user1 (who is trying to access them using Thunderbird). config: # 2.1.10: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mail mail_plugins = acl mail_uid = vmail namespace { inbox = no list = children location = mdbox:%%h/mail prefix = INBOX.shared.%%u. separator = . subscriptions = no type = shared } namespace default { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/mail/shared-mailboxes } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Mon Oct 22 18:59:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 22 Oct 2012 18:59:52 +0300 Subject: [Dovecot] auth timeout state=2, bad? In-Reply-To: References: Message-ID: <9790CECF-1FCE-4125-929B-CE0A53483495@iki.fi> On 22.10.2012, at 17.23, Bernhard Schmidt wrote: > we run Dovecot 2.1.7 as SASL backend for our Postfix SMTP-AUTH farm and > see this error message occasionally > > Oct 22 16:15:32 lxmhs52 dovecot: auth: PLAIN(?,xx.xx.xx.xx): Request > 0.21 timeouted after 150 secs, state=2 state 2 = "waiting for auth data from client", so what it means is that the client sent: AUTH PLAIN and then just didn't do anything. > Since it is mostly the same IP repeating I'm assuming it's a client > issue. Is that correct? What could cause this? Can we safely ignore it? You can safely ignore it. It should have been logged with "info" level and only with auth_verbose=yes. Although I guess the message could be a bit nicer. This is better I think: http://hg.dovecot.org/dovecot-2.1/rev/49bb6cc43d03 From kgc at corp.sonic.net Tue Oct 23 01:29:21 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 22 Oct 2012 15:29:21 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: <20121022222921.GE3370@corp.sonic.net> On Mon, Oct 22, 2012 at 03:39:34PM +0300, Timo Sirainen wrote: > On 26.9.2012, at 21.06, Kelsey Cummings wrote: > > > 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) > > 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) > > 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) > > Looks like I had broken this in v2.1.8. http://hg.dovecot.org/dovecot-2.1/rev/e4c337f38ed6 fixes this. I also added a bunch of other things to give better error messages and to try to fix any unexpected problems. Thanks Timo! -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From dovecot at knutejohnson.com Tue Oct 23 03:29:54 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Mon, 22 Oct 2012 17:29:54 -0700 Subject: [Dovecot] Anybody recognize these log lines? In-Reply-To: <50849E93.8040706@neurotica.com> References: <50849E10.1080404@knutejohnson.com> <50849E93.8040706@neurotica.com> Message-ID: <5085E502.3080802@knutejohnson.com> On 10/21/2012 6:17 PM, Dave McGuire wrote: > On 10/21/2012 09:14 PM, Knute Johnson wrote: >> WARN: Duplicate profile 'Dovecot POP3', using last found >> WARN: Duplicate profile 'Dovecot Secure POP3', using last found >> WARN: Duplicate profile 'Dovecot IMAP', using last found >> WARN: Duplicate profile 'Dovecot Secure IMAP', using last found >> >> Anybody know if these are dovecot generated? > > Looks like output from the "ufw" firewall package. > > -Dave > Thanks, I'll look at that. -- Knute Johnson From list at airstreamcomm.net Tue Oct 23 08:49:47 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 23 Oct 2012 00:49:47 -0500 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <20121021194320.GA7977@daniel.localdomain> References: <5083D963.3000700@wk-serv.de> <20121021194320.GA7977@daniel.localdomain> Message-ID: <50862FFB.2060108@airstreamcomm.net> On 10/21/12 2:43 PM, Daniel Parthey wrote: > Patrick Westenberg wrote: >> Is there no built in failover mechanism for the director service to >> handle a backend failure? > No, the director's job is to keep a hash table and direct > the connection for each user to its associated backend. > Currently, there is no built-in backend monitoring. > > In order to handle maintenance of backends, you will need the > poolmon daemon, which enables/disables backends in the director > depending on their availability: > > https://github.com/brandond/poolmon > > Regards > Daniel Considering the intention of the director was to alleviate locking issues in a shared storage environment are there any current solutions to improving the scalability/availability of Dovecot by implementing an alternative message storage systems such as nosql or maybe object storage that could abstract away the complexity of replicating data? We would love to finally have the ability to set our mail cluster on top of a storage subsystem that can span multiple geographic regions and do away with the NFS backend. From alessio at skye.it Tue Oct 23 09:00:52 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 23 Oct 2012 08:00:52 +0200 Subject: [Dovecot] Dovecot LDA message save logging In-Reply-To: <50848051.6050308@optusnet.com.au> References: <50848051.6050308@optusnet.com.au> Message-ID: <50863294.2010404@skye.it> Il 22/10/2012 01:08, Dominic Malolepszy ha scritto: > Hi, > > Is there any option in Dovecot that enables logging the full path of > where a message is saved in the backend? > > Dominic > With mail_debug=yes you can see it. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From eliezer at ngtech.co.il Tue Oct 23 18:19:04 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Tue, 23 Oct 2012 17:19:04 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 Message-ID: <5086B568.1010905@ngtech.co.il> Since I have lots of filtering rules in thunderbird I was thinking of using sieve instead. I want to filter incoming mail into subdirectories. like "from" store at folder "old". the script is: require ["fileinto", "envelope"]; if envelope :is "from" "eliezer at test.dom" { fileinto "old"; } else { # The rest goes into INBOX # default is "implicit keep", we do it explicitly here keep; } the result is that the mail is stored in two folders instead of just one, INBOX and old. the logs shows: Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing script from /home/vmail/domain/eliezer/home/.dovecot.svbin Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' from unknown reason(or I didnt understood how sieve works?) plugin section from dovecot -n plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.debug +imapflags +relational +comparator-i;ascii-numeric } Thanks, Eliezer From stsiol at yahoo.co.uk Tue Oct 23 19:03:38 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 23 Oct 2012 17:03:38 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb [Resolved] Message-ID: <1351008218.44057.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello all, I would like to thank you all for your kind replies and feedback in regards to migrating from a smaller hdd to a bigger one (namely from 72gb to 146gb). I finally found a painless way of doing this. Since I believe that this is still an off-topic post, if anyone is interested in the solution i've adopted for this, let me know by replying to me privately. If, however, you don't mind me posting here, let me know. Many many thanks go to?Alexander Hoogerhuis, the "mad Norwegian" :-) who helped me on this too much to describe here. Alex, you are a true sport. Thank you again people. All the Best, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From dg at dguhl.org Tue Oct 23 19:21:27 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 23 Oct 2012 18:21:27 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <20121023162125.GA7983@PC211.ikt.de> On Tue, Oct 23, 2012 at 05:19:04PM +0200, Eliezer Croitoru wrote: [..] > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; stop; # seems to be needed with explicit keep > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } Comparing various sieve scripts I concluded (but I didn't actually test it) that you need a stop; in your if or elsif clauses in case you have an else clause with an explicit keep; -- IMHO a behaviour violating RFC 5228. RFC 5228, Section 3.1 ... If the test of the "if" is false, it evaluates the test of the first "elsif" (if any). If the test of "elsif" is true, it runs the elsif's block. An elsif may be followed by an elsif, in which case, the interpreter repeats this process until it runs out of elsifs. When the interpreter runs out of elsifs, there may be an "else" case. If there is, and none of the if or elsif tests were true, the interpreter runs the else's block. ... Stefan, can you enlighten us? Dennis [..] From john.roman at dreamhost.com Tue Oct 23 20:04:12 2012 From: john.roman at dreamhost.com (john roman) Date: Tue, 23 Oct 2012 10:04:12 -0700 Subject: [Dovecot] index files created improperly in dovecot 1.2.16 Message-ID: Greetings, It seems to be a problem that ive seen occasionally on the web with few results as to a solution, but im experiencing it as well. Namely, dovecot creates index logs for users with a 600 permission, when it should create with a 700 permission. My indexes are stored in /var/indexes with the directory at 777 permission, the users are identified in MySQL. The error is as follows: Oct 23 09:03:13 mailer01 dovecot: POP3 (johnr at testing.com): stat(/var/indexes/j/johnr at testing.com/.INBOX) failed: Permission denied (euid=10509305(x10509305) egid=81607(pg199275) missing +x perm: /var/indexes/j) the permissions of /var/indexes/j are 700, owned by the euid and egid effectively. the only file that does not have this permission is in /var/indexes/j/johnr at testing.com/.INBOX/dovecot.index.log, at 600. From trashcan at odo.in-berlin.de Tue Oct 23 22:06:51 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 23 Oct 2012 21:06:51 +0200 Subject: [Dovecot] [2.2-UNSTABLE] compilation error: 'POSIX_FADV_WILLNEED' undeclared Message-ID: <198596C8-3989-4041-B96A-5D5AFEE8E3D0@odo.in-berlin.de> Hi -- I am trying to compile 2.2 (acd76b5272e9) at FreeBSD 9.0: | libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I/usr/local/include -std=gnu99 -O2 -Wall -W -Wmissing-prototypes \ -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 \ -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 \ -I/usr/local/include -MT fs-posix.lo -MD -MP -MF .deps/fs-posix.Tpo \ -c fs-posix.c -fPIC -DPIC -o .libs/fs-posix.o | fs-posix.c: In function 'fs_posix_prefetch': | fs-posix.c:298: warning: implicit declaration of function 'posix_fadvise' | fs-posix.c:298: error: 'POSIX_FADV_WILLNEED' undeclared (first use in this function) | fs-posix.c:298: error: (Each undeclared identifier is reported only once | fs-posix.c:298: error: for each function it appears in.) | gmake[3]: *** [fs-posix.lo] Error 1 | gmake[3]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src/lib-fs' | gmake[2]: *** [all-recursive] Error 1 | gmake[2]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src' | gmake[1]: *** [all-recursive] Error 1 | gmake[1]: Leaving directory `/usr/local/etc/dovecot/SOURCE/dovecot-2.2' | gmake: *** [all] Error 2 From configure logfile: | checking for posix_fadvise... no After a modification (stolen from src/lib-storage/index/index-mail.c) ... | --- dovecot-2.2-modified/src/lib-fs/fs-posix.c 2012-10-23 20:27:31.348919455 +0200 | +++ dovecot-2.2/src/lib-fs/fs-posix.c 2012-10-23 20:26:39.435300269 +0200 | @@ -295,10 +295,12 @@ | return TRUE; | } | | +#if defined(HAVE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED) | if (posix_fadvise(file->fd, 0, length, POSIX_FADV_WILLNEED) < 0) { | i_error("posix_fadvise(%s) failed: %m", _file->path); | return TRUE; | } | +#endif | return FALSE; | } ... the compilations runs to completion, and dovecot-2.2 UNSTABLE is running. But: I do not have the knowledge to judge if that "fix" will be the right one. Just to let you know and with regards, Michael From stephan at rename-it.nl Tue Oct 23 22:40:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 23 Oct 2012 21:40:43 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <5086F2BB.7010704@rename-it.nl> On 10/23/2012 5:19 PM, Eliezer Croitoru wrote: > Since I have lots of filtering rules in thunderbird I was thinking of > using sieve instead. > I want to filter incoming mail into subdirectories. > like "from" store at folder "old". > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } > > the result is that the mail is stored in two folders instead of just > one, INBOX and old. > the logs shows: > Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing > script from /home/vmail/domain/eliezer/home/.dovecot.svbin > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' > > from unknown reason(or I didnt understood how sieve works?) This behavior would definitely be a bug, but I cannot reproduce it even with Dovecot 2.0.17 and Pigeonhole v0.2.5: stephan at host:~/src/devel/dovecot-2.0-pigeonhole$ src/sieve-tools/sieve-test -t - -Tlevel=matching -e ~/frop.sieve ~/frop.eml ## Started executing script 'frop' 2: envelope test 2: starting `:is' match with `i;ascii-casemap' comparator: 2: getting `from' part from message envelope 2: extracting `all' part from address `eliezer at test.dom' 2: matching value `eliezer at test.dom' 2: with key `eliezer at test.dom' => 1 2: finishing match with result: matched 2: jump if result is false 2: not jumping 3: fileinto action 3: store message in mailbox `old' 3: jumping to line 7 ## Finished executing script 'frop' info: msgid=unspecified: stored mail into mailbox 'old'. sieve-test(stephan): Info: final result: success Could you test this at your end? Be careful, the above command adds a message to the user's mailbox, so read the sieve-test man page first before you try anything. Also, I executed this from my development tree, because I haven't got an operational Dovecot v2.0 installation. I don't remember any bug that was solved since that version that could explain what you're seeing. What is your Pigeonhole version? I've tried with v0.2.5 at this end. Also, could you provide your full configuration as output from `dovecot -n` ? Regards, Stephan. From benedetto.vassallo at unipa.it Tue Oct 23 23:18:06 2012 From: benedetto.vassallo at unipa.it (Benedetto Vassallo) Date: Tue, 23 Oct 2012 22:18:06 +0200 Subject: [Dovecot] Maildir hardlinks In-Reply-To: <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> References: <20121004150003.82273ocvoezpeus3@webmail.unipa.it> <304874C3-190D-43AA-8035-7272C65FBB30@iki.fi> <20121016091153.15601eysq5n040qh@webmail.unipa.it> <948A0991-BD2B-4F42-8827-9BBC64BB43DD@iki.fi> Message-ID: <20121023221806.43795tha204qxgfy@webmail.unipa.it> Def. Quota Timo Sirainen : > On 16.10.2012, at 10.11, Benedetto Vassallo wrote: > >>> What are the permissions of the MailDir directory for user1/user2? >>> >>> ls -ld /home/user1/MailDir >>> ls -ld /home/user2/MailDir >>> >>> >> >> Thank you for your reply. >> They are different groups: >> >> drwxr-xr-x 9 user1 grp1 4096 15 ott 14:52 /home/user1/MailDir/ >> drwxr-xr-x 5 user2 grp2 4096 4 ott 23:43 /home/user2/MailDir/ >> drwxr-xr-x 10 user3 grp3 4096 15 ott 14:52 /home/user3/MailDir/ > > Not very secure permissions.. Maybe would be easiest to just have > one vmail user for everyone? > >> I tryed to issue: >> chgrp -R mail /home/user1/MailDir >> chgrp -R mail /home/user2/MailDir >> chgrp -R mail /home/user3/MailDir > > Dovecot doesn't do hard linking when it looks like the permissions > aren't compatible. The current code checks that if the owner UIDs > are different, then the group needs to be writable. On my production server with dovecot 2.0.13 I have same permissions and it works. I changed my permissions in any mode, changed the owner, the group but it still don't work. Any suggestion? Thank you -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Universit? degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface. From marc at perkel.com Tue Oct 23 23:51:39 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 23 Oct 2012 13:51:39 -0700 Subject: [Dovecot] Can Dovecot authenticate against an external email server? Message-ID: <5087035B.7060208@perkel.com> Just wondering if anyone has done this. I have a spam filtering service where I am now storing spam for users I filter for. It's a filter and forward service so I don't control the recipient's email server. What I would like to do somehow is have the user enter their email address and password and then look up their POP/IMAP server from a database and try to authenticate from it. If sucessful then the user will be able to access their stored spam using Dovecot and Squirrelmail. Any suggestions? Thanks in advance From troy at troyvit.com Tue Oct 23 23:52:45 2012 From: troy at troyvit.com (Troy Vitullo) Date: Tue, 23 Oct 2012 14:52:45 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver Message-ID: <20121023145245.124dd362@hrafn> Hi, My server uses a system comprised of postfix, dovecot and dspam to filter and deliver mail. Postfix used the following flags in calling spamc and dovecot: flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} after an upgrade from Debian lenny to squeeze we were able to get everything working except spam filtering. Spamassassin is able to judge whether the mail coming in is spam but everything stops there. In mail.err I see: pamc[3608]: exec failed: Permission denied spamc shows the same thing in syslog: exec failed: Permission denied postfix delays the email: postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system resource problem) Here are the permissions for deliver: -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver Here are the relevant groups: s1:~# grep dovecot /etc/group secmail:x:119:postfix,spamd,dovecot dovecot:x:111: here's the dovecot user: s1:~# grep dovecot /etc/passwd dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false here's dovecot -n: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 base_dir: /var/run/dovecot/ protocols: imap imaps pop3s pop3 ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert ssl_key_file: /etc/ssl/private/s1.troyvit.com.key ssl_cipher_list: ALL:!LOW disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_location: maildir:%h/Maildir/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s namespace: type: private separator: / inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmaster at sphere.local auth_socket_path: /var/run/dovecot/auth-master mail_plugin_dir: /usr/lib/dovecot/modules/lda/ mail_plugins: sieve auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: pam args: dovecot passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 438 user: dovecot plugin: sieve_global_path: /etc/dovecot/default.sieve sieve: /srv/%d/mail/%n/%n.sieve Many thanks in advance for any advice you can give. Troy From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 24 01:12:33 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 24 Oct 2012 00:12:33 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <50862FFB.2060108@airstreamcomm.net> References: <5083D963.3000700@wk-serv.de> <20121021194320.GA7977@daniel.localdomain> <50862FFB.2060108@airstreamcomm.net> Message-ID: <20121023221233.GA22084@daniel.localdomain> list at airstreamcomm.net wrote: > Considering the intention of the director was to alleviate locking > issues in a shared storage environment are there any current > solutions to improving the scalability/availability of Dovecot by > implementing an alternative message storage systems such as nosql or > maybe object storage that could abstract away the complexity of > replicating data? We would love to finally have the ability to set > our mail cluster on top of a storage subsystem that can span > multiple geographic regions and do away with the NFS backend. Key/value object store is planned for Dovecot v2.2 and has been discussed in this thread: http://dovecot.org/list/dovecot/2012-September/068257.html Regards Daniel -- https://plus.google.com/103021802792276734820 From Bill at KnoxvilleChristian.org Wed Oct 24 04:06:17 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:06:17 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121023145245.124dd362@hrafn> References: <20121023145245.124dd362@hrafn> Message-ID: <50873F09.4070604@KnoxvilleChristian.org> On 10/23/2012 4:52 PM, Troy Vitullo wrote: > Hi, > > My server uses a system comprised of postfix, dovecot and dspam to filter and deliver mail. > > Postfix used the following flags in calling spamc and dovecot: > > flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} > > after an upgrade from Debian lenny to squeeze we were able to get everything working except spam filtering. Spamassassin is able to judge whether the mail coming in is spam but everything stops there. > > In mail.err I see: > > pamc[3608]: exec failed: Permission denied > > spamc shows the same thing in syslog: > > exec failed: Permission denied > > postfix delays the email: > > postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system resource problem) > > Here are the permissions for deliver: > > -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver > > Here are the relevant groups: > > s1:~# grep dovecot /etc/group > secmail:x:119:postfix,spamd,dovecot > dovecot:x:111: > > here's the dovecot user: > s1:~# grep dovecot /etc/passwd > dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false > > here's dovecot -n: > > # 1.2.15: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 > base_dir: /var/run/dovecot/ > protocols: imap imaps pop3s pop3 > ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert > ssl_key_file: /etc/ssl/private/s1.troyvit.com.key > ssl_cipher_list: ALL:!LOW > disable_plaintext_auth: no > verbose_ssl: yes > login_dir: /var/run/dovecot/login > login_executable(default): /usr/lib/dovecot/imap-login > login_executable(imap): /usr/lib/dovecot/imap-login > login_executable(pop3): /usr/lib/dovecot/pop3-login > mail_location: maildir:%h/Maildir/ > mbox_write_locks: fcntl dotlock > mail_executable(default): /usr/lib/dovecot/imap > mail_executable(imap): /usr/lib/dovecot/imap > mail_executable(pop3): /usr/lib/dovecot/pop3 > mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > pop3_enable_last(default): no > pop3_enable_last(imap): no > pop3_enable_last(pop3): yes > pop3_client_workarounds(default): > pop3_client_workarounds(imap): > pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh > pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s > namespace: > type: private > separator: / > inbox: yes > list: yes > subscriptions: yes > lda: > postmaster_address: postmaster at sphere.local > auth_socket_path: /var/run/dovecot/auth-master > mail_plugin_dir: /usr/lib/dovecot/modules/lda/ > mail_plugins: sieve > auth default: > mechanisms: plain login > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: pam > args: dovecot > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: passwd > userdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: postfix > master: > path: /var/run/dovecot/auth-master > mode: 438 > user: dovecot > plugin: > sieve_global_path: /etc/dovecot/default.sieve > sieve: /srv/%d/mail/%n/%n.sieve > > Many thanks in advance for any advice you can give. > > Troy What is your mailbox_command in main.cf? I just use: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" I don't need anything in master.cf. But you should be using -u ${user} for spamc. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 04:15:34 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:15:34 -0400 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086B568.1010905@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> Message-ID: <50874136.7000903@KnoxvilleChristian.org> On 10/23/2012 11:19 AM, Eliezer Croitoru wrote: > Since I have lots of filtering rules in thunderbird I was thinking of > using sieve instead. > I want to filter incoming mail into subdirectories. > like "from" store at folder "old". > the script is: > require ["fileinto", "envelope"]; > if envelope :is "from" "eliezer at test.dom" { > fileinto "old"; > } else { > # The rest goes into INBOX > # default is "implicit keep", we do it explicitly here > keep; > } > > the result is that the mail is stored in two folders instead of just > one, INBOX and old. > the logs shows: > Oct 23 17:12:26 lda(eliezer at ngtech.co.il): Debug: sieve: executing > script from /home/vmail/domain/eliezer/home/.dovecot.svbin > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'INBOX' > Oct 23 17:12:26 lda(eliezer at test1.dom): Info: sieve: > msgid=<5086B3C9.5030909 at test.dom>: stored mail into mailbox 'old' > > from unknown reason(or I didnt understood how sieve works?) > > plugin section from dovecot -n > > plugin { > ... > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > sieve_extensions = +vnd.dovecot.debug +imapflags +relational > +comparator-i;ascii-numeric > } > > Thanks, > Eliezer Why to people bother with all these complex if...elsif...else structures. I just use stop a lot. This is included from my Main.seive: # # 2012-07-05 # require "include"; require "fileinto"; require "copy"; #require "body"; #require "imap4flags"; # put this in main #if header :contains "list-id" "dovecot.dovecot.org" { include "Dovecot"; } if address :is :localpart "to" "dovecot" { fileinto :copy "SystemFolders.Ham"; fileinto "Lists.Dovecot"; stop; } fileinto "Lists"; stop; As you can see, I also train Spamassassin with the mail from the list. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 04:32:59 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Tue, 23 Oct 2012 21:32:59 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50873F09.4070604@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> Message-ID: <5087454B.5030704@KnoxvilleChristian.org> On 10/23/2012 9:06 PM, Bill Shirley wrote: > > On 10/23/2012 4:52 PM, Troy Vitullo wrote: >> Hi, >> >> My server uses a system comprised of postfix, dovecot and dspam to >> filter and deliver mail. >> >> Postfix used the following flags in calling spamc and dovecot: >> >> flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} >> -e /usr/lib/dovecot/deliver -d ${recipient} >> >> after an upgrade from Debian lenny to squeeze we were able to get >> everything working except spam filtering. Spamassassin is able to >> judge whether the mail coming in is spam but everything stops there. >> >> In mail.err I see: >> >> pamc[3608]: exec failed: Permission denied >> >> spamc shows the same thing in syslog: >> >> exec failed: Permission denied >> >> postfix delays the email: >> >> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred (system >> resource problem) >> >> Here are the permissions for deliver: >> >> -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver >> >> Here are the relevant groups: >> >> s1:~# grep dovecot /etc/group >> secmail:x:119:postfix,spamd,dovecot >> dovecot:x:111: >> >> here's the dovecot user: >> s1:~# grep dovecot /etc/passwd >> dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false >> >> here's dovecot -n: >> >> # 1.2.15: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.26-2-686 i686 Debian 6.0.6 >> base_dir: /var/run/dovecot/ >> protocols: imap imaps pop3s pop3 >> ssl_cert_file: /etc/ssl/certs/s1.troyvit.com.cert >> ssl_key_file: /etc/ssl/private/s1.troyvit.com.key >> ssl_cipher_list: ALL:!LOW >> disable_plaintext_auth: no >> verbose_ssl: yes >> login_dir: /var/run/dovecot/login >> login_executable(default): /usr/lib/dovecot/imap-login >> login_executable(imap): /usr/lib/dovecot/imap-login >> login_executable(pop3): /usr/lib/dovecot/pop3-login >> mail_location: maildir:%h/Maildir/ >> mbox_write_locks: fcntl dotlock >> mail_executable(default): /usr/lib/dovecot/imap >> mail_executable(imap): /usr/lib/dovecot/imap >> mail_executable(pop3): /usr/lib/dovecot/pop3 >> mail_plugin_dir(default): /usr/lib/dovecot/modules/imap >> mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap >> mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 >> pop3_enable_last(default): no >> pop3_enable_last(imap): no >> pop3_enable_last(pop3): yes >> pop3_client_workarounds(default): >> pop3_client_workarounds(imap): >> pop3_client_workarounds(pop3): outlook-no-nuls, oe-ns-eoh >> pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s >> pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s >> pop3_logout_format(pop3): top=%t/%T, retr=%r/%R, del=%d/%m, size=%s >> namespace: >> type: private >> separator: / >> inbox: yes >> list: yes >> subscriptions: yes >> lda: >> postmaster_address: postmaster at sphere.local >> auth_socket_path: /var/run/dovecot/auth-master >> mail_plugin_dir: /usr/lib/dovecot/modules/lda/ >> mail_plugins: sieve >> auth default: >> mechanisms: plain login >> verbose: yes >> debug: yes >> debug_passwords: yes >> passdb: >> driver: pam >> args: dovecot >> passdb: >> driver: sql >> args: /etc/dovecot/dovecot-sql.conf >> userdb: >> driver: passwd >> userdb: >> driver: sql >> args: /etc/dovecot/dovecot-sql.conf >> socket: >> type: listen >> client: >> path: /var/spool/postfix/private/auth >> mode: 432 >> user: postfix >> group: postfix >> master: >> path: /var/run/dovecot/auth-master >> mode: 438 >> user: dovecot >> plugin: >> sieve_global_path: /etc/dovecot/default.sieve >> sieve: /srv/%d/mail/%n/%n.sieve >> >> Many thanks in advance for any advice you can give. >> >> Troy > > What is your mailbox_command in main.cf? I just use: > mailbox_command = /usr/bin/spamc -u "$USER" -e > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" > > I don't need anything in master.cf. But you should be using -u > ${user} for spamc. > > Bill > Forgot to ask, are you using Spamassassin's per-user configs? If you're not, that probably is your problem. It's probably trying to update bayes tokens and it doesn't have permission. I use per-user configs which are nice. One man's spam is another man's ham. Plus each user can have his/her own whitelist. I use these spamd args: -d -c -m10 --user-config You usually can find the args in /etc/sysconfig. Bill From rs at sys4.de Wed Oct 24 09:33:26 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 08:33:26 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5087454B.5030704@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> Message-ID: <50878BB6.2090309@sys4.de> Am 24.10.2012 03:32, schrieb Bill Shirley: > What is your mailbox_command in main.cf? I just use: > mailbox_command = /usr/bin/spamc -u "$USER" -e > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" > > I don't need anything in master.cf. But you should be using -u ${user} > for spamc. long time ago i tested this with dovecot lda postfix master.cf with a total virtual setup dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} but i strongly do not recommand this !!! use spamass-milter, amavis etc with dovecot lmtp as described on many sites Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From carsten.delellis at delellis.net Wed Oct 24 12:48:34 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Wed, 24 Oct 2012 11:48:34 +0200 Subject: [Dovecot] dovecot auth against AD on samba4 Message-ID: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> Hi group I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and I am thinking about installing it because of it's native outlook support capabilities. The ZEG appliance wouldn't be an option for me because I use a virtual server from a provider where I can't install my own vm or even an iso. When I go thru the documentation there is a part with installing OpenChange based on samba4. As far as I understood the OpenChange authentication is against the samba4 AD. Actually there is no support in syncing the AD against an OpenLdap Server and I would have to change the OpenLdap port because the AD is listening on port 389. To change the port wouldn't be a big deal, but what i was thinking about to run the dovecot auth also against the samba 4 AD. I searched around on the internet but didn't find a doc yet how to do that. Does anyone here could provide me with a link or a how-to ? Thanks very much in advance. Regards, Carsten Laun-De Lellis Hauptstrasse 13 D-67705 Trippstadt Phone: +49 6306 992140 Fax: +49 6306 992142 Mobile: +49 151 27530865 email: carsten.delellis at delellis.net From listen at mjh.name Wed Oct 24 14:28:11 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Wed, 24 Oct 2012 13:28:11 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox Message-ID: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Hello all, I have a problem with an incosistent mdbox: Oct 24 10:43:23 two dovecot: imap-login: Login: user=<...>, method=PLAIN, rip=..., lip=..., mpid=4977, TLS Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-24 10:43:23] bytes=115/53726 Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: Inconsistency in map index (467,31960 != 467,36768) Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: rebuilding indexes Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Panic: file mdbox-storage-rebuild.c: line 773 (rebuild_update_refcounts): assertion failed: (map_uid < msgs[i]->map_uid) Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x380ca) [0x7f99cf35b0ca] -> /usr/lib/dovecot/libdovecot.so.0(+0x3810e) [0x7f99cf35b10e] -> /usr/li b/dovecot/libdovecot.so.0(i_fatal+0) [0x7f99cf334a67] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x10a5) [0x7f99cf5f42d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_s torage_rebuild+0x24) [0x7f99cf5f4414] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x53565) [0x7f99cf5f4565] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7f99cf5c8caa] -> /usr/lib/dovec ot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7f99cf5c8cf3] -> dovecot/imap(client_destroy+0x109) [0x7f99cfaa69e9] -> dovecot/imap(client_input+0xaa) [0x7f99cfaa6dba] -> /usr/lib/dovecot/libdovecot.so.0(i o_loop_call_io+0x48) [0x7f99cf366c98] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f99cf367d27] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f99cf366c28] -> /usr/lib/dovecot /libdovecot.so.0(master_service_run+0x13) [0x7f99cf354e33] -> dovecot/imap(main+0x304) [0x7f99cfa9e554] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f99cef8576d] -> dovecot/imap(+0x95e5) [0 x7f99cfa9e5e5] Oct 24 10:45:19 two dovecot: master: Error: service(imap): child 4977 killed with signal 6 (core dumps disabled) I use: Dovecot 2.0.19-0ubuntu1 Ubuntu 12.04, x86-64, Kernel 3.2.0-32-generic local XFS filesystem for the mdbox The problem appeared out of nowhere. Many messages been continously delivered to this mailbox on this installation since May 2012, and the mdbox was only accessed with deliver and imap/pop3 from dovecot. About four hours after the problem initially appeared, I did a hard reset of the system because it was unresponsive. The error message is exactly the same before and after the hard reset. The problem is triggered by both IMAP access and dovecot deliver access. The whole mdbox is 6.6 GiB large and I guess that it contains about 300k-600k messages. It's an archive of public mailing lists, so I could give access to the files. Can anybody say something about this? May the mdbox be repaired? Regards, Milan Holz?pfel -- Milan Holz?pfel -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doveconf.txt URL: From rs at sys4.de Wed Oct 24 14:43:19 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 13:43:19 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <5087D457.6040205@sys4.de> Am 24.10.2012 13:28, schrieb Milan Holz?pfel: > The whole mdbox is 6.6 GiB large and I guess that it contains about > 300k-600k messages. It's an archive of public mailing lists, so I could > give access to the files. > > Can anybody say something about this? May the mdbox be repaired? perhaps this helps http://wiki2.dovecot.org/Tools/Doveadm/ForceResync however upgrading to dovecot latest might be a good idea Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stan at hardwarefreak.com Wed Oct 24 17:01:24 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 09:01:24 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <5087F4B4.2060107@hardwarefreak.com> On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: > I have a problem with an incosistent mdbox: ... > four hours after the problem initially appeared, I did a hard reset of > the system because it was unresponsive. ... > Can anybody say something about this? May the mdbox be repaired? If the box is truly unresponsive, i.e. hard locked, then the corrupted indexes are only a symptom of the underlying problem, which is unrelated to Dovecot, UNLESS, the lack of responsiveness was due to massive disk access, which will occur when rebuilding indexes on a 6.6GB mailbox. You need to know the difference so we have accurate information to troubleshoot with. If the there's a kernel or hardware problem, you should see related errors in dmesg. Please share those. Neither Timo nor anyone here can fix your index problem if the cause lie elsewhere. You must fix the root problem first. -- Stan From CMarcus at Media-Brokers.com Wed Oct 24 17:45:01 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 24 Oct 2012 10:45:01 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087F4B4.2060107@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> Message-ID: <5087FEED.7060007@Media-Brokers.com> On 2012-10-24 10:01 AM, Stan Hoeppner wrote: > If the box is truly unresponsive, i.e. hard locked, then the corrupted > indexes are only a symptom of the underlying problem, which is unrelated > to Dovecot, UNLESS, the lack of responsiveness was due to massive disk > access, which will occur when rebuilding indexes on a 6.6GB mailbox. > You need to know the difference so we have accurate information to > troubleshoot with. Hmmm... I wonder would it be possible for dovecot to automatically lower the 'niceness' for index rebuilds (on systems that support such) to avoid causing such distress? -- Best regards, Charles From weber at papaya-cms.com Wed Oct 24 17:46:39 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Wed, 24 Oct 2012 16:46:39 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. Message-ID: <5087FF4F.8050103@papaya-cms.com> Hi, I've got some trouble here.. i created some sieve rules, but the debug log says that there is a invalid mailbox name error: msgid=<*>: failed to store into mailbox '/home/shared/.automail.Bugtracker/': Invalid mailbox name. ~/.dovecot.sieve if address :is "to" "mantis-admin@<*>" { fileinto "/home/shared/.automail.Bugtracker/"; } here's my doveconf -n output # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-32-generic x86_64 Ubuntu 12.04.1 LTS base_dir = /var/run/dovecot/ disable_plaintext_auth = no hostname = * mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = yes location = maildir:/home/shared:CONTROL=~/.Maildir/control/Shared:INDEX=~/.Maildir/index/Shared prefix = shared/ separator = / subscriptions = yes type = public } namespace { inbox = yes location = prefix = separator = / type = private } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@* protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } ssl_ca = was automatically rejected:%n%r } protocol imap { mail_plugins = " zlib, acl, imap_zlib" } any idea? Mit freundlichen Gruessen / best regards papaya Software GmbH i.A. Alexander Weber -- papaya Software GmbH | Im MediaPark 5 | 50670 Koeln | Germany Tel./Ph.: +49-221-5743-8070 | Fax: +49-221-5743-8099 mailto:weber at papaya-cms.com | http://www.papaya-cms.com/ -- Geschaeftsfuehrer: Andreas Jacobi, Andr? Schnitzler, Daniel Sch?fer Sitz& Registergericht: Koeln | HRB 60030 | USt.-Id.-Nr.: DE 255642963 -- From sandro.tosi at dada.eu Wed Oct 24 17:48:44 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Wed, 24 Oct 2012 16:48:44 +0200 Subject: [Dovecot] Clarifications on Pigeonhole and MySQL lookups In-Reply-To: <50808A57.8040201@rename-it.nl> References: <50753E85.5060904@dada.eu> <50772D89.4050601@rename-it.nl> <507BBE00.9010007@dada.eu> <50808A57.8040201@rename-it.nl> Message-ID: <5087FFCC.5080504@dada.eu> Hello Stephan, sorry for this late reply. On 10/19/2012 01:01 AM, Stephan Bosch wrote: > On 10/15/2012 9:40 AM, Sandro Tosi wrote: >> Hi Stephan, >> thanks a lot for your reply. >> >> On 10/11/2012 10:35 PM, Stephan Bosch wrote: >>> On 10/10/2012 11:23 AM, Sandro Tosi wrote: >>>> Hello, >>>> we're scouting if it's possible to use Pigeonhole (currently v0.3.1, >>>> as this will be provided with an upcoming Debian package) with MySQL >>>> dict lookups with the mail setup we're designing. >>>> >>>> Our (main) goals are: >>>> >>>> 1. store the filters on the database >>> That is possible with some limitations. >> >> Are the ones below the only limitatios (ie one script per user) or are >> there any other worth knowing? > > You cannot currently use ManageSieve when the active script is located > in a dict database. > > And 'one script per user' is not an fully accurate description. It is > technically possible to access multiple different scripts from the dict > database. It is however not possible to use dict support combination > with multiscript support ( > http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Executing_Multiple_Scripts_Sequentially) > to execute multiple scripts in a sequence. Multiscript currently only > works for Sieve scripts that are located in the filesystem. > >> In our situation, what would you suggest? We're now thinking of >> keeping the scripts list on a separate table, and merge the "user >> selected ones" in a single script to write in the filters table. Is >> that what would you suggest? Is there a better solution? > > You can use the include extension > (https://tools.ietf.org/html/draft-ietf-sieve-include-05) to access > scripts in a dict database from a main active script to combine them. I > believe you could even dynamically construct that main script in SQL > using some string manipulation in the query, but that is a bit ugly. > > Could you send me an overview of your configuration, including your > database layout? Provided that I have some time in the next week, I > could investigate building a simple working configuration for the sake > of example. I will follow this up privately (you know, we can't disclose too much) but JFTR we decided to follow a half-and-half solution: - we keep on the backend database all the scripts the customer could activate in separate rows - from them, we merge into a single sieve script file all the filter the customer has decided to activate. This way we still record the script separately in the db, so once we'll be able to feed pigeonhole with multiple lines, it's already there, and then merging into a single file is the most straightforward and simple solution to make what we need to work. Thanks for the support, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From Bill at KnoxvilleChristian.org Wed Oct 24 18:47:07 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 11:47:07 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50878BB6.2090309@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> Message-ID: <50880D7B.4090407@KnoxvilleChristian.org> On 10/24/2012 2:33 AM, Robert Schetterer wrote: > Am 24.10.2012 03:32, schrieb Bill Shirley: >> What is your mailbox_command in main.cf? I just use: >> mailbox_command = /usr/bin/spamc -u "$USER" -e >> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >> >> I don't need anything in master.cf. But you should be using -u ${user} >> for spamc. > long time ago i tested this with dovecot lda postfix master.cf > with a total virtual setup > > dovecot unix - n n - - pipe > flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e > /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} > > but i strongly do not recommand this !!! > > use spamass-milter, amavis etc with dovecot lmtp > as described on many sites > > > Best Regards > MfG Robert Schetterer > Can you get per-user Spamassassin configs this way? Why user=vmail:vmail? Is this for virtual domains? I didn't think we were talking about them. Instead of strongly recommending against this, why not elaborate on the problems with using spamc in the mailbox_command? Bill From bob at computerisms.ca Wed Oct 24 19:04:39 2012 From: bob at computerisms.ca (Bob Miller) Date: Wed, 24 Oct 2012 09:04:39 -0700 Subject: [Dovecot] dovecot auth against AD on samba4 In-Reply-To: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> References: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> Message-ID: <1351094679.2143.474.camel@worklian> I don't have it in production yet because there are other things I am still trying to add to samba4, but my test server has dovecot authenticating against samba4. Without openchange or any other non-native mechanism. Dovecot supports authenticating against ldap, the settings are in your auth-ldap.conf file. Samba4/Active Directory is just another ldap implementation. between the config files and the wiki, I believe all the documentation you need is there... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Wed, 2012-10-24 at 11:48 +0200, Carsten Laun-De Lellis wrote: > Hi group > > I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, > dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and > I am thinking about installing it because of it's native outlook support > capabilities. > > The ZEG appliance wouldn't be an option for me because I use a virtual > server from a provider where I can't install my own vm or even an iso. > > When I go thru the documentation there is a part with installing > OpenChange based on samba4. As far as I understood the OpenChange > authentication is against the samba4 AD. Actually there is no support in > syncing the AD against an OpenLdap Server and I would have to change the > OpenLdap port because the AD is listening on port 389. To change the port > wouldn't be a big deal, but what i was thinking about to run the dovecot > auth also against the samba 4 AD. > > I searched around on the internet but didn't find a doc yet how to do that. > > Does anyone here could provide me with a link or a how-to ? > > Thanks very much in advance. > > Regards, > > Carsten Laun-De Lellis > > Hauptstrasse 13 > D-67705 Trippstadt > > Phone: +49 6306 992140 > Fax: +49 6306 992142 > Mobile: +49 151 27530865 > email: carsten.delellis at delellis.net > > > From rs at sys4.de Wed Oct 24 19:09:12 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 18:09:12 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50880D7B.4090407@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> Message-ID: <508812A8.8000603@sys4.de> Am 24.10.2012 17:47, schrieb Bill Shirley: > > On 10/24/2012 2:33 AM, Robert Schetterer wrote: >> Am 24.10.2012 03:32, schrieb Bill Shirley: >>> What is your mailbox_command in main.cf? I just use: >>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >>> >>> I don't need anything in master.cf. But you should be using -u ${user} >>> for spamc. >> long time ago i tested this with dovecot lda postfix master.cf >> with a total virtual setup >> >> dovecot unix - n n - - pipe >> flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e >> /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} >> >> but i strongly do not recommand this !!! >> >> use spamass-milter, amavis etc with dovecot lmtp >> as described on many sites >> >> >> Best Regards >> MfG Robert Schetterer >> > > Can you get per-user Spamassassin configs this way? > > Why user=vmail:vmail? Is this for virtual domains? I didn't think we > were talking about them. > > Instead of strongly recommending against this, why not elaborate on the > problems with using spamc in the mailbox_command? > > Bill > Hi Bill, you missed my "i tested this with dovecot lda" in hope you may adapt the syntax to your needs by your own here are the recommanded setups http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix http://wiki.dovecot.org/LDA/Postfix --snip mailbox_command = /usr/bin/spamc -e /usr/lib/dovecot/deliver --snipend by the way using dovecot lmtp and i.e amavis or spamass-milter/clamav-milter might be better choice in many ways Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From troy at troyvit.com Wed Oct 24 19:10:38 2012 From: troy at troyvit.com (Troy Vitullo) Date: Wed, 24 Oct 2012 10:10:38 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5087454B.5030704@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> Message-ID: <20121024101038.5f3316f2@hrafn> On Tue, 23 Oct 2012 21:32:59 -0400 Bill Shirley wrote: > On 10/23/2012 9:06 PM, Bill Shirley wrote: > > > > > > What is your mailbox_command in main.cf? I just use: > > mailbox_command = /usr/bin/spamc -u "$USER" -e > > /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m > > "$EXTENSION" > > > > I don't need anything in master.cf. But you should be using -u > > ${user} for spamc. > > > > Bill > > > Forgot to ask, are you using Spamassassin's per-user configs? If > you're not, that probably is your problem. It's probably trying to > update bayes tokens and it doesn't have permission. > > I use per-user configs which are nice. One man's spam is another > man's ham. Plus each user can have his/her own whitelist. > > I use these spamd args: -d -c -m10 --user-config > You usually can find the args in /etc/sysconfig. > > Bill Thanks for getting back to me Bill. Actually I'm using per-user prefs and permissions look great all the way down. When I send a test mail with everything turned on the bayes tokens are updated. Things appear to die later in the process. Regarding the mailbox command I was using: mailbox_command = /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" I tried removing the flags from master.cf and changing my command to: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" and then: mailbox_command = /usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} -m "$EXTENSION" and everything in between. No mail made it through, so I kept this in master.cf: dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/lib/dovecot/deliver -d ${recipient} and of course it over-rode my mailbox_command. Mail came thrrough but it contained no spamassassin header. I'm starting to thing that spamc doesn't have the permissions to write its headers to the message. How can I test that theory? spamd runs witht these flags: /usr/sbin/spamd --create-prefs -x --max-children 3 --username spamd --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid It's pretty much the same as yours, I just use the long versions of the args. the spamd user exists: spamd:x:1010:1011::/var/lib/spamassassin:/bin/false I was missing /etc/dovecot/default.sieve, which had to be a big problem, but I recovered it. Here's are its contents: require "fileinto"; if exists "X-Spam-Flag" { if header :contains "X-Spam-Flag" "NO" { } else { discard; stop; } } Anything else I could be missing? I even insanely running spamd as the root user: /usr/sbin/spamd --create-prefs -x --max-children 3 --username root --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid Thanks, Troy From rs at sys4.de Wed Oct 24 19:16:43 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 18:16:43 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087FEED.7060007@Media-Brokers.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> Message-ID: <5088146B.606@sys4.de> Am 24.10.2012 16:45, schrieb Charles Marcus: > On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Hmmm... I wonder would it be possible for dovecot to automatically lower > the 'niceness' for index rebuilds (on systems that support such) to > avoid causing such distress? > i think you missed Stans point ,looking for some hardware problems first Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From Bill at KnoxvilleChristian.org Wed Oct 24 19:28:48 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 12:28:48 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508812A8.8000603@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> Message-ID: <50881740.90207@KnoxvilleChristian.org> On 10/24/2012 12:09 PM, Robert Schetterer wrote: > Am 24.10.2012 17:47, schrieb Bill Shirley: >> On 10/24/2012 2:33 AM, Robert Schetterer wrote: >>> Am 24.10.2012 03:32, schrieb Bill Shirley: >>>> What is your mailbox_command in main.cf? I just use: >>>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" >>>> >>>> I don't need anything in master.cf. But you should be using -u ${user} >>>> for spamc. >>> long time ago i tested this with dovecot lda postfix master.cf >>> with a total virtual setup >>> >>> dovecot unix - n n - - pipe >>> flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e >>> /usr/lib/dovecot/deliver -f ${sender} -d ${recipient} >>> >>> but i strongly do not recommand this !!! >>> >>> use spamass-milter, amavis etc with dovecot lmtp >>> as described on many sites >>> >>> >>> Best Regards >>> MfG Robert Schetterer >>> >> Can you get per-user Spamassassin configs this way? >> >> Why user=vmail:vmail? Is this for virtual domains? I didn't think we >> were talking about them. >> >> Instead of strongly recommending against this, why not elaborate on the >> problems with using spamc in the mailbox_command? >> >> Bill >> > Hi Bill, you missed > > my > > "i tested this with dovecot lda" > in hope you may adapt the syntax to your needs by your own > > here are the recommanded setups > > http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix > http://wiki.dovecot.org/LDA/Postfix > > --snip > mailbox_command = /usr/bin/spamc -e /usr/lib/dovecot/deliver > --snipend > > by the way using dovecot lmtp and i.e amavis or spamass-milter/clamav-milter > > might be better choice in many ways > > > Best Regards > MfG Robert Schetterer > I'm saying I have a WORKING setup (local and virtual) where spamc runs and then uses dovecot deliver. spamd uses spamassassin per-user configs. master.cf has (caution, line wraps around in email): vdovecot unix - n n - 5 pipe flags=DRuh user=vmail:vmail argv=/usr/bin/spamc -p 784 -u ${recipient} -e /usr/lib64/dovecot/deliver -d ${user}@${domain} -a {recipient} -f ${sender} -n -m ${extension} main.cf has: mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" virtual_transport = vdovecot vdovecot_destination_recipient_limit = 1 I don't understand why you strongly recommend against using the mailbox_command. Is there a security risk here? I've read all the howtos. There are many ways to setup a mail server. That's the beauty of postfix, spamassassin, dovecot, etc; you can make it do what you want. Yes, some setups are bad. I am not the original poster. Hope this clears things up, Bill From rob0 at gmx.co.uk Wed Oct 24 19:32:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 11:32:55 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121023145245.124dd362@hrafn> References: <20121023145245.124dd362@hrafn> Message-ID: <20121024163255.GI3672@harrier.slackbuilds.org> There seems to be much confusion in this thread. I might be able to help clear up some of it, but probably not all, because I agree with Robert about using amavisd-new for filtering and LMTP for delivery. On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > My server uses a system comprised of postfix, dovecot and dspam to > filter and deliver mail. > > Postfix used the following flags in calling spamc and dovecot: > > flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} > -e /usr/lib/dovecot/deliver -d ${recipient} This looks like you might be using pipe(8). If so, refer to the manual, and note that you are invoking this command as user "dovecot" and group "secmail". That is wrong use of the "dovecot" user. You probably should have made and used a dedicated "vmail" user. And according to your own post, q.v., the group "secmail" is definitely wrong. > after an upgrade from Debian lenny to squeeze we were able to get > everything working except spam filtering. Spamassassin is able to > judge whether the mail coming in is spam but everything stops > there. Automated or semi-automated upgrades are often a source of pain. > In mail.err I see: > > pamc[3608]: exec failed: Permission denied I guess that is spamc, and yes, of course. > spamc shows the same thing in syslog: > > exec failed: Permission denied > > postfix delays the email: > > postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > (system resource problem) > > Here are the permissions for deliver: > > -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver The pipe command is not executed as root. Nor is it invoked with the GID "dovecot". You specified group "secmail". Therefore the "other" permissions are what apply. "---" is no read, no write, no execute. > Here are the relevant groups: > > s1:~# grep dovecot /etc/group > secmail:x:119:postfix,spamd,dovecot This is not relevant. The process has EGID secmail, and the fact that dovecot is a member of secmail does not matter. Bottom line here: it seems that you misunderstood what the group permissions meant. > dovecot:x:111: > > here's the dovecot user: > s1:~# grep dovecot /etc/passwd > dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false > > here's dovecot -n: > > # 1.2.15: /etc/dovecot/dovecot.conf You upgraded -- to 1.2.15? Why? snip > Many thanks in advance for any advice you can give. Again, you should check on the wiki about the appropriate use of the "dovecot" user, and also read the wiki about virtual mailboxes. Fix that. Even if you make it work with permissions, you are breaking Dovecot's security model of privilege separation. The "dovecot" user is for Dovecot's internal use only, not for delivering mail and ownership of mailboxes. The poster who was talking about postconf(5) mailbox_command was bringing in a red herring. That is for local(8) delivery, and you evidently are using pipe(8). -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rob0 at gmx.co.uk Wed Oct 24 19:44:48 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 11:44:48 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50881740.90207@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> Message-ID: <20121024164448.GJ3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 12:28:48PM -0400, Bill Shirley wrote: > I don't understand why you strongly recommend against using the > mailbox_command. Is there a security risk here? One issue is that mailbox_command is only used for local(8) delivery. You brought that up for the OP, who is reporting a problem in trying to use pipe(8). mailbox_command is not relevant for pipe. That added more confusion to the issue at hand. I can't speak for Robert, but as I said in the other post I agree with him, so I will say why. You will get better overall performance with amavisd-new and LMTP, rather than invoking a command via pipe for every delivery. No, mailbox_command in itself is not a security risk, except insofar as you could DoS yourself with more deliveries at once than the system is able to handle. Some risk of DoS is present for any kind of content filtering, though. But amavisd-new after-queue reduces that risk. > I've read all the howtos. Eww. I have not. I have made extensive referral to the documentation, however, and that is what I recommend. Many thousands of people who are generating web content do not know much about email. You don't want to turn to them for advice about this! (FWIW, many of the howtos I have looked at are very bad.) > There are many ways to setup a mail server. That's the beauty of > postfix, spamassassin, dovecot, etc; you can make it do what you > want. Yes, some setups are bad. Yes and yes. > I am not the original poster. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From Bill at KnoxvilleChristian.org Wed Oct 24 20:13:42 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:13:42 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024101038.5f3316f2@hrafn> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <20121024101038.5f3316f2@hrafn> Message-ID: <508821C6.4010608@KnoxvilleChristian.org> On 10/24/2012 12:10 PM, Troy Vitullo wrote: > On Tue, 23 Oct 2012 21:32:59 -0400 > Bill Shirley wrote: > >> On 10/23/2012 9:06 PM, Bill Shirley wrote: >>> >>> What is your mailbox_command in main.cf? I just use: >>> mailbox_command = /usr/bin/spamc -u "$USER" -e >>> /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m >>> "$EXTENSION" >>> >>> I don't need anything in master.cf. But you should be using -u >>> ${user} for spamc. >>> >>> Bill >>> >> Forgot to ask, are you using Spamassassin's per-user configs? If >> you're not, that probably is your problem. It's probably trying to >> update bayes tokens and it doesn't have permission. >> >> I use per-user configs which are nice. One man's spam is another >> man's ham. Plus each user can have his/her own whitelist. >> >> I use these spamd args: -d -c -m10 --user-config >> You usually can find the args in /etc/sysconfig. >> >> Bill > Thanks for getting back to me Bill. Actually I'm using per-user prefs and permissions look great all the way down. When I send a test mail with everything turned on the bayes tokens are updated. Things appear to die later in the process. > > Regarding the mailbox command I was using: > mailbox_command = /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" > > I tried removing the flags from master.cf and changing my command to: > mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib/dovecot/deliver -d "$USER" -m "$EXTENSION" What was your setting for mailbox_transport (in main.cf) when you did this? mailbox_transport could be overriding mailbox_command. > > and then: > mailbox_command = /usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient} -m "$EXTENSION" > > and everything in between. > > No mail made it through, so I kept this in master.cf: > > dovecot unix - n n - - pipe > flags=DRhu user=dovecot:dovecot argv=/usr/lib/dovecot/deliver -d ${recipient} Where are you calling spamc with this? > > and of course it over-rode my mailbox_command. Mail came thrrough but it contained no spamassassin header. > > I'm starting to thing that spamc doesn't have the permissions to write its headers to the message. How can I test that theory? > > spamd runs witht these flags: > /usr/sbin/spamd --create-prefs -x --max-children 3 --username spamd --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid > > It's pretty much the same as yours, I just use the long versions of the args. > > the spamd user exists: > spamd:x:1010:1011::/var/lib/spamassassin:/bin/false Your permissions on /var/lib/spamassassin are probably right, but check them and the subdirectories. > > I was missing /etc/dovecot/default.sieve, which had to be a big problem, but I recovered it. Here's are its contents: > > require "fileinto"; > if exists "X-Spam-Flag" { > if header :contains "X-Spam-Flag" "NO" { > } else { > discard; > stop; > } > } > > Anything else I could be missing? I even insanely running spamd as the root user: > > /usr/sbin/spamd --create-prefs -x --max-children 3 --username root --helper-home-dir /var/lib/spamassassin -s /var/lib/spamassassin/spamd.log --virtual-config-dir=/var/lib/spamassassin/users/%d/%l -d --pidfile=/var/run/spamd.pid > > Thanks, > > Troy I have two instances of spamd running. One for local users and the other for virtual users (note the port here and in master.cf): [root at elmo includes]# ps aux | grep spamd root 2684 0.1 1.0 173760 88484 ? SN 03:30 0:34 spamd child root 23987 0.0 0.7 147524 61900 ? SNs Oct23 0:05 /usr/bin/spamd -d -c -m10 --user-config root 24004 0.0 0.7 147504 61844 ? SNs Oct23 0:05 /usr/bin/spamd -d -c -m5 -x --virtual-config-dir=/home/vmail/domains/%d/%l/.spamassassin -u vmail --port=784 -H vmail 24014 0.0 0.9 161204 75880 ? SN Oct23 0:05 spamd child vmail 24015 0.0 0.7 147504 59700 ? SN Oct23 0:00 spamd child root 25772 0.0 0.8 155020 69188 ? SN 12:07 0:00 spamd child root 28981 0.0 0.0 16688 940 pts/4 S+ 12:36 0:00 grep --color spamd My vmail user: [root at elmo includes]# grep vmail /etc/{group,passwd} /etc/group:vmail:x:399: /etc/passwd:vmail:x:399:399:Virtual Mail:/home/vmail:/bin/bash My virtual user .spamassassin permissions: [root at elmo includes]# ldp /home/vmail/domains/example.com/bill/.spamassassin drwxr-xr-x 20 root root 4096 May 8 2011 /home drwxr-xr-x 10 vmail vmail 4096 Oct 22 10:59 /home/vmail drwxr-x--- 9 vmail vmail 4096 Oct 21 21:24 /home/vmail/domains drwxr-x--- 6 vmail vmail 4096 Jul 4 2007 /home/vmail/domains/example.com drwxr-x--- 4 vmail vmail 4096 Jul 4 2007 /home/vmail/domains/example.com/bill drwxr-s--- 3 vmail vmail 4096 Jan 30 2012 /home/vmail/domains/example.com/bill/.spamassassin My local user: [root at elmo includes]# ldp /home/bill/.spamassassin drwxr-xr-x 20 root root 4096 May 8 2011 /home drwxr-xr-x 32 bill bill 4096 Oct 22 17:42 /home/bill drwxr-s--- 2 bill bill 4096 Oct 24 12:42 /home/bill/.spamassassin My main.cf: mailbox_transport = mailbox_command = /usr/bin/spamc -u "$USER" -e /usr/lib64/dovecot/deliver -a "$RECIPIENT" -f "$SENDER" -m "$EXTENSION" virtual_transport = vdovecot vdovecot_destination_recipient_limit = 1 My master.cf: vdovecot unix - n n - 5 pipe flags=DRuh user=vmail:vmail argv=/usr/bin/spamc -p 784 -u ${recipient} -e /usr/lib64/dovecot/deliver -d ${user}@${domain} -a {recipient} -f ${sender} -n -m ${extension} You could try my config substituting your user and directory for mine: I'm using user=vmail:vmail and --virtual-config-dir=/home/vmail/domains/%d/%l/.spamassassin You're using user=dovecot:secmail and --virtual-config-dir=/var/lib/spamassassin/users/%d/%l Currently, your user=dovecot:secmail should probably be user=spamd:spamd in master.cf unless group secmail has write permissions on /var/lib/spamassassin and subdirectories. Hope this helps, Bill From Bill at KnoxvilleChristian.org Wed Oct 24 20:21:58 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:21:58 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024164448.GJ3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> Message-ID: <508823B6.3040208@KnoxvilleChristian.org> On 10/24/2012 12:44 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 12:28:48PM -0400, Bill Shirley wrote: >> I don't understand why you strongly recommend against using the >> mailbox_command. Is there a security risk here? > One issue is that mailbox_command is only used for local(8) delivery. > You brought that up for the OP, who is reporting a problem in trying > to use pipe(8). mailbox_command is not relevant for pipe. That added > more confusion to the issue at hand. It was my understanding that he is implementing local users. > > I can't speak for Robert, but as I said in the other post I agree > with him, so I will say why. You will get better overall performance > with amavisd-new and LMTP, rather than invoking a command via pipe > for every delivery. Admittedly, I have not used amavisd-new or LMTP; they may be better. But will they allow spamassassin per-user prefs? Performance is a plus; another daemon is not. That saying, I'll run another daemon if I get something out of it. Any benchmarks on this? > > No, mailbox_command in itself is not a security risk, except insofar > as you could DoS yourself with more deliveries at once than the > system is able to handle. Some risk of DoS is present for any kind of > content filtering, though. But amavisd-new after-queue reduces that > risk. > >> I've read all the howtos. > Eww. I have not. I have made extensive referral to the documentation, > however, and that is what I recommend. Many thousands of people who > are generating web content do not know much about email. You don't > want to turn to them for advice about this! Probably mis-spoke; I said howtos instead of documentation. Yes, there are many bad howtos out there. > > (FWIW, many of the howtos I have looked at are very bad.) > >> There are many ways to setup a mail server. That's the beauty of >> postfix, spamassassin, dovecot, etc; you can make it do what you >> want. Yes, some setups are bad. > Yes and yes. > >> I am not the original poster. Respectfully, Bill From Bill at KnoxvilleChristian.org Wed Oct 24 20:28:41 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:28:41 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024163255.GI3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> Message-ID: <50882549.3020505@KnoxvilleChristian.org> On 10/24/2012 12:32 PM, /dev/rob0 wrote: > There seems to be much confusion in this thread. I might be able to > help clear up some of it, but probably not all, because I agree with > Robert about using amavisd-new for filtering and LMTP for delivery. > > On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: >> My server uses a system comprised of postfix, dovecot and dspam to >> filter and deliver mail. >> >> Postfix used the following flags in calling spamc and dovecot: >> >> flags=DRhu user=dovecot:secmail argv=/usr/bin/spamc -u ${recipient} >> -e /usr/lib/dovecot/deliver -d ${recipient} > This looks like you might be using pipe(8). If so, refer to the > manual, and note that you are invoking this command as user "dovecot" > and group "secmail". > > That is wrong use of the "dovecot" user. You probably should have > made and used a dedicated "vmail" user. And according to your own > post, q.v., the group "secmail" is definitely wrong. > >> after an upgrade from Debian lenny to squeeze we were able to get >> everything working except spam filtering. Spamassassin is able to >> judge whether the mail coming in is spam but everything stops >> there. > Automated or semi-automated upgrades are often a source of pain. > >> In mail.err I see: >> >> pamc[3608]: exec failed: Permission denied > I guess that is spamc, and yes, of course. > >> spamc shows the same thing in syslog: >> >> exec failed: Permission denied >> >> postfix delays the email: >> >> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >> (system resource problem) >> >> Here are the permissions for deliver: >> >> -rwsr-x--- 1 root dovecot 865084 May 25 2011 /usr/lib/dovecot/deliver > The pipe command is not executed as root. Nor is it invoked with the > GID "dovecot". You specified group "secmail". Therefore the "other" > permissions are what apply. "---" is no read, no write, no execute. > >> Here are the relevant groups: >> >> s1:~# grep dovecot /etc/group >> secmail:x:119:postfix,spamd,dovecot > This is not relevant. The process has EGID secmail, and the fact that > dovecot is a member of secmail does not matter. Bottom line here: it > seems that you misunderstood what the group permissions meant. > >> dovecot:x:111: >> >> here's the dovecot user: >> s1:~# grep dovecot /etc/passwd >> dovecot:x:108:111:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false >> >> here's dovecot -n: >> >> # 1.2.15: /etc/dovecot/dovecot.conf > You upgraded -- to 1.2.15? Why? > > snip >> Many thanks in advance for any advice you can give. > Again, you should check on the wiki about the appropriate use of the > "dovecot" user, and also read the wiki about virtual mailboxes. Fix > that. Even if you make it work with permissions, you are breaking > Dovecot's security model of privilege separation. The "dovecot" user > is for Dovecot's internal use only, not for delivering mail and > ownership of mailboxes. > > The poster who was talking about postconf(5) mailbox_command was > bringing in a red herring. That is for local(8) delivery, and you > evidently are using pipe(8). Just a note: the original post did NOT have the word 'virtual' in it. If it did, I missed it and apologize for introducing confusion. Bill From rs at sys4.de Wed Oct 24 20:37:35 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 19:37:35 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50881740.90207@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> Message-ID: <5088275F.1030507@sys4.de> Am 24.10.2012 18:28, schrieb Bill Shirley: > I don't understand why you strongly recommend against using the > mailbox_command. Is there a security risk here? no ,until you dont have made any setup failures... your right there are tons of working possible setups your free to configure as you like, but lmtp with dovecot is state of the art in my eyes, these days in my tests lda combined with spamc had not enough performance for my needs and used to much resources compared to lmtp sometimes it crashed, but as i said ,long time ago however i found total virtual setups much more easy then with local by permissions stuff etc, and milters are much more easy to use and setup, also i.e amavis gives great other choices beside spamassassin stuff but do as you like ,no need to flame Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From rob0 at gmx.co.uk Wed Oct 24 20:39:18 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 12:39:18 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50882549.3020505@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> Message-ID: <20121024173918.GK3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: > On 10/24/2012 12:32 PM, /dev/rob0 wrote: > >There seems to be much confusion in this thread. I might be able > >able to help clear up some of it, but probably not all, because I > >agree with Robert about using amavisd-new for filtering and LMTP > >for delivery. > > > >On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: snip > >>postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > >>delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > >>(system resource problem) > >The poster who was talking about postconf(5) mailbox_command > >was bringing in a red herring. That is for local(8) delivery, > >and you evidently are using pipe(8). > Just a note: the original post did NOT have the word 'virtual' in > it. If it did, I missed it and apologize for introducing confusion. It did not, but it did indeed include the pipe log output shown above, and therefore ^mailbox_.* postconf settings do not apply. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From jeff at bubble.org Wed Oct 24 20:40:25 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Wed, 24 Oct 2012 13:40:25 -0400 Subject: [Dovecot] Snarf plugin Message-ID: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> I've now upgraded dovecot from 2.0.21 to 2.1.10 and the good news is I no longer see dovecot crashing when loading the snarf plugin however snarf still does not do anything except make the inbox disappear. I've come to the conclusion that either snarf does not actually work, possible, but I doubt it, or more likely I have a configuration issue preventing it from working. The system is simple, all email is stored in /var/spool/mail/{username} and I want all the mail moved to ~/mbox when the user logs in via imap, similar to uw-imap. Any guidance would really be appreciated. Thanks, Jeff dovecot -n # 2.1.10: //etc/dovecot/dovecot.conf # OS: Linux 3.6.2-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mbox mail_plugins = snarf zlib mbox_write_locks = fcntl namespace default { inbox = yes location = prefix = separator = / } namespace snarf { hidden = yes list = no location = mbox:/run/dovecot/empty:INBOX=/var/spool/mail/%u:INDEX=MEMORY prefix = /snarf separator = / } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mbox_snarf = ~/mbox snarf = /snarf/INBOX } service imap-login { inet_listener imap { address = localhost } } service pop3-login { inet_listener pop3 { address = localhost } } ssl_cert = , method=PLAIN, rip=::1, lip=::1, mpid=28089, secured, session= Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Loading modules from directory: /usr/lib64/dovecot Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib05_snarf_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap: Debug: Module loaded: /usr/lib64/dovecot/lib30_imap_zlib_plugin.so Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Effective uid=500, gid=500, home=/home/jeff Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Namespace default: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=~/mbox Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: fs: root=/home/jeff/mail, index=, control=, inbox=/home/jeff/mbox, alt= Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: Namespace snarf: type=private, prefix=/snarf, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=mbox:/run/dovecot/empty:INBOX=/var/spool/mail/jeff:INDEX=MEMORY Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Debug: fs: root=/run/dovecot/empty, index=, control=, inbox=/var/spool/mail/jeff, alt= Oct 24 13:33:29 xyzzy dovecot: imap(jeff): Disconnected: Logged out in=117 out=1504 From rob0 at gmx.co.uk Wed Oct 24 20:49:03 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 12:49:03 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508823B6.3040208@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> Message-ID: <20121024174903.GL3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 01:21:58PM -0400, Bill Shirley wrote: > On 10/24/2012 12:44 PM, /dev/rob0 wrote: > >I can't speak for Robert, but as I said in the other post I > >agree with him, so I will say why. You will get better overall > >performance with amavisd-new and LMTP, rather than invoking a > >command via pipe for every delivery. > Admittedly, I have not used amavisd-new or LMTP; they may be > better. But will they allow spamassassin per-user prefs? Amavisd-new is indeed capable of per-user preferences. > Performance is a plus; another daemon is not. That saying, I'll > run another daemon if I get something out of it. Any benchmarks > on this? A daemon is generally (I'd almost daresay "always") less overhead than the invocation of many single-delivery processes. No benchmarking is needed to support this fact. That said, for many small sites, it does not matter much. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From Bill at KnoxvilleChristian.org Wed Oct 24 20:56:18 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 13:56:18 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5088275F.1030507@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <5088275F.1030507@sys4.de> Message-ID: <50882BC2.2010702@KnoxvilleChristian.org> On 10/24/2012 1:37 PM, Robert Schetterer wrote: > Am 24.10.2012 18:28, schrieb Bill Shirley: >> I don't understand why you strongly recommend against using the >> mailbox_command. Is there a security risk here? > no ,until you dont have made any setup failures... > > your right there are tons of working possible setups > your free to configure as you like, but lmtp with dovecot is state of > the art in my eyes, these days > > in my tests lda combined with spamc had not enough > performance for my needs and used to much resources compared to lmtp > sometimes it crashed, but as i said ,long time ago > > however i found total virtual setups much more easy then with local > by permissions stuff etc, and milters are much more easy to use and > setup, also i.e amavis gives great other choices beside spamassassin stuff > > but do as you like ,no need to flame > > Best Regards > MfG Robert Schetterer > I don't see a flame anywhere in my posts. The list is for respectfully exchanging information. I thought that was what we were doing. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 21:04:39 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 14:04:39 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024173918.GK3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> Message-ID: <50882DB7.5030202@KnoxvilleChristian.org> On 10/24/2012 1:39 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: >> On 10/24/2012 12:32 PM, /dev/rob0 wrote: >>> There seems to be much confusion in this thread. I might be able >>> able to help clear up some of it, but probably not all, because I >>> agree with Robert about using amavisd-new for filtering and LMTP >>> for delivery. >>> >>> On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > snip >>>> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >>>> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >>>> (system resource problem) >>> The poster who was talking about postconf(5) mailbox_command >>> was bringing in a red herring. That is for local(8) delivery, >>> and you evidently are using pipe(8). >> Just a note: the original post did NOT have the word 'virtual' in >> it. If it did, I missed it and apologize for introducing confusion. > It did not, but it did indeed include the pipe log output shown > above, and therefore ^mailbox_.* postconf settings do not apply. Could be he was going about it the wrong way; mixing the two. Do you know whether he's trying to do virtual or local? My postings describe my implementation. I'm just trying to help him. But I don't think my posts are being received that way. Bill From carsten.delellis at delellis.net Wed Oct 24 21:22:14 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Wed, 24 Oct 2012 20:22:14 +0200 Subject: [Dovecot] dovecot auth against AD on samba4 In-Reply-To: <1351094679.2143.474.camel@worklian> References: <7c0aa73aee741e3d9e9dcb61b4289073.squirrel@mail.delellis.de> <1351094679.2143.474.camel@worklian> Message-ID: <296201cdb214$7ef15e50$7cd41af0$@delellis.net> Dear Bob Thankx for your hint. I tried with jxplorer to connect to the AD ldap and I am pretty sure that I will get it up and running like I did with the openldap server. Carsten -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Bob Miller Sent: Mittwoch, 24. Oktober 2012 18:05 To: dovecot at dovecot.org Subject: Re: [Dovecot] dovecot auth against AD on samba4 I don't have it in production yet because there are other things I am still trying to add to samba4, but my test server has dovecot authenticating against samba4. Without openchange or any other non-native mechanism. Dovecot supports authenticating against ldap, the settings are in your auth-ldap.conf file. Samba4/Active Directory is just another ldap implementation. between the config files and the wiki, I believe all the documentation you need is there... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Wed, 2012-10-24 at 11:48 +0200, Carsten Laun-De Lellis wrote: > Hi group > > I am currently running a mail server on ubuntu 11.04 with postfix 2.8.5, > dovecot dovecot 2.0.13 and openldap 2.4.23. I have now read about sogo and > I am thinking about installing it because of it's native outlook support > capabilities. > > The ZEG appliance wouldn't be an option for me because I use a virtual > server from a provider where I can't install my own vm or even an iso. > > When I go thru the documentation there is a part with installing > OpenChange based on samba4. As far as I understood the OpenChange > authentication is against the samba4 AD. Actually there is no support in > syncing the AD against an OpenLdap Server and I would have to change the > OpenLdap port because the AD is listening on port 389. To change the port > wouldn't be a big deal, but what i was thinking about to run the dovecot > auth also against the samba 4 AD. > > I searched around on the internet but didn't find a doc yet how to do that. > > Does anyone here could provide me with a link or a how-to ? > > Thanks very much in advance. > > Regards, > > Carsten Laun-De Lellis > > Hauptstrasse 13 > D-67705 Trippstadt > > Phone: +49 6306 992140 > Fax: +49 6306 992142 > Mobile: +49 151 27530865 > email: carsten.delellis at delellis.net > > > From rs at sys4.de Wed Oct 24 21:24:31 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 20:24:31 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508823B6.3040208@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> Message-ID: <5088325F.7020102@sys4.de> Am 24.10.2012 19:21, schrieb Bill Shirley: > Admittedly, I have not used amavisd-new or LMTP; they may be better. > But will they allow spamassassin per-user prefs? Performance is a plus; > another daemon is not. That saying, I'll run another daemon if I get > something out of it. Any benchmarks on this? this went away from the orig post, it went to general design of a email system, i think rob did explain the possible problems to the orginal poster very fine some people may start with local users as traditional mailsetup depend on this next steps they are going to use lda perhaps trying combined with spamc with local users so there is nothing bad on it, its somehow old school, after all, as said ,there are many broken advices out in www by all setups, and sometimes there are mixed up by local and virtual, so people may fail with permissions of local users , daemons etc sometimes later if more domains should be hosted pure virtual setups are the better way, and making stuff more simple ( but often people fail first in seeing virtual more easy ), lmtp is the best choice for it compared starting a deliver process for each mail, its working as a service So anyone should think about what he needs before starting to setup i.e amavis is a well supported framework since long time, it has tons of features you might wanna have and as well it can be used with per-user prefs if you dont like the complex amavis style ( many functions have many config points ), you could simple use a chain of milter i.e spamass-milter ( also with per-user prefs ), clamav-milter with milter you are able to reject on smtp income stage which is very cool anyway milters also have their pros an contras, read postfix sites about them i didnt tested dspam looks like it chained between lmtp so perhaps also good choice, and could be combined with milters i had other setups with chained spampd/clamsmtp amavis on seperate filter hosts etc all worked fine but as dovecot/postfix development going forward , i redesigned all these depending to have more functions and performance so i recommand, use your working setups as i.e lifetime of your hardware etc, but if building new mailserver choose modern setup ideas and daemon combinations Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From rob0 at gmx.co.uk Wed Oct 24 21:25:52 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 24 Oct 2012 13:25:52 -0500 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50882DB7.5030202@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> Message-ID: <20121024182552.GM3672@harrier.slackbuilds.org> On Wed, Oct 24, 2012 at 02:04:39PM -0400, Bill Shirley wrote: > On 10/24/2012 1:39 PM, /dev/rob0 wrote: > >On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: > >>On 10/24/2012 12:32 PM, /dev/rob0 wrote: > >>>On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: > >snip > >>>>postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, > >>>>delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred > >>>>(system resource problem) > >>>The poster who was talking about postconf(5) mailbox_command > >>>was bringing in a red herring. That is for local(8) delivery, > >>>and you evidently are using pipe(8). > >>Just a note: the original post did NOT have the word 'virtual' > >>in it. If it did, I missed it and apologize for introducing > >>confusion. > >It did not, but it did indeed include the pipe log output shown > >above, and therefore ^mailbox_.* postconf settings do not apply. > > Could be he was going about it the wrong way; mixing the two. > Do you know whether he's trying to do virtual or local? There are lots of wrong ways. The most wrongful of the OP's ways I found was the misuse of the dovecot user. The second most wrong, which was the actual problem at hand, was a misunderstanding of how group permissions are applied. Mixing virtual and local in Postfix and Dovecot is no problem at all, and in fact multiple modes of delivery are possible, even within a given address class or even within a domain. All we know here is what the OP posted. You don't usually use pipe for delivery to local (Unix) users. > My postings describe my implementation. For the OP to change to local delivery would require reworking his setup extensively, on the Postfix side, and here we are on the Dovecot list, so I wouldn't go into that here. But sure, there are other (and for many purposes, better) means of doing what he might want to do. > I'm just trying to help him. But I don't think my posts are > being received that way. Regarding Robert's "flame" comment in the other subthread, I agree with you; I saw no flame. And I did not suggest that you were not trying to help. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rs at sys4.de Wed Oct 24 21:32:19 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 20:32:19 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024182552.GM3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> Message-ID: <50883433.8010609@sys4.de> Am 24.10.2012 20:25, schrieb /dev/rob0: > Regarding Robert's "flame" comment in the other subthread, I agree > with you; I saw no flame. And I did not suggest that you were not > trying to help take my sorry, as non native english, perhaps i missused "flame" here Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From CMarcus at Media-Brokers.com Wed Oct 24 21:48:57 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 24 Oct 2012 14:48:57 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5088146B.606@sys4.de> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> <5088146B.606@sys4.de> Message-ID: <50883819.7010005@Media-Brokers.com> On 2012-10-24 12:16 PM, Robert Schetterer wrote: > Am 24.10.2012 16:45, schrieb Charles Marcus: >> On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >>> If the box is truly unresponsive, i.e. hard locked, then the corrupted >>> indexes are only a symptom of the underlying problem, which is unrelated >>> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >>> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >>> You need to know the difference so we have accurate information to >>> troubleshoot with. >> Hmmm... I wonder would it be possible for dovecot to automatically lower >> the 'niceness' for index rebuilds (on systems that support such) to >> avoid causing such distress? > i think you missed Stans point ,looking for some hardware problems first No, I was simply commenting on the one point about heavy load during large index rebuilds - which is why I trimmed the quoted text... maybe I could have trimmed more? -- Best regards, Charles From Bill at KnoxvilleChristian.org Wed Oct 24 21:51:12 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 14:51:12 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024182552.GM3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> Message-ID: <508838A0.6090100@KnoxvilleChristian.org> On 10/24/2012 2:25 PM, /dev/rob0 wrote: > On Wed, Oct 24, 2012 at 02:04:39PM -0400, Bill Shirley wrote: >> On 10/24/2012 1:39 PM, /dev/rob0 wrote: >>> On Wed, Oct 24, 2012 at 01:28:41PM -0400, Bill Shirley wrote: >>>> On 10/24/2012 12:32 PM, /dev/rob0 wrote: >>>>> On Tue, Oct 23, 2012 at 02:52:45PM -0600, Troy Vitullo wrote: >>> snip >>>>>> postfix/pipe[3607]: 50DEFF180EE: to=<[mail]>, relay=dovecot, >>>>>> delay=1.7, delays=0.07/0.01/0/1.6, dsn=4.3.0, status=deferred >>>>>> (system resource problem) >>>>> The poster who was talking about postconf(5) mailbox_command >>>>> was bringing in a red herring. That is for local(8) delivery, >>>>> and you evidently are using pipe(8). >>>> Just a note: the original post did NOT have the word 'virtual' >>>> in it. If it did, I missed it and apologize for introducing >>>> confusion. >>> It did not, but it did indeed include the pipe log output shown >>> above, and therefore ^mailbox_.* postconf settings do not apply. >> Could be he was going about it the wrong way; mixing the two. >> Do you know whether he's trying to do virtual or local? > There are lots of wrong ways. The most wrongful of the OP's ways I > found was the misuse of the dovecot user. The second most wrong, > which was the actual problem at hand, was a misunderstanding of how > group permissions are applied. > > Mixing virtual and local in Postfix and Dovecot is no problem at all, > and in fact multiple modes of delivery are possible, even within a > given address class or even within a domain. > > All we know here is what the OP posted. You don't usually use pipe > for delivery to local (Unix) users. > >> My postings describe my implementation. > For the OP to change to local delivery would require reworking his > setup extensively, on the Postfix side, and here we are on the > Dovecot list, so I wouldn't go into that here. But sure, there are > other (and for many purposes, better) means of doing what he might > want to do. > >> I'm just trying to help him. But I don't think my posts are >> being received that way. > Regarding Robert's "flame" comment in the other subthread, I agree > with you; I saw no flame. And I did not suggest that you were not > trying to help. Thank you for saying this. My intent was to help. I make my living setting up/programming with open source software. I don't want to only 'take'. I want to show my gratitude for is so freely given to me by also giving. I don't program in C so I can't help with that. But I can share configurations/experiences and hopefully that is a contribution. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 23:04:20 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 16:04:20 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <5088325F.7020102@sys4.de> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> <5088325F.7020102@sys4.de> Message-ID: <508849C4.9060800@KnoxvilleChristian.org> On 10/24/2012 2:24 PM, Robert Schetterer wrote: > Am 24.10.2012 19:21, schrieb Bill Shirley: >> Admittedly, I have not used amavisd-new or LMTP; they may be better. >> But will they allow spamassassin per-user prefs? Performance is a plus; >> another daemon is not. That saying, I'll run another daemon if I get >> something out of it. Any benchmarks on this? > this went away from the orig post, it went to general design > of a email system, i think rob did explain the possible problems > to the orginal poster very fine > > some people may start with local users as traditional > mailsetup depend on this next steps they are going to use lda > perhaps trying combined with spamc with local users > so there is nothing bad on it, its somehow old school, > after all, as said ,there are many broken advices out in www by all > setups, and sometimes there are mixed up by local and virtual, so people > may fail with permissions of local users , daemons etc > > sometimes later if more domains should be hosted > pure virtual setups are the better way, and making stuff more simple ( > but often people fail first in seeing virtual more easy ), > > lmtp is the best choice for it compared starting a deliver process for > each mail, its working as a service > > So anyone should think about what he needs before starting to setup > > i.e amavis is a well supported framework since long time, it has tons of > features > you might wanna have and as well it can be used with per-user prefs > > if you dont like the complex amavis style ( many functions have many > config points ), you could simple use a chain of milter i.e > spamass-milter ( also with per-user prefs ), clamav-milter > > with milter you are able to reject on smtp income stage > which is very cool > anyway milters also have their pros an contras, read postfix sites about > them > > i didnt tested dspam looks like it chained between lmtp > so perhaps also good choice, and could be combined with milters > > i had other setups with chained spampd/clamsmtp > amavis on seperate filter hosts etc > all worked fine > > but as dovecot/postfix development going forward , i redesigned all > these depending to have more functions and performance > > so i recommand, use your working setups as i.e lifetime of your hardware > etc, but if building new mailserver choose modern setup ideas > and daemon combinations > > > Best Regards > MfG Robert Schetterer > Thank you for a very informative post. I took a quick look at spamass-milter but I can't find any configuration information on how to use spamasssassin's per-user prefs. I thought the only way to support per-user prefs was post queue since you have to know who is getting the email to check their prefs. I am using clamav-milter. Milters are nice. I set my mail server up 15+ years ago, so it's time for me to have a re-think here. At that time there were no milters for postfix (don't remember a Dovecot either). I've try to steer away from re-injects since they affect the mail received numbers. Are we saying Dovecot's LMTP can call spamd? I'm on Dovecot 1.2 at home until I can upgrade. There is no LMTP in Dovecot 1.x, right? I have a few mail servers running Dovecot 2.0 and 2.1 and yes, I want them to perform well. Bill From Bill at KnoxvilleChristian.org Wed Oct 24 23:06:01 2012 From: Bill at KnoxvilleChristian.org (Bill Shirley) Date: Wed, 24 Oct 2012 16:06:01 -0400 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <50883433.8010609@sys4.de> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> <50882549.3020505@KnoxvilleChristian.org> <20121024173918.GK3672@harrier.slackbuilds.org> <50882DB7.5030202@KnoxvilleChristian.org> <20121024182552.GM3672@harrier.slackbuilds.org> <50883433.8010609@sys4.de> Message-ID: <50884A29.1020901@KnoxvilleChristian.org> On 10/24/2012 2:32 PM, Robert Schetterer wrote: > Am 24.10.2012 20:25, schrieb /dev/rob0: >> Regarding Robert's "flame" comment in the other subthread, I agree >> with you; I saw no flame. And I did not suggest that you were not >> trying to help > take my sorry, as non native english, perhaps i missused "flame" here > > > > Best Regards > MfG Robert Schetterer > No problem. You do very well at speaking English. Bill From roundcube222 at alaadin.org Wed Oct 24 23:04:31 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Wed, 24 Oct 2012 23:04:31 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot Message-ID: <4c9608dd97036a48885e68205922d6b2@Coptics.org> Hello, I have a question regarding mailbox locking and hope any one can help me to better understanding the locking of mbox My Postfix lock option is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is fcntl only mbox_write_locks = fcntl mbox_read_locks = fcntl now, when user is download a large mail (20 MB) for example, dovecot locks /var/mail/user with fcntl until the users finish downloading the mail ok here comes my question. While the user is downloading the mail , a mail arrives so postfix make some thing weird 1- Postfix creates /var/mail/user.lock for 5 secs , then postfix defer the mail with reason that the /var/mail/user is locked and then delete the /var/mail/user.lock (after 5 secs) My question is ? why postfix create user.lock although it shouldn't because already dovecot fcntl it ? 1- Dovecot locks /var/mail/user using FCNTL 2- Posttix at the same time tries to FCNTL /var/mail/user .. but it fail since dovecot already fcnl it. 3- Postfix at the same time add dot lock /var/mail/user for 5 secs then remove the lock. the question is how come postfix dot lock /var/mail although it couldn't FCNTL the file in the first place ??????? what i was expecting is 1- Dovecot locks /var/mail/user using FCNTL 2- Postfix tries to FCNTL /var/mail/user 3- POSTFIX WILL NOT CREATE DOTLOCK file unless the FCNTL is released by dovecot!!!! Please advise if postfix will dot lock the file even if it couldn't FCNTL the file in the first place?? Regards From mailadmin at cubixys.com Thu Oct 25 00:07:24 2012 From: mailadmin at cubixys.com (Fasil) Date: Thu, 25 Oct 2012 00:07:24 +0300 Subject: [Dovecot] Dovecot: pipe() failed: Too many open files In-Reply-To: <507F65C5.3090803@brightok.net> References: <502C4458.8090808@cubixys.com> <5B19308C-D60C-4CBD-9CD2-519C98DFCC5B@esiee.fr> <507F3C67.5020900@cubixys.com> <507F65C5.3090803@brightok.net> Message-ID: <5088588C.2060005@cubixys.com> Thanks for the input Jack. As I am using debian, the location to edit the ulimit is /etc/default/dovecot. There is no effect even after changing this value. I tried putting the value in the /etc/init.d/dovecot script without success. Fasil. On 10/18/2012 05:13 AM, Jack Bates wrote: > I'm using RHEL6 instead of ubuntu, but check the startup scripts. In > RHEL's case, the following file is sourced, so I updated it instead of > the startup scripts. > > cat /etc/sysconfig/dovecot > # Here you can specify your dovecot command line options. > # > #OPTIONS="" > ulimit -n 4096 > ulimit -u 5120 > > In addition, I had to also up the max allowed processes in the dovecot > config. 2.x and 1.x are different on this. > > http://wiki1.dovecot.org/LoginProcess <-1.x method > > > Jack > > > > On 10/17/2012 6:16 PM, Fasil wrote: >> Thanks Frank. >> Followed the URL and could not find any luck. >> >> Is there a way to change the value of 'max open files' of dovecot. >> I have tried to set the value in /etc/default/dovecot by setting >> ulimit. But the value is not getting applied. >> Could anyone help on this regard. >> >> Fasil. >> >> On 08/16/2012 09:17 AM, Frank Bonnet wrote: >>> hello >>> >>> here some useful informations >>> >>> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ >>> >>> Envoy? de mon iPhone. >>> >>> >>> Le 16 ao?t 2012 ? 02:52, Fasil a ?crit : >>> >>>> Dear all, >>>> >>>> Thank you all for such a wonderful support. Hats off to all :) >>>> >>>> Few times I came across imap login issues where new users will not >>>> be allowed to login. >>>> /var/log/mail.err shows the error below >>>> Aug 12 07:57:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 07:57:46 mail dovecot: dovecot: Temporary failure in >>>> creating login processes, slowing down for now >>>> Aug 12 07:58:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 07:59:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> Aug 12 08:00:46 mail dovecot: dovecot: pipe() failed: Too many open >>>> files >>>> >>>> I have a dovecot (V 1.2.9) +postfix (V 2.7.0) setup on ubuntu 10.04 >>>> >>>> # ulimit -Hn >>>> 1024 >>>> >>>> # ulimit -Sn >>>> 1024 >>>> >>>> # cat /proc/sys/fs/file-max >>>> 1238548 >>>> >>>> # cat /proc/`pidof dovecot`/limits | grep 'Max open' >>>> Max open files 1024 1024 files >>>> >>>> Please advice how to get rid off this. >>>> >>>> Fasil. >> >> > From troy at troyvit.com Thu Oct 25 00:34:18 2012 From: troy at troyvit.com (Troy Vitullo) Date: Wed, 24 Oct 2012 15:34:18 -0600 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <20121024163255.GI3672@harrier.slackbuilds.org> References: <20121023145245.124dd362@hrafn> <20121024163255.GI3672@harrier.slackbuilds.org> Message-ID: <20121024153418.7a183681@hrafn> On Wed, 24 Oct 2012 11:32:55 -0500 /dev/rob0 wrote: > There seems to be much confusion in this thread. I might be able to > help clear up some of it, but probably not all, because I agree with > Robert about using amavisd-new for filtering and LMTP for delivery. > Thanks for the reality check Rob. I'm circling back with the guy who originally set this up to see if we can get back on the right track. We are using pipe with virtual users, and dovecot doesn't own the mailboxes. If/when we get our collective act together and have more questions I'll respond in more detail. Thanks again, Troy From rs at sys4.de Thu Oct 25 00:40:36 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 24 Oct 2012 23:40:36 +0200 Subject: [Dovecot] spamc can't seem to call /usr/lib/dovecot/deliver In-Reply-To: <508849C4.9060800@KnoxvilleChristian.org> References: <20121023145245.124dd362@hrafn> <50873F09.4070604@KnoxvilleChristian.org> <5087454B.5030704@KnoxvilleChristian.org> <50878BB6.2090309@sys4.de> <50880D7B.4090407@KnoxvilleChristian.org> <508812A8.8000603@sys4.de> <50881740.90207@KnoxvilleChristian.org> <20121024164448.GJ3672@harrier.slackbuilds.org> <508823B6.3040208@KnoxvilleChristian.org> <5088325F.7020102@sys4.de> <508849C4.9060800@KnoxvilleChristian.org> Message-ID: <50886054.9090503@sys4.de> Am 24.10.2012 22:04, schrieb Bill Shirley: > > On 10/24/2012 2:24 PM, Robert Schetterer wrote: >> Am 24.10.2012 19:21, schrieb Bill Shirley: >>> Admittedly, I have not used amavisd-new or LMTP; they may be better. >>> But will they allow spamassassin per-user prefs? Performance is a plus; >>> another daemon is not. That saying, I'll run another daemon if I get >>> something out of it. Any benchmarks on this? >> this went away from the orig post, it went to general design >> of a email system, i think rob did explain the possible problems >> to the orginal poster very fine >> >> some people may start with local users as traditional >> mailsetup depend on this next steps they are going to use lda >> perhaps trying combined with spamc with local users >> so there is nothing bad on it, its somehow old school, >> after all, as said ,there are many broken advices out in www by all >> setups, and sometimes there are mixed up by local and virtual, so people >> may fail with permissions of local users , daemons etc >> >> sometimes later if more domains should be hosted >> pure virtual setups are the better way, and making stuff more simple ( >> but often people fail first in seeing virtual more easy ), >> >> lmtp is the best choice for it compared starting a deliver process for >> each mail, its working as a service >> >> So anyone should think about what he needs before starting to setup >> >> i.e amavis is a well supported framework since long time, it has tons of >> features >> you might wanna have and as well it can be used with per-user prefs >> >> if you dont like the complex amavis style ( many functions have many >> config points ), you could simple use a chain of milter i.e >> spamass-milter ( also with per-user prefs ), clamav-milter >> >> with milter you are able to reject on smtp income stage >> which is very cool >> anyway milters also have their pros an contras, read postfix sites about >> them >> >> i didnt tested dspam looks like it chained between lmtp >> so perhaps also good choice, and could be combined with milters >> >> i had other setups with chained spampd/clamsmtp >> amavis on seperate filter hosts etc >> all worked fine >> >> but as dovecot/postfix development going forward , i redesigned all >> these depending to have more functions and performance >> >> so i recommand, use your working setups as i.e lifetime of your hardware >> etc, but if building new mailserver choose modern setup ideas >> and daemon combinations >> >> >> Best Regards >> MfG Robert Schetterer >> > Thank you for a very informative post. I took a quick look at > spamass-milter but I can't find any configuration information on how to > use spamasssassin's per-user prefs. I thought the only way to support > per-user prefs was post queue since you have to know who is getting the > email to check their prefs. you have to study its parameters ( they may differ by version and distro ) http://linux.die.net/man/1/spamass-milter i use it like /usr/sbin/spamass-milter -P /var/spool/postfix/spamass-milter/spamass.pid -f -p /var/spool/postfix/spamass/spamass.sock -f -e -x -I -u vmail -r 15 -i 127.0.0.1 i have my spamassassin setup with mysql for users self settings use i.e.e webmail horde with sam module, or something equal with i.e squirrelmail or roundcube but i managed it before ,also in using local files with maildrop as i said ,its not ideal cause of pre queue design, but reality shows good enough for big isp setup and it may be combined > > I am using clamav-milter. Milters are nice. for antispam using sanesecurity antispam signatures are nice thats "cheaper" then spamassassin > > I set my mail server up 15+ years ago, so it's time for me to have a > re-think here. At that time there were no milters for postfix (don't > remember a Dovecot either). I've try to steer away from re-injects > since they affect the mail received numbers. Are we saying Dovecot's > LMTP can call spamd? i dont tested ,looks like dspam can do it http://wiki2.dovecot.org/HowTo/Virtual%2BPostfix%2BDspam%2BDovecot I'm on Dovecot 1.2 at home until I can upgrade. > There is no LMTP in Dovecot 1.x, right? yes ,you should use 2.1.x > > I have a few mail servers running Dovecot 2.0 and 2.1 and yes, I want > them to perform well. so you may change setup layout > > Bill > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stocton12 at yahoo.com Thu Oct 25 01:13:14 2012 From: stocton12 at yahoo.com (b m) Date: Wed, 24 Oct 2012 15:13:14 -0700 (PDT) Subject: [Dovecot] Public folders and groups Message-ID: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? From stan at hardwarefreak.com Thu Oct 25 06:48:33 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 22:48:33 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087FEED.7060007@Media-Brokers.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> Message-ID: <5088B691.7030100@hardwarefreak.com> On 10/24/2012 9:45 AM, Charles Marcus wrote: > On 2012-10-24 10:01 AM, Stan Hoeppner wrote: >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Hmmm... I wonder would it be possible for dovecot to automatically lower > the 'niceness' for index rebuilds (on systems that support such) to > avoid causing such distress? Changing the process priority would not help. Indexing a large mailbox is an IO bound, not a compute bound, operation. With Linux, changing from the CFQ to deadline scheduler may help some with low responsiveness. But the only real solution for such a case where iowait is bringing the system to its knees is to acquire storage with far greater IOPS and concurrent IO capability. I.e. a server. -- Stan From stan at hardwarefreak.com Thu Oct 25 06:57:47 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 24 Oct 2012 22:57:47 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <4c9608dd97036a48885e68205922d6b2@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> Message-ID: <5088B8BB.9000109@hardwarefreak.com> On 10/24/2012 3:04 PM, Robert JR wrote: > I have a question regarding mailbox locking and hope any one can help me > to better understanding the locking of mbox > > My Postfix lock option is fcntl dotlock > mailbox_delivery_lock = fcntl, dotlock > virtual_mailbox_lock = fcntl, dotlock > > My Dovecot lock option is fcntl only > mbox_write_locks = fcntl > mbox_read_locks = fcntl Postfix is delivering the mail to dovecot. This is done via the deliver program or lmtp which are pipes, not files. Thus, why is Postfix attempting to write files in the user's mail directory? You write new mail to the mailbox file with either Dovecot or Postfix, not both. Fix that problem and the locking problem disappears. -- Stan From eliezer at ngtech.co.il Thu Oct 25 10:16:28 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 25 Oct 2012 09:16:28 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5086F2BB.7010704@rename-it.nl> References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> Message-ID: <5088E74C.9030006@ngtech.co.il> On 10/23/2012 9:40 PM, Stephan Bosch wrote: > Also, could you provide your full configuration as output from `dovecot > -n` ? > > Regards, > > Stephan. > Thanks Stephan, I just upgraded from 2.0.17 to 2.1.9 and ph 0.3.1 (gentoo) and it seems to work as expected and dont leave any traces in the INBOX with the same script. I dont know the old ph version and since it was resolved i'm ok with it. My only problem is that it will put the file in the folder but will not mark the folder with the new file until I actually check the folder manually. it's not that much hustle but if there is a way to solve it I will be more then happy to hear about it. Thanks, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From roundcube222 at alaadin.org Thu Oct 25 10:23:29 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 10:23:29 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <5088B8BB.9000109@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: On 2012-10-25 06:57, Stan Hoeppner wrote: > On 10/24/2012 3:04 PM, Robert JR wrote: > >> I have a question regarding mailbox locking and hope any one can >> help >> me to better understanding the locking of mbox My Postfix lock >> option >> is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock >> virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is >> fcntl >> only mbox_write_locks = fcntl mbox_read_locks = fcntl > > > > Postfix is delivering the mail to dovecot. This is done via the > deliver > program or lmtp which are pipes, not files. Thus, why is Postfix > attempting to write files in the user's mail directory? > > You write new mail to the mailbox file with either Dovecot or > Postfix, > not both. Fix that problem and the locking problem disappears. Stan, sorry but you didnot understand my question at all, dovecot in this case is reading the mailbox file while user downloading the mail and not WRITING. only postfix write when a mail arrives and DOVECOT only read the mail. And even if both write to the file, I have already set the locking option of both to FCNTL so no problem should happen. My question is postfix locking option is = FCNTL, DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, why it dotlock the file 5 secs then remove the dotlock and say resource unaviable? why in the first place it dotlock the file, althought it couldnot FCNTL it in the first place since it is FCNTL by dovecot while reading the in the moment Any help will be greatly appreciated..... Regards Robert. From dg at dguhl.org Thu Oct 25 11:56:00 2012 From: dg at dguhl.org (Dennis Guhl) Date: Thu, 25 Oct 2012 10:56:00 +0200 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025085559.GA7323@laptop-dg.leere.eu> On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: [..] > should happen. My question is postfix locking option is = FCNTL, > DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, > why it dotlock the file 5 secs then remove the dotlock and say > resource unaviable? why in the first place it dotlock the file, > althought it couldnot FCNTL it in the first place since it is FCNTL > by dovecot while reading the in the moment You are on the wrong mailing list, this is no problem of Dovecot. Ask this question on postfix-users (maybe you are requested to show evidence of this behaviour). Dennis From roundcube222 at alaadin.org Thu Oct 25 12:26:10 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 12:26:10 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025085559.GA7323@laptop-dg.leere.eu> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025085559.GA7323@laptop-dg.leere.eu> Message-ID: On 2012-10-25 11:56, Dennis Guhl wrote: > On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > > [..] > >> should happen. My question is postfix locking option is = FCNTL, >> DOTLOCK , and dovecot = FCNTL, if postfix find a file already FCNTL, >> why it dotlock the file 5 secs then remove the dotlock and say >> resource >> unaviable? why in the first place it dotlock the file, althought it >> couldnot FCNTL it in the first place since it is FCNTL by dovecot >> while >> reading the in the moment > > You are on the wrong mailing list, this is no problem of Dovecot. > > Ask this question on postfix-users (maybe you are requested to show > evidence of this behaviour). > > Dennis\ Thanks dennis for your reply. But, This is also an dovecot issue, because how dovecot use the FCNTL lock maybe different on how Postfix use it , that's why i had to ask also here at dovecot mailist this question. I think This behavior is understood by any one already have experience on how locking is made to /var/mail/files, I am just trying to understand how/why it happened. My question is postfix has locking option: FCNTL, DOTLOCK, and a file is already FCNTL by dovecot, will postfix Apply the DOTLOCK and continue try to FCNTL , althugh postfix already found that the file is already FCNTL by dovecot. So it shouldnot dotlock the mbx file untill it frist FCNTL the mailbox after dovecot finished. When i asked this question on irc postfix, they said that maybe FCNTL is used in different option with dovecot and this Might be the reason of the issue and recommended to ask here for help .. I guess Timo will be able to respond to this on the spot ... Thanks again. Robert JR. From jg at softjury.de Thu Oct 25 13:35:53 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Thu, 25 Oct 2012 12:35:53 +0200 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> Message-ID: <50891609.9070709@softjury.de> Am 25.10.2012 00:13, schrieb b m: > Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? Here a sentence to this, found in the dovecot wiki. (http://wiki2.dovecot.org/ACL) ACL groups support works by returning a comma-separated acl_groups extra field from userdb, which contains all the groups the user belongs to. It seems to be possible, I had an acl_groups field in my MySQL Database for this, I'am sure it is something like that in an AD too. From r.ordinas at math.univ-paris-diderot.fr Thu Oct 25 16:09:47 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Thu, 25 Oct 2012 15:09:47 +0200 Subject: [Dovecot] Small issue with "submission host" Message-ID: <50893A1B.2060205@math.univ-paris-diderot.fr> Hi everyone, I'm facing a small issue with the lda/lmtp "submission_host" feature in dovecot 2.0.14. When sending mail to MTA (in case of sieve filter forwarding for example), dovecot pass a RCPT TO command just after the EHLO. He's missing the MAIL FROM command. Therefore, my MTA show me a warning like this : "improper command pipelining after EHLO". How can i solve that ? Regards, Raphael From tlx at leuxner.net Thu Oct 25 17:08:52 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 25 Oct 2012 16:08:52 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <50893A1B.2060205@math.univ-paris-diderot.fr> References: <50893A1B.2060205@math.univ-paris-diderot.fr> Message-ID: <20121025140852.GA15639@nihlus.leuxner.net> On Thu, Oct 25, 2012 at 03:09:47PM +0200, Raphael Ordinas wrote: > When sending mail to MTA (in case of sieve filter forwarding for > example), dovecot pass a RCPT TO command just after the EHLO. He's > missing the MAIL FROM command. > Therefore, my MTA show me a warning like this : "improper command > pipelining after EHLO". Works for me with latest and greatest although I'm not using the 'submission_host' option but pure LMTP Unix socket: [...] service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } Best to show your 'doveconf -n' for more thoughts. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From rob0 at gmx.co.uk Thu Oct 25 17:28:00 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 25 Oct 2012 09:28:00 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025142800.GN3672@harrier.slackbuilds.org> On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > Stan, sorry but you didnot understand my question at all, dovecot > in this case is reading the mailbox file while user downloading the > mail and not WRITING. only postfix write when a mail arrives and > DOVECOT only read the mail. And even if both write to the file, I I can't answer (don't know), but I can tell you that this is not true. Dovecot also writes to the file: updating message read flags and such. > Any help will be greatly appreciated..... Maildir is not for everyone, but it does handle issues like this smoothly. The delivery agent is always able to deliver new mail. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From roundcube222 at alaadin.org Thu Oct 25 19:08:25 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 19:08:25 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025142800.GN3672@harrier.slackbuilds.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025142800.GN3672@harrier.slackbuilds.org> Message-ID: <23542f848cc61c879822b03810621256@Coptics.org> On 2012-10-25 17:28, /dev/rob0 wrote: > On Thu, Oct 25, 2012 at 10:23:29AM +0300, Robert JR wrote: > >> Stan, sorry but you didnot understand my question at all, dovecot in >> this case is reading the mailbox file while user downloading the >> mail >> and not WRITING. only postfix write when a mail arrives and DOVECOT >> only read the mail. And even if both write to the file, I > > I can't answer (don't know), but I can tell you that this is not > true. Dovecot also writes to the file: updating message read flags > and such. > >> Any help will be greatly appreciated..... > > Maildir is not for everyone, but it does handle issues like this > smoothly. The delivery agent is always able to deliver new mail. We can not convert to maildir now as we have alot of users and converting each account will take a huge time .. well.. we can live with /var/mail/mailbox format , I just need to understand the locking issue and hope Timo will be able to answer our question.. Thanks again From stan at hardwarefreak.com Thu Oct 25 21:00:24 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:00:24 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <50897E38.6070304@hardwarefreak.com> On 10/25/2012 2:23 AM, Robert JR wrote: > On 2012-10-25 06:57, Stan Hoeppner wrote: > >> On 10/24/2012 3:04 PM, Robert JR wrote: >> >>> I have a question regarding mailbox locking and hope any one can help >>> me to better understanding the locking of mbox My Postfix lock option >>> is fcntl dotlock mailbox_delivery_lock = fcntl, dotlock >>> virtual_mailbox_lock = fcntl, dotlock My Dovecot lock option is fcntl >>> only mbox_write_locks = fcntl mbox_read_locks = fcntl >> >> >> >> Postfix is delivering the mail to dovecot. This is done via the deliver >> program or lmtp which are pipes, not files. Thus, why is Postfix >> attempting to write files in the user's mail directory? >> >> You write new mail to the mailbox file with either Dovecot or Postfix, >> not both. Fix that problem and the locking problem disappears. > > Stan, sorry but you didnot understand my question at all Yes, actually I did, but I missed one part of it because I assumed you had Dovecot setup properly. It doesn't matter if the mbox locks are write or read or both. Locks are the problem, period, because you have two daemons fighting over the same files. The fix is absolutely trivial: Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates the file locking issue. Thus it also increases throughput as lock latency is eliminated. It also enables using all the Dovecot delivery plugins such as Sieve, Quota, anti-spam, etc. I had the same Postfix/Dovecot mbox locking problem many years ago when I first started using Dovecot. After the suggestions from the fine folks on this list I switched to LDA. It not only eliminated locking, it completely changed the character of my Dovecot install, both in performance and capabilities, as well as fixed some message flag problems, etc. If you're not using LDA with Postfix/Dovecot you're insane. ;) Some might say you're insane for using mbox but I feel it's fine for many installations. I use it myself. Our setups are very similar. To switch to LDA... If you're using 1.2.x http://wiki.dovecot.org/LDA/Postfix If you're using 2.x http://wiki2.dovecot.org/LDA/Postfix -- Stan From brintoul at sbcglobal.net Thu Oct 25 21:01:53 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 11:01:53 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process Message-ID: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Hello: I'm using Dovecot to provide IMAP services for Thunderbird clients. The user's mail is stored in Maildir format and the individual emails which the user "receives" are actually downloaded and put into the Maildir "repository" using a Python email client (POP3 client). I am trying to create the dovecot-uidlist file and maintain the "next UID" value within it, but I am having trouble with the UIDVALIDITY and the 128 bit mailbox global UID among other things. How can I "seed" a dovecot-uidlist file with these values..? Sorry if the information is not much to go on, please let me know what more information you might need to help point me in the right direction. Thanks, B. RIntoul From sven at svenhartge.de Thu Oct 25 21:06:17 2012 From: sven at svenhartge.de (Sven Hartge) Date: Thu, 25 Oct 2012 20:06:17 +0200 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <697ejv7veev8@mids.svenhartge.de> Bradley Rintoul wrote: > I'm using Dovecot to provide IMAP services for Thunderbird clients. > The user's mail is stored in Maildir format and the individual emails > which the user "receives" are actually downloaded and put into the > Maildir "repository" using a Python email client (POP3 client). > I am trying to create the dovecot-uidlist file and maintain the "next > UID" value within it, but I am having trouble with the UIDVALIDITY and > the 128 bit mailbox global UID among other things. How can I "seed" a > dovecot-uidlist file with these values..? > Sorry if the information is not much to go on, please let me know what > more information you might need to help point me in the right > direction. How about injecting the mails into the LDA. It will take care of proper delivery without the need for your programm to know the internal workings of dovecot. S? -- Sigmentation fault. Core dumped. From stan at hardwarefreak.com Thu Oct 25 21:23:37 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:23:37 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50897E38.6070304@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> Message-ID: <508983A9.9090605@hardwarefreak.com> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: > Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > the file locking issue. Thus it also increases throughput as lock > latency is eliminated. It also enables using all the Dovecot delivery > plugins such as Sieve, Quota, anti-spam, etc. I forgot to mention one very important feature of Dovecot LDA: New messages delivered by Postfix are indexed by LDA as they are written to the mailbox, flags updated at this time, etc. Thus when a mailbox is opened in an IMAP MUA, new messages are displayed instantly (I don't use POP but it's probably faster as well). With your current setup it can take from a few to many seconds to show new mail, depending on message count. With LDA new message count seems to have no impact on the speed of display. -- Stan From stan at hardwarefreak.com Thu Oct 25 21:37:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 13:37:50 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <508986FE.6060309@hardwarefreak.com> On 10/25/2012 1:01 PM, Bradley Rintoul wrote: > the individual emails which the user > "receives" are actually downloaded and put into the Maildir "repository" using a > Python email client (POP3 client). Tell us more about this. This doesn't sound like 'normal' email being fetched from an external service provider over a slow link scenario. This sounds more like an application server generating data files that are then POP'd down to the Dovecot server. Assuming that for now... If you're able to run a popd on this application server, why not run a simple smtp MTA and send these files directly to the user email addresses? Injecting the payload is a pretty simply shell command line, or from within a Python/Perl/etc script. Dovecot handles the rest as it arrives. Problem solved. The more you can do with smtp the better off you are and the easier it is. -- Stan From roundcube222 at alaadin.org Thu Oct 25 20:38:26 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 20:38:26 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508983A9.9090605@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> Message-ID: <239f53fb233beb2b45e68638cca24260@Coptics.org> On 2012-10-25 21:23, Stan Hoeppner wrote: > On 10/25/2012 1:00 PM, Stan Hoeppner wrote: > >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place >> of >> the Postfix local/virtual delivery agent. Using Dovecot LDA >> eliminates >> the file locking issue. Thus it also increases throughput as lock >> latency is eliminated. It also enables using all the Dovecot >> delivery >> plugins such as Sieve, Quota, anti-spam, etc. > > I forgot to mention one very important feature of Dovecot LDA: > > New messages delivered by Postfix are indexed by LDA as they are > written > to the mailbox, flags updated at this time, etc. Thus when a mailbox > is > opened in an IMAP MUA, new messages are displayed instantly (I don't > use > POP but it's probably faster as well). With your current setup it can > take from a few to many seconds to show new mail, depending on > message > count. With LDA new message count seems to have no impact on the > speed > of display. Thanks stan very much for your detailed answer, i will read about LDA to know how it works. But i still wonder why this mailbox locking issue and I hope for my referece some one at this form explain to me the issue reason. as since postfix,dovecot has the same lockign settings so why this issue happen ? I have spend 3 days searching for an answer and i couldnot find any.. I think only Timo can answer my question... Thanks again stan From brintoul at sbcglobal.net Thu Oct 25 22:13:34 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 12:13:34 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <697ejv7veev8@mids.svenhartge.de> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> Message-ID: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> This could be good. I'd never looked into the LDA - I will study up on it. Someone else was helping out here and I thought I'd shed some more light on what I'm doing here... Let's say someone has an account with Yahoo, for example. My Python code is fetching email from the user's Yahoo! account and placing it into the Dovecot Maildir storage for a particular user. Now when the user retrieves their mail, they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, of sorts... Thanks for the responses! (Is there an IRC channel?) ________________________________ From: Sven Hartge To: dovecot at dovecot.org Sent: Thu, October 25, 2012 11:07:26 AM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process Bradley Rintoul wrote: > I'm using Dovecot to provide IMAP services for Thunderbird clients. > The user's mail is stored in Maildir format and the individual emails > which the user "receives" are actually downloaded and put into the > Maildir "repository" using a Python email client (POP3 client). > I am trying to create the dovecot-uidlist file and maintain the "next > UID" value within it, but I am having trouble with the UIDVALIDITY and > the 128 bit mailbox global UID among other things. How can I "seed" a > dovecot-uidlist file with these values..? > Sorry if the information is not much to go on, please let me know what > more information you might need to help point me in the right > direction. How about injecting the mails into the LDA. It will take care of proper delivery without the need for your programm to know the internal workings of dovecot. S? -- Sigmentation fault. Core dumped. From slitt at troubleshooters.com Thu Oct 25 22:31:38 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 15:31:38 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <697ejv7veev8@mids.svenhartge.de> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> Message-ID: <20121025153138.39c6b363@mydesk> On Thu, 25 Oct 2012 20:06:17 +0200, Sven Hartge said: > Bradley Rintoul wrote: > > > I'm using Dovecot to provide IMAP services for Thunderbird clients. > > The user's mail is stored in Maildir format and the individual > > emails which the user "receives" are actually downloaded and put > > into the Maildir "repository" using a Python email client (POP3 > > client). > > > I am trying to create the dovecot-uidlist file and maintain the > > "next UID" value within it, but I am having trouble with the > > UIDVALIDITY and the 128 bit mailbox global UID among other things. > > How can I "seed" a dovecot-uidlist file with these values..? > > > Sorry if the information is not much to go on, please let me know > > what more information you might need to help point me in the right > > direction. > > How about injecting the mails into the LDA. It will take care of > proper delivery without the need for your programm to know the > internal workings of dovecot. What is an LDA? SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From slitt at troubleshooters.com Thu Oct 25 22:35:06 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 15:35:06 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <20121025153138.39c6b363@mydesk> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <20121025153138.39c6b363@mydesk> Message-ID: <20121025153506.1f959544@mydesk> On Thu, 25 Oct 2012 15:31:38 -0400, Steve Litt said: > On Thu, 25 Oct 2012 20:06:17 +0200, Sven Hartge said: > > How about injecting the mails into the LDA. It will take care of > > proper delivery without the need for your programm to know the > > internal workings of dovecot. > > What is an LDA? Nevermind, somebody had already answered that question. Sorry for asking too quickly. SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From loshkovskyi at yandex.ua Thu Oct 25 13:03:23 2012 From: loshkovskyi at yandex.ua (Loshkovskyi Andrii) Date: Thu, 25 Oct 2012 13:03:23 +0300 Subject: [Dovecot] Dovecot sends BYE while fetching X-GM-MSGID Message-ID: <1698191351159403@web29e.yandex.ru> Hello everyone, While using the following set of commands, I am having the error as below: FETCH 7 (X-GM-MSGID) A15 FETCH 7 (X-GM-MSGID) A15 BAD Error in IMAP command FETCH: Unknown parameter X-GM-MSGID Can I somehow disable such errors so that Dovecot won't send BYE on X-GM-MSGID but just proceed with following emails? -- My configuration is below: [root at server ~]# cat /etc/issue CentOS release 6.3 (Final) [root at server ~]# dovecot --version 2.0.9 [root at server ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.17.1.el6.centos.plus.i686 i686 CentOS release 6.3 (Final) ext4 auth_debug = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes default_client_limit = 10000 default_process_limit = 300 disable_plaintext_auth = no listen = * mail_location = maildir:/var/spool/mail/%d/%n/Maildir mbox_write_locks = fcntl passdb { args = scheme=SSHA512 username_format=%u /etc/dovecot/users driver = passwd-file } service imap-login { service_count = 0 vsz_limit = 128 M } ssl_cert = References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <508995D2.3070705@whyscream.net> On 25/10/12 21:13, Bradley Rintoul wrote: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on what > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... I'm intrigued by this. Why are you using some self-baked(?) python script to fetch the mail in stead of using ready-made components like fetchmail? Unless there's a special reason not to, try using the LDA (and fetchmail/getmail for that matter). This sounds exactly what you want: http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal -- Tom > > Thanks for the responses! > > (Is there an IRC channel?) > > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > From rs at sys4.de Thu Oct 25 22:46:20 2012 From: rs at sys4.de (Robert Schetterer) Date: Thu, 25 Oct 2012 21:46:20 +0200 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <5089970C.1080900@sys4.de> Am 25.10.2012 21:13, schrieb Bradley Rintoul: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on what > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... > > Thanks for the responses! > > (Is there an IRC channel?) > perhaps you could use parts from here http://wiki.dovecot.org/HowTo/RefilterMail > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > -- Best Regards MfG Robert Schetterer sys4 AG Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Sitz der Gesellschaft M?nchen Amtsgericht M?nchen HRB 0000 Vorstandsmitglieder Patrick Ben Koetter Axel von der Ohe Marc Schifbauer Vorstandsvorsitzender Patrick Ben Koetter Aufsichtsratsvorsitzender J?rg Heidrich From brintoul at sbcglobal.net Thu Oct 25 22:54:43 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 12:54:43 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <508995D2.3070705@whyscream.net> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> Message-ID: <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> I am brand new to this whole "email" thing. I am looking at this article right now: http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot Thanks for the input! ________________________________ From: Tom Hendrikx To: dovecot at dovecot.org Sent: Thu, October 25, 2012 12:41:24 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process On 25/10/12 21:13, Bradley Rintoul wrote: > This could be good. I'd never looked into the LDA - I will study up on it. > > Someone else was helping out here and I thought I'd shed some more light on >what > > I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their >mail, > > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > > of sorts... I'm intrigued by this. Why are you using some self-baked(?) python script to fetch the mail in stead of using ready-made components like fetchmail? Unless there's a special reason not to, try using the LDA (and fetchmail/getmail for that matter). This sounds exactly what you want: http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal -- Tom > > Thanks for the responses! > > (Is there an IRC channel?) > > > > ________________________________ > From: Sven Hartge > To: dovecot at dovecot.org > Sent: Thu, October 25, 2012 11:07:26 AM > Subject: Re: [Dovecot] Creating Maildir and populating with emails via external > > Python process > > Bradley Rintoul wrote: > >> I'm using Dovecot to provide IMAP services for Thunderbird clients. >> The user's mail is stored in Maildir format and the individual emails >> which the user "receives" are actually downloaded and put into the >> Maildir "repository" using a Python email client (POP3 client). > >> I am trying to create the dovecot-uidlist file and maintain the "next >> UID" value within it, but I am having trouble with the UIDVALIDITY and >> the 128 bit mailbox global UID among other things. How can I "seed" a >> dovecot-uidlist file with these values..? > >> Sorry if the information is not much to go on, please let me know what >> more information you might need to help point me in the right >> direction. > > How about injecting the mails into the LDA. It will take care of proper > delivery without the need for your programm to know the internal > workings of dovecot. > > S? > From rob0 at gmx.co.uk Thu Oct 25 23:10:59 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 25 Oct 2012 15:10:59 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> Message-ID: <20121025201059.GO3672@harrier.slackbuilds.org> > From: Tom Hendrikx > I'm intrigued by this. Why are you using some self-baked(?) python > script to fetch the mail in stead of using ready-made components > like fetchmail? > > Unless there's a special reason not to, try using the LDA (and > fetchmail/getmail for that matter). > > This sounds exactly what you want: > http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal > On Thu, Oct 25, 2012 at 12:54:43PM -0700, Bradley Rintoul wrote: > I am brand new to this whole "email" thing. I am looking at this > article right now: > http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot I did not see where you described the ultimate goal. That should have been the starting point of this thread. Describe the problem, not how you think it should be solved, because you are new to this, and your ideas might benefit from some scrutiny. Use plain language. I have not reviewed your howto, but personally I would recommend neither fetchmail (I'd choose getmail) nor procmail (other choices exist, depending on what you are trying to do.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Thu Oct 25 23:18:51 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 15:18:51 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <239f53fb233beb2b45e68638cca24260@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> Message-ID: <50899EAB.2030003@hardwarefreak.com> On 10/25/2012 12:38 PM, Robert JR wrote: > On 2012-10-25 21:23, Stan Hoeppner wrote: > >> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: >> >>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >>> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >>> the file locking issue. Thus it also increases throughput as lock >>> latency is eliminated. It also enables using all the Dovecot delivery >>> plugins such as Sieve, Quota, anti-spam, etc. >> >> I forgot to mention one very important feature of Dovecot LDA: >> >> New messages delivered by Postfix are indexed by LDA as they are written >> to the mailbox, flags updated at this time, etc. Thus when a mailbox is >> opened in an IMAP MUA, new messages are displayed instantly (I don't use >> POP but it's probably faster as well). With your current setup it can >> take from a few to many seconds to show new mail, depending on message >> count. With LDA new message count seems to have no impact on the speed >> of display. > > Thanks stan very much for your detailed answer, i will read about LDA to > know how it works. But i still wonder why this mailbox locking issue and I > hope for my referece some one at this form explain to me the issue reason. > as since postfix,dovecot has the same lockign settings so why this issue > happen ? I have spend 3 days searching for an answer and i couldnot find > any.. I think only Timo can answer my question... I chose to focus on the permanent and proper solution, which is eliminating your lock contention altogether and enabling maximum performance and features. Learning about file locking problems between applications may be a noble endeavor, but at this point it's simply a waste of your time. A gazillion papers have been written about this subject over the years. If you're that interested hunt them down and read them. I'm sure Google can find 1000 of them quickly. You'll be looking for academic papers, not forum posts. > Thanks again stan You're welcome. -- Stan From slitt at troubleshooters.com Thu Oct 25 23:18:52 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 25 Oct 2012 16:18:52 -0400 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <20121025161852.5d3c448f@mydesk> On Thu, 25 Oct 2012 12:13:34 -0700 (PDT), Bradley Rintoul said: > This could be good. I'd never looked into the LDA - I will study up > on it. > > Someone else was helping out here and I thought I'd shed some more > light on what I'm doing here... > > Let's say someone has an account with Yahoo, for example. My Python > code is fetching email from the user's Yahoo! account and placing it > into the Dovecot Maildir storage for a particular user. Now when > the user retrieves their mail, they are doing so using my Dovecot - > my Dovecot instance is acting as a proxy, of sorts... > > Thanks for the responses! > > (Is there an IRC channel?) Hi Bradley, I'm doing almost the exact same thing, but with fetchmail and procmail. I go out and grab my email from about five different places using fetchmail, which feeds the messages to procmail, with .procmailrc deciding where in the Dovecot maildir tree to put them. Your only need I *haven't* accomplished is having different users get their mail from my Dovecot, and to make sure each users' email goes where they can get it via IMAP connection to your Dovecot. If you can get different IMAP mailboxes for different users, you can put each user in .procmailrc so as to deliver to the correct box. Anyway, Procmail knows exactly how to submit an email to Dovecot, so you don't need to worry about actually placing the file into the tree, or anything like that. You mention you've written some Python code. If the purpose of your Python code is just to retrieve from SMTP servers, you can drop your Python code in favor of Fetchmail and Procmail. If your Python code actually does something with the emails, you can call a subset of your Python code from Procmail, to do its magic on each email. Here's how my fetchmail is running: 29588 ? Ss 0:21 fetchmail -f /home/slitt/.fetchmailrc And here's a partial view of my .fetchmailrc, showing my retrieval from Bluehost and gmail: =================================== set postmaster "slitt" set bouncemail set no spambounce set properties "" set daemon 180 #poll mail.a3b3.com protocol POP3: poll mail.a3b3.com protocol IMAP: user 'slitt at troubleshooters.com' there is 'slitt' here pass wouldnt_you_like_to_know limit 50000000 warnings 3200 expunge 60 ssl #Use ssl encryption sslcommonname "*.bluehost.com" sslcertck mda "/usr/bin/procmail -d %T" fetchall; poll imap.gmail.com protocol IMAP user 'litttest at gmail.com' there is 'slitt' here pass 'I_just_cant_say' #portnumber 993 limit 50000000 warnings 3200 expunge 60 mda "/usr/bin/procmail -d %T" fetchlimit 50 ssl; ==================================== Do you notice the "mda" line on both pulls? That means "use procmail as your mda", which just ships each email to Procmail. Procmail knows exactly how to deliver stuff to Dovecot. The following are the top several lines of my .procmailrc: =================================== DEFAULT=$HOME/mail/Maildir/.INBOX/ MAILDIR=$HOME/mail/Maildir/ LOCKFILE=$HOME/mail/.lock VERBOSE=no LOGFILE=$HOME/procmail/log #GARBAGE=.garbage/ GARBAGE=/dev/null PURGATORY=.garbage/ SUPREMUM=9876543210 #PROCMAIL SUPREMUM NUMBER, SEE http://www.perlcode.org/tutorials/procmail/proctut/proctip2.pod #### HANDLE STUFF FROM littdom at gmail.com and litttest at gmail.com #### :0: * ^Delivered-To:.*littdom at gmail.com .littdom_gmail/ :0: * ^Delivered-To:.*litttest at gmail.com .litttest_gmail/ =================================== A few explanations: First, I couldn't include my actual filters, because they are full of very unflattering comments concerning various trolls, ignos, blabbermouths, and proudly helpless fools. The $MAILDIR environment variable is the rood directory of your Maildir tree. $DEFAULT is the location of the main inbox for that -- I think it's where you put email that doesn't get routed elsewhere by Procmail. $GARBAGE is an environment var I made up as code for where filtered stuff gets sent. It's usually /dev/null because I don't want to see that junk again. However, I can temporarily change it to an actual IMAP directory for troubleshooting. $PURGATORY is junk that I actually want to OK the deletion of. I actually currently have nothing filtered to $PURGATORY, but it's there. $SUPREMUM is a very large number that is used in making OR logic, which is otherwise difficult. I couldn't make the $SUPREMUM env var work, so I had to use a literal, and here's a way I got all my magazines into one mailbox: :0: * 9876543210^0 ^From.*onsale.com * 9876543210^0 ^From.*pcmag.com * 9876543210^0 ^From.*itworld.com * 9876543210^0 ^From.*networkworld.info * 9876543210^0 ^From.*infoworld.com * 9876543210^0 ^From.*whatsnewnow.com * 9876543210^0 ^From.*eweek.com * 9876543210^0 ^From.*computerworld.com .mags/ By the way, BE SURE to note the slash after the directory name. That trailing slash tells Procmail that it's delivering to a Maildir, not to an (ugh) mbox. Anyway, I think you and I are doing very similar things, albeit for very different reasons. My motivation is that I consider all currently available email clients to be junk, and don't want them holding my email, so I hold it in a Dovecot hosted Maildir instead. I'll be interested in how you solve this. Please keep me (and probably everyone on this list) in the loop. Thank you so much! SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From mike at alaadin.org Thu Oct 25 22:21:24 2012 From: mike at alaadin.org (Mike John) Date: Thu, 25 Oct 2012 22:21:24 +0300 Subject: [Dovecot] Changing password for users Message-ID: <7a6a7cc64756b895059f7384b256019b@coptics.org> Hello, I am using dovecot (2.0.9) and using virtual users using passdb { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i make my virtual users change their passwords using web interface ? My users already uses squirrelmail to access their mail. is there a program to add to squirrelmail to add this function to the clients ? or should i user different separate website for password changing ? and what program/tool can help me with this ? Any ideas is greatly appreciated. Mike. From roundcube222 at alaadin.org Thu Oct 25 22:42:32 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Thu, 25 Oct 2012 22:42:32 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50899EAB.2030003@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> <50899EAB.2030003@hardwarefreak.com> Message-ID: On 2012-10-25 23:18, Stan Hoeppner wrote: > On 10/25/2012 12:38 PM, Robert JR wrote: > >> On 2012-10-25 21:23, Stan Hoeppner wrote: >> >>> On 10/25/2012 1:00 PM, Stan Hoeppner wrote: >>> >>>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in >>>> place of the Postfix local/virtual delivery agent. Using Dovecot >>>> LDA eliminates the file locking issue. Thus it also increases >>>> throughput as lock latency is eliminated. It also enables using >>>> all >>>> the Dovecot delivery plugins such as Sieve, Quota, anti-spam, etc. >>> I forgot to mention one very important feature of Dovecot LDA: New >>> messages delivered by Postfix are indexed by LDA as they are >>> written >>> to the mailbox, flags updated at this time, etc. Thus when a >>> mailbox >>> is opened in an IMAP MUA, new messages are displayed instantly (I >>> don't use POP but it's probably faster as well). With your current >>> setup it can take from a few to many seconds to show new mail, >>> depending on message count. With LDA new message count seems to >>> have >>> no impact on the speed of display. >> Thanks stan very much for your detailed answer, i will read about >> LDA >> to know how it works. But i still wonder why this mailbox locking >> issue >> and I hope for my referece some one at this form explain to me the >> issue reason. as since postfix,dovecot has the same lockign settings >> so >> why this issue happen ? I have spend 3 days searching for an answer >> and >> i couldnot find any.. I think only Timo can answer my question... > > I chose to focus on the permanent and proper solution, which is > eliminating your lock contention altogether and enabling maximum > performance and features. > > Learning about file locking problems between applications may be a > noble > endeavor, but at this point it's simply a waste of your time. A > gazillion papers have been written about this subject over the years. > If you're that interested hunt them down and read them. I'm sure > Google > can find 1000 of them quickly. You'll be looking for academic papers, > not forum posts. > >> Thanks again stan > > You're welcome. Thanks again Stan, you are very helpfull, i will start learning how to configure LDA, but hopefully i can also have an answer from Timo about why this issue happened.. i am sure he is aware of it and can explain it.. From stocton12 at yahoo.com Thu Oct 25 23:49:52 2012 From: stocton12 at yahoo.com (b m) Date: Thu, 25 Oct 2012 13:49:52 -0700 (PDT) Subject: [Dovecot] Public folders and groups In-Reply-To: <50891609.9070709@softjury.de> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> Message-ID: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> No AD doesn't have such a field, but I could use some unused field to get what I want. Let's say set "Attribute1" to "group1". The problem is how to get that info. I guess I have to edit dovecot-ldap.conf and put in user_attrs something like that ",=acl_groups=Attribute1". Any suggestions? ________________________________ From: Jan Phillip Greimann To: b m ; Dovecot Mailing List Sent: Thursday, October 25, 2012 1:35 PM Subject: Re: [Dovecot] Public folders and groups Am 25.10.2012 00:13, schrieb b m: > Currently I have dovecot working with Active Directory authentication and public folders with acl. In acl I have the users I want to access the public folders. It'll be easier for me to use one group instead of 50 users but I can't get it to work. From where does dovecot get the "group" attribute for a user? Can it read the groups that a user belongs from AD? Here a sentence to this, found in the dovecot wiki. (http://wiki2.dovecot.org/ACL) ACL groups support works by returning a comma-separated acl_groups extra field from userdb, which contains all the groups the user belongs to. It seems to be possible, I had an acl_groups field in my MySQL Database for this, I'am sure it is something like that in an AD too. From stan at hardwarefreak.com Fri Oct 26 00:01:58 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 16:01:58 -0500 Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> Message-ID: <5089A8C6.1090308@hardwarefreak.com> On 10/25/2012 2:13 PM, Bradley Rintoul wrote: > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their mail, > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > of sorts... There are already a gazillion email services that do mailbox consolidation--the term for what you're attempting. Why are you reinventing the wheel? Must be a family and the dog project... A quick browse of your Wordpress blog confirms such. It also explains why you wrote code to create maildir files from scratch. Only a programmer assumes that the first path to a solution is to write new code. A system administrator on the other hand, most of us here, Google's for suitable FOSS, then looks in his distro repos for it, and builds from source if a package isn't available. We only script when out of Elmer's and duct tape. ;) -- Stan From ben at morrow.me.uk Fri Oct 26 00:15:09 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 25 Oct 2012 22:15:09 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> Message-ID: <20121025211509.GE5388@anubis.morrow.me.uk> At 10AM +0300 on 25/10/12 you (Robert JR) wrote: > > Stan, sorry but you didnot understand my question at all, dovecot in > this case is reading the mailbox file while user downloading the mail > and not WRITING. only postfix write when a mail arrives and DOVECOT > only read the mail. And even if both write to the file, I have already > set the locking option of both to FCNTL so no problem should happen. > My question is postfix locking option is = FCNTL, DOTLOCK , and > dovecot = FCNTL, if postfix find a file already FCNTL, why it dotlock > the file 5 secs then remove the dotlock and say resource unaviable? As Stan said earlier, this is a Postfix question. The rule for dotlocking is that you must create the .lock *before* opening the file, in case whoever has it locked will be replacing the file altogether; but with fcntl locking you must acquire the lock *after* opening the file, since that's the way the syscall works. This means that if Postfix is going to use both forms of lock, it has to acquire a dotlock before it can look for a fcntl lock. In other words: the methods in mailbox_delivery_lock are *not* tried in order, because they can't be. Dotlock is always tried first. You should have compatible locking settings for all your programs accessing your mboxes. If Postfix is using dotlock, Dovecot should be using dotlock as well. If you don't have any local programs (mail clients, for instance) which require dotlocks, you should probably change Postfix to just use fcntl locks. Ben From ben at morrow.me.uk Fri Oct 26 00:24:03 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Thu, 25 Oct 2012 22:24:03 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <50897E38.6070304@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> Message-ID: <20121025212403.GF5388@anubis.morrow.me.uk> At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > > Yes, actually I did, but I missed one part of it because I assumed you > had Dovecot setup properly. > > It doesn't matter if the mbox locks are write or read or both. Locks > are the problem, period, because you have two daemons fighting over the > same files. The fix is absolutely trivial: > > Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > the file locking issue. Thus it also increases throughput as lock > latency is eliminated. Nonsense. deliver and imap are still separate processes accessing the same mbox, so they still need to use locks. The only difference is that since they are both dovecot programs, they will automatically be using the *same* locking strategies, and things will Just Work. Ben From brintoul at sbcglobal.net Fri Oct 26 01:02:05 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 15:02:05 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <5089A8C6.1090308@hardwarefreak.com> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <5089A8C6.1090308@hardwarefreak.com> Message-ID: <1351202525.21830.YahooMailRC@web184702.mail.ne1.yahoo.com> I didn't actually write code to create Maildir files from scratch, technically. I used 15-20 lines of Python to do that and to actually fetch the mail in addition... Mailbox consolidation isn't the only thing which this project I'm working on requires, but it's a big part of it. :) Anyway, I really appreciate all the advice and help! ________________________________ From: Stan Hoeppner To: dovecot at dovecot.org Sent: Thu, October 25, 2012 2:02:10 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process On 10/25/2012 2:13 PM, Bradley Rintoul wrote: > Let's say someone has an account with Yahoo, for example. My Python code is > fetching email from the user's Yahoo! account and placing it into the Dovecot > Maildir storage for a particular user. Now when the user retrieves their >mail, > > they are doing so using my Dovecot - my Dovecot instance is acting as a proxy, > > of sorts... There are already a gazillion email services that do mailbox consolidation--the term for what you're attempting. Why are you reinventing the wheel? Must be a family and the dog project... A quick browse of your Wordpress blog confirms such. It also explains why you wrote code to create maildir files from scratch. Only a programmer assumes that the first path to a solution is to write new code. A system administrator on the other hand, most of us here, Google's for suitable FOSS, then looks in his distro repos for it, and builds from source if a package isn't available. We only script when out of Elmer's and duct tape. ;) -- Stan From brintoul at sbcglobal.net Fri Oct 26 01:04:02 2012 From: brintoul at sbcglobal.net (Bradley Rintoul) Date: Thu, 25 Oct 2012 15:04:02 -0700 (PDT) Subject: [Dovecot] Creating Maildir and populating with emails via external Python process In-Reply-To: <20121025201059.GO3672@harrier.slackbuilds.org> References: <1351188113.88948.YahooMailRC@web184702.mail.ne1.yahoo.com> <697ejv7veev8@mids.svenhartge.de> <1351192414.82194.YahooMailRC@web184706.mail.ne1.yahoo.com> <508995D2.3070705@whyscream.net> <1351194883.4617.YahooMailRC@web184702.mail.ne1.yahoo.com> <20121025201059.GO3672@harrier.slackbuilds.org> Message-ID: <1351202642.93491.YahooMailRC@web184703.mail.ne1.yahoo.com> Cool. Thanks for the input. Can you tell me briefly why you'd choose getmail? Thanks again. B. RIntoul ________________________________ From: /dev/rob0 To: dovecot at dovecot.org Sent: Thu, October 25, 2012 1:11:13 PM Subject: Re: [Dovecot] Creating Maildir and populating with emails via external Python process > From: Tom Hendrikx > I'm intrigued by this. Why are you using some self-baked(?) python > script to fetch the mail in stead of using ready-made components > like fetchmail? > > Unless there's a special reason not to, try using the LDA (and > fetchmail/getmail for that matter). > > This sounds exactly what you want: > http://pyropus.ca/software/getmail/configuration.html#destination-mdaexternal > On Thu, Oct 25, 2012 at 12:54:43PM -0700, Bradley Rintoul wrote: > I am brand new to this whole "email" thing. I am looking at this > article right now: > http://www.tuxradar.com/content/get-started-fetchmail-procmail-and-dovecot I did not see where you described the ultimate goal. That should have been the starting point of this thread. Describe the problem, not how you think it should be solved, because you are new to this, and your ideas might benefit from some scrutiny. Use plain language. I have not reviewed your howto, but personally I would recommend neither fetchmail (I'd choose getmail) nor procmail (other choices exist, depending on what you are trying to do.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Fri Oct 26 01:33:48 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 17:33:48 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> <239f53fb233beb2b45e68638cca24260@Coptics.org> <50899EAB.2030003@hardwarefreak.com> Message-ID: <5089BE4C.4010709@hardwarefreak.com> On 10/25/2012 2:42 PM, Robert JR wrote: > Thanks again Stan, you are very helpfull, i will start learning how to > configure LDA, but hopefully i can also have an answer from Timo about > why this issue happened.. i am sure he is aware of it and can explain it.. Probably not. You describe a dot locking "problem" with Postfix. If you want information about that you need to ask on the Postfix list, not the Dovecot list. Wietse will answer you properly. Give it a subject "dot lock problem". -- Stan From jeff at bubble.org Fri Oct 26 01:57:47 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Thu, 25 Oct 2012 18:57:47 -0400 Subject: [Dovecot] Snarf plugin In-Reply-To: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> References: <0a50a53c7cbe03a7013f55bd1e317cb8.squirrel@xyzzy.bubble.org> Message-ID: <5089C3EB.1040301@bubble.org> Just thought I'd follow up on my original post, I got the snarf plugin to work properly with some help from Jonathan at PSU (need to give credit where credit is due) For anybody else looking for the configuration here is the relevant output from dovecot -n # 2.1.10: //etc/dovecot/dovecot.conf # OS: Linux 3.6.2-1.fc16.x86_64 x86_64 Fedora release 16 (Verne) ext4 mail_location = mbox:/home/%u/mail:INBOX=/home/%u/mbox mail_plugins = snarf zlib namespace Snarf { hidden = yes list = no location = mbox:/home/%u/mbox:INBOX=/var/spool/mail/%u:INDEX=MEMORY prefix = ~~Snarfbox/ separator = / } namespace default { inbox = yes location = prefix = separator = / } plugin { snarf = ~~Snarfbox/INBOX } What I found was I was specifying the snarf mbox location as ~/mbox, once I changed to /home/%u/mbox things started working. Thanks, Jeff From jeff at bubble.org Fri Oct 26 02:07:04 2012 From: jeff at bubble.org (Jeffrey Ross) Date: Thu, 25 Oct 2012 19:07:04 -0400 Subject: [Dovecot] Changing password for users In-Reply-To: <7a6a7cc64756b895059f7384b256019b@coptics.org> References: <7a6a7cc64756b895059f7384b256019b@coptics.org> Message-ID: <5089C618.6090605@bubble.org> On 10/25/2012 03:21 PM, Mike John wrote: > Hello, > > I am using dovecot (2.0.9) and using virtual users using > > passdb { > args = /etc/dovecot/dovecotpasswd > driver = passwd-file > } > > How can i make my virtual users change their passwords using web > interface ? > > My users already uses squirrelmail to access their mail. is there a > program to add to squirrelmail to add this function to the clients ? > or should i user different separate website for password changing ? > and what program/tool can help me with this ? > > Any ideas is greatly appreciated. > > Mike. Mike, I don't know about forcing users to change their passwords however with Squirrelmail there are several password change plugins available that use "poppasswd" to actually change the password. Of course poppasswd will probably need to be modified to go against your password data base, in my case it simply uses PAM. The version I use is poppassd version 1.8.5. Oh you probably want to restrict access to the port from the local host only since passwords are transmitted in clear text. Jeff From roundcube222 at alaadin.org Fri Oct 26 01:11:14 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Fri, 26 Oct 2012 01:11:14 +0300 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025211509.GE5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> Message-ID: <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> On 2012-10-26 00:15, Ben Morrow wrote: > At 10AM +0300 on 25/10/12 you (Robert JR) wrote: > >> Stan, sorry but you didnot understand my question at all, dovecot in >> this case is reading the mailbox file while user downloading the >> mail >> and not WRITING. only postfix write when a mail arrives and DOVECOT >> only read the mail. And even if both write to the file, I have >> already >> set the locking option of both to FCNTL so no problem should happen. >> My >> question is postfix locking option is = FCNTL, DOTLOCK , and dovecot >> = >> FCNTL, if postfix find a file already FCNTL, why it dotlock the file >> 5 >> secs then remove the dotlock and say resource unaviable? > > As Stan said earlier, this is a Postfix question. The rule for > dotlocking is that you must create the .lock *before* opening the > file, > in case whoever has it locked will be replacing the file altogether; > but > with fcntl locking you must acquire the lock *after* opening the > file, > since that's the way the syscall works. This means that if Postfix is > going to use both forms of lock, it has to acquire a dotlock before > it > can look for a fcntl lock. > > In other words: the methods in mailbox_delivery_lock are *not* tried > in > order, because they can't be. Dotlock is always tried first. > > You should have compatible locking settings for all your programs > accessing your mboxes. If Postfix is using dotlock, Dovecot should be > using dotlock as well. If you don't have any local programs (mail > clients, for instance) which require dotlocks, you should probably > change Postfix to just use fcntl locks. > > Ben Thanks Ben for your valuable support and detailed explanation .. but according to your explanation dovecot documentation contains wrong explanation to the dotlock and that's why i was confused. it is written as follow ** If multiple lock methods are used, which is usually the case since dotlocks aren't typically used for read locking, the order in which the locking is done is important. Consider if two programs were running at the same time, both use dotlock and fcntl locking but in different order: Program A: fcntl locks the mbox Program B at the same time: dotlocks the mbox Program A continues: tries to dotlock the mbox, but since it's already dotlocked by B, it starts waiting Program B continues: tries to fcntl lock the mbox, but since it's already fcntl locked by A, it starts waiting Now both of them are waiting for each others locks. Finally after a couple of minutes they time out and fail the operation. ** So this means that the documentation as mentioned above is wrong and dotlock is always first even if the order is different .. Anyway i have sent this question to postfix maillist and i will wait their reply . Thanks again Robert. From mike at alaadin.org Fri Oct 26 01:17:43 2012 From: mike at alaadin.org (Mike John) Date: Fri, 26 Oct 2012 01:17:43 +0300 Subject: [Dovecot] Changing password for users Message-ID: <02a35ba19c559b258dba0de278e31a4d@coptics.org> > Hello, > > I am using dovecot (2.0.9) and using virtual users using > > passdb { > args = /etc/dovecot/dovecotpasswd > driver = passwd-file > } > > How can i make my virtual users change their passwords using web > interface ? > > My users already uses squirrelmail to access their mail. is there a > program to add to squirrelmail to add this function to the clients ? > or should i user different separate website for password changing ? > and what program/tool can help me with this ? > > Any ideas is greatly appreciated. > > Mike. >Mike, >I don't know about forcing users to change their passwords however > with >Squirrelmail there are several password change plugins available that >use "poppasswd" to actually change the password. >Of course poppasswd will probably need to be modified to go against > your >password data base, in my case it simply uses PAM. The version I use > is >poppassd version 1.8.5. >Oh you probably want to restrict access to the port from the local > host >only since passwords are transmitted in clear text. >Jeff I know about poppassd , but it works only for /etc/passwd , /etc/shadow, but my dovecot virtual users password files are in different location and i do not know how to modify poppassd, any idea how can i do that? and is there another way other than poppassd? From stan at hardwarefreak.com Fri Oct 26 03:25:53 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 19:25:53 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121025212403.GF5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> Message-ID: <5089D891.9080207@hardwarefreak.com> On 10/25/2012 4:24 PM, Ben Morrow wrote: > At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >> >> Yes, actually I did, but I missed one part of it because I assumed you >> had Dovecot setup properly. >> >> It doesn't matter if the mbox locks are write or read or both. Locks >> are the problem, period, because you have two daemons fighting over the >> same files. The fix is absolutely trivial: >> >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >> the file locking issue. Thus it also increases throughput as lock >> latency is eliminated. > > Nonsense. deliver and imap are still separate processes accessing the > same mbox, so they still need to use locks. The only difference is that > since they are both dovecot programs, they will automatically be using > the *same* locking strategies, and things will Just Work. "Nonsense" implies what I stated was factually incorrect, which is not the case. There's a difference between factual incorrectness and simply staying out of the weeds. If you want to get into the weeds, and have me call you out for "nonsense", LDA/deliver is not a separate UNIX process. The LDA code runs within the imap process for the given user. This is what allows Dovecot to perform 'simultaneous' reads/writes to an mbox file, avoiding filesystem level locking latency. Using filesystem level locking to control read/write access between processes of own's program would be insane on many levels. -- Stan From simon.buongiorno at gmail.com Fri Oct 26 05:59:13 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Thu, 25 Oct 2012 22:59:13 -0400 Subject: [Dovecot] Changing password for users In-Reply-To: <02a35ba19c559b258dba0de278e31a4d@coptics.org> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> Message-ID: On Oct 25, 2012 7:20 PM, "Mike John" wrote: >> >> Hello, >> >> I am using dovecot (2.0.9) and using virtual users using >> >> passdb { >> args = /etc/dovecot/dovecotpasswd >> driver = passwd-file >> } >> >> How can i make my virtual users change their passwords using web >> interface ? >> >> My users already uses squirrelmail to access their mail. is there a >> program to add to squirrelmail to add this function to the clients ? >> or should i user different separate website for password changing ? >> and what program/tool can help me with this ? >> >> Any ideas is greatly appreciated. >> >> Mike. >> Mike, > > >> I don't know about forcing users to change their passwords however with >> Squirrelmail there are several password change plugins available that >> use "poppasswd" to actually change the password. > > >> Of course poppasswd will probably need to be modified to go against your >> password data base, in my case it simply uses PAM. The version I use is >> poppassd version 1.8.5. > > >> Oh you probably want to restrict access to the port from the local host >> only since passwords are transmitted in clear text. > > >> Jeff > > > I know about poppassd , but it works only for /etc/passwd , /etc/shadow, but my dovecot virtual users password files > are in different location and i do not know how to modify poppassd, any idea how can i do that? and is there another way other than poppassd? Horde has a change password module too. And essentially it's trivial to write your own php page to do it. I'll do it if you want to contract it out. Simon From stan at hardwarefreak.com Fri Oct 26 06:12:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 25 Oct 2012 22:12:34 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508983A9.9090605@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <508983A9.9090605@hardwarefreak.com> Message-ID: <5089FFA2.9050105@hardwarefreak.com> On 10/25/2012 1:23 PM, Stan Hoeppner wrote: > I forgot to mention one very important feature of Dovecot LDA: > > New messages delivered by Postfix are indexed by LDA as they are written > to the mailbox, flags updated at this time, etc. Thus when a mailbox is > opened in an IMAP MUA, new messages are displayed instantly (I don't use > POP but it's probably faster as well). With your current setup it can > take from a few to many seconds to show new mail, depending on message > count. With LDA new message count seems to have no impact on the speed > of display. Robert JR, you posted relevant information to the Postfix list that you omitted here, or at least I didn't see it. This may directly affect the advice myself and others gave you. You stated there that you're using Dovecot POP only and not IMAP. Given the nature of POP, using LDA may not help much even if it eliminates the filesystem locking contention between processes. I don't know if Dovecot will append an mbox file while reading the entire file. This Timo will have an answer to. Timo may also state, as he has before on this list, that: 1. there are better, faster, POPers available 2. Dovecot is developed primarily as an IMAP server 3. the POP function has received little development for quite some time -- Stan From ben at morrow.me.uk Fri Oct 26 06:54:59 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 04:54:59 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <5089D891.9080207@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> Message-ID: <20121026035458.GG5388@anubis.morrow.me.uk> At 7PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > On 10/25/2012 4:24 PM, Ben Morrow wrote: > > At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: > >> > >> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of > >> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates > >> the file locking issue. Thus it also increases throughput as lock > >> latency is eliminated. > > > > Nonsense. deliver and imap are still separate processes accessing the > > same mbox, so they still need to use locks. The only difference is that > > since they are both dovecot programs, they will automatically be using > > the *same* locking strategies, and things will Just Work. > > "Nonsense" implies what I stated was factually incorrect, which is not > the case. There's a difference between factual incorrectness and simply > staying out of the weeds. What you stated was factually incorrect. > If you want to get into the weeds, and have me call you out for > "nonsense", LDA/deliver is not a separate UNIX process. The LDA code > runs within the imap process for the given user. Nonsense. dovecot-lda runs in its own process, and does not involve the imap process in any way. As such it has to do locking. If I have the following in my dovecot.conf: mbox_read_locks = fcntl mbox_write_locks = dotlock fcntl namespace { location = mbox:~/mbox separator = / type = private list = yes prefix = MBOX/ } and I run ktrace dovecot-lda -f mauzo at localhost -m MBOX/foo < mail then the ktrace contains 44973 dovecot-lda CALL access(0x8021f5f68,0x6) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo" 44973 dovecot-lda RET access 0 [Check the mbox exists and is accessible] 44973 dovecot-lda CALL lstat(0x8020196c0,0x7fffffffcb60) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat -1 errno 2 No such file or directory [Look for a .lock file] 44973 dovecot-lda CALL open(0x8020196c0,0xa02, 0x1b6) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET open 9 44973 dovecot-lda CALL write(0x9,0x802019830,0x19) 44973 dovecot-lda RET write 25/0x19 44973 dovecot-lda CALL clock_gettime(0xd,0x7fffffffcbf0) 44973 dovecot-lda RET clock_gettime 0 44973 dovecot-lda CALL fstat(0x9,0x7fffffffcd90) 44973 dovecot-lda RET fstat 0 44973 dovecot-lda CALL close(0x9) 44973 dovecot-lda RET close 0 44973 dovecot-lda CALL lstat(0x8020196c0,0x7fffffffce60) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat 0 [Create a .lock file, and check it was successful] 44973 dovecot-lda CALL open(0x8021f5f68,0x2,0xfac3c0) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo" 44973 dovecot-lda RET open 9 [Open the mbox file itself] 44973 dovecot-lda CALL fcntl(0x9,F_SETLKW,0xffffffffffffcfd0) 44973 dovecot-lda RET fcntl 0 [Set a fcntl lock on the mbox file] 44973 dovecot-lda CALL pread(0,0x802031000,0x1000,0) 44973 dovecot-lda RET pread 43/0x2b 44973 dovecot-lda CALL write(0x9,0x802228000,0xf5) 44973 dovecot-lda RET write 245/0xf5 44973 dovecot-lda CALL fsync(0x9) 44973 dovecot-lda RET fsync 0 [Read from stdin and write to the mbox file] 44973 dovecot-lda CALL lstat(0x802028440,0x7fffffffd010) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET lstat 0 44973 dovecot-lda CALL unlink(0x802028440) 44973 dovecot-lda NAMI "/home/mauzo/mbox/foo.lock" 44973 dovecot-lda RET unlink 0 [Remove the .lock file] 44973 dovecot-lda CALL fcntl(0x9,F_SETLK,0xffffffffffffd160) 44973 dovecot-lda RET fcntl 0 [Clear the fcntl lock] 44973 dovecot-lda CALL close(0x9) 44973 dovecot-lda RET close 0 [Close the mbox file] > This is what allows > Dovecot to perform 'simultaneous' reads/writes to an mbox file, avoiding > filesystem level locking latency. Using filesystem level locking to > control read/write access between processes of own's program would be > insane on many levels. I'm not sure what you mean by 'processes of [one's own] program' but it's extremely common for a process to have to take locks against another copy of itself. All traditional Unix LDAs and MUAs do this; for instance, procmail will take locks in part so that if another instance of procmail is delivering another mail to the same user at the same time the mbox won't end up corrupted. Ben From ben at morrow.me.uk Fri Oct 26 07:16:28 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 05:16:28 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> Message-ID: <20121026041628.GH5388@anubis.morrow.me.uk> At 1AM +0300 on 26/10/12 you (Robert JR) wrote: > On 2012-10-26 00:15, Ben Morrow wrote: > > > As Stan said earlier, this is a Postfix question. The rule for [Looking back at the thread it wasn't Stan, it was Dennis Guhl. Sorry about that.] > > dotlocking is that you must create the .lock *before* opening the > > file, in case whoever has it locked will be replacing the file > > altogether; but with fcntl locking you must acquire the lock *after* > > opening the file, since that's the way the syscall works. This means > > that if Postfix is going to use both forms of lock, it has to > > acquire a dotlock before it can look for a fcntl lock. > > > > In other words: the methods in mailbox_delivery_lock are *not* tried > > in order, because they can't be. Dotlock is always tried first. > > > > You should have compatible locking settings for all your programs > > accessing your mboxes. If Postfix is using dotlock, Dovecot should be > > using dotlock as well. If you don't have any local programs (mail > > clients, for instance) which require dotlocks, you should probably > > change Postfix to just use fcntl locks. > > Thanks Ben for your valuable support and detailed explanation .. but > according to your explanation dovecot documentation contains wrong > explanation to the dotlock and that's why i was confused. > > it is written as follow > > ** > If multiple lock methods are used, which is usually the case since > dotlocks aren't typically used for read locking, the order in which the > locking is done is important. Consider if two programs were running at > the same time, both use dotlock and fcntl locking but in different > order: > > Program A: fcntl locks the mbox > Program B at the same time: dotlocks the mbox > Program A continues: tries to dotlock the mbox, but since it's already > dotlocked by B, it starts waiting > Program B continues: tries to fcntl lock the mbox, but since it's > already fcntl locked by A, it starts waiting > Now both of them are waiting for each others locks. Finally after a > couple of minutes they time out and fail the operation. > ** > > So this means that the documentation as mentioned above is wrong and > dotlock is always first even if the order is different .. I just checked, and you are right: Dovecot *will* use the locking strategies in the order listed. This is different from Postfix, which *will not*, so the 'Postfix' section of the MboxLocking page on the wiki is incorrect. For compatibility with Postfix, dotlock should always be listed first. Note that you will still frequently see Postfix acquiring a dotlock but then failing to acquire a fcntl lock. Dovecot's mbox_read_locks is usually set to just 'fcntl', which means that when Dovecot has an mbox open read-only it won't take a dotlock. This isn't really a problem, assuming you don't use any programs locally which only take dotlocks; but if that is the case you might as well configure everything to just use fcntl locks, and forget dotlocks altogether. Stan's earlier point is fundamentally correct: if you can treat the Dovecot mailstore as a black box, with mail going in through the LDA and LMTP and mail coming out through POP and IMAP, your life will be much easier. Traditional Unix mailbox locking strategies are *completely* insane, and if all you are doing is delivering mail from Postfix and reading it from Dovecot it would be better to avoid them altogether, and switch to dbox if you can. However, if you have any other programs which touch the mail spool (local or NFS mail clients, deliveries through procmail) this may not be possible. Ben From stan at hardwarefreak.com Fri Oct 26 09:24:00 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 01:24:00 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026035458.GG5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> Message-ID: <508A2C80.6010803@hardwarefreak.com> On 10/25/2012 10:54 PM, Ben Morrow wrote: > At 7PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >> On 10/25/2012 4:24 PM, Ben Morrow wrote: >>> At 1PM -0500 on 25/10/12 you (Stan Hoeppner) wrote: >>>> >>>> Switch Postfix to use the Dovecot Local Deliver Agent (LDA) in place of >>>> the Postfix local/virtual delivery agent. Using Dovecot LDA eliminates >>>> the file locking issue. Thus it also increases throughput as lock >>>> latency is eliminated. >>> >>> Nonsense. deliver and imap are still separate processes accessing the >>> same mbox, so they still need to use locks. The only difference is that >>> since they are both dovecot programs, they will automatically be using >>> the *same* locking strategies, and things will Just Work. >> >> "Nonsense" implies what I stated was factually incorrect, which is not >> the case. There's a difference between factual incorrectness and simply >> staying out of the weeds. > > What you stated was factually incorrect. > >> If you want to get into the weeds, and have me call you out for >> "nonsense", LDA/deliver is not a separate UNIX process. The LDA code >> runs within the imap process for the given user. > > Nonsense. dovecot-lda runs in its own process, and does not involve the > imap process in any way. As such it has to do locking. You apparently know your tools better than I do. Neither ps nor top show a 'dovecot-lda' or similarly named process on my systems. When I send a test message from gmail through Postfix I only see CPU or memory activity in an imap process. When I close the MUA to end the imap processes and then send a test message I don't see any CPU or memory activity in any dovecot processes, only Postfix processes, including local, and spamd. So is devecot-lda running as a sub-process or thread of Postfix' local process? Or is it part of the 'dovecot' process, and the message goes through so quick that top doesn't show any CPU usage by the 'dovecot' process? > If I have the following in my dovecot.conf: ... ... > I'm not sure what you mean by 'processes of [one's own] program' but I.e. Dovecot has its own set of processes, Postfix has its processes, etc. With "one's one processes" I'd think it makes more sense to use IPC and other tricks to accomplish concurrent access to a file rather than filesystem locking features. > it's extremely common for a process to have to take locks against > another copy of itself. All traditional Unix LDAs and MUAs do this; for > instance, procmail will take locks in part so that if another instance > of procmail is delivering another mail to the same user at the same time > the mbox won't end up corrupted. I guess I've given MDAs w/mbox too much credit, without actually looking at the guts. Scalable databases such Oracle, db2, etc, are far more intelligent about this, and can have many thousands of processes reading and writing the same file concurrently, usually via O_DIRECT, not buffered IO, so they have complete control over IO. This is accomplished with a record lock manager and IPC, preventing more than one process from accessing one record concurrently, but allowing massive read/write concurrency to multiple records in a file. I'd think the same concurrency optimization could be done with Dovecot. However, as Timo has pointed out, so few people use mbox these days that he simply hasn't spent much, if any, time optimizing mbox. Implementing some kind of lock manager and client code just for mbox IO concurrency simply wouldn't be worth the time. Unless he's already done something similar with mdbox. If he has, maybe that could be 'ported' to mbox as well. But again, it's probably not worth the effort given the number of mbox users, and the fact that nobody is complaining about mbox performance. I'm certainly not. It works great here. -- Stan From jg at softjury.de Fri Oct 26 09:49:04 2012 From: jg at softjury.de (Jan Phillip Greimann) Date: Fri, 26 Oct 2012 08:49:04 +0200 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <508A3260.3060803@softjury.de> I didn't know ADs well, but...can't you simply add the Field? In LDAP it should be possible, if you use MS AD, i dunno. Am 25.10.2012 22:49, schrieb b m:> No AD doesn't have such a field, but I could use some unused field to > get what I want. Let's say set "Attribute1" to "group1". The problem is > how to get that info. I guess I have to edit dovecot-ldap.conf and put > in user_attrs something like that ",=acl_groups=Attribute1". Any > suggestions? From stan at hardwarefreak.com Fri Oct 26 10:05:31 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 02:05:31 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026041628.GH5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <20121025211509.GE5388@anubis.morrow.me.uk> <87d68f21ba2bc3f0343bc1a0d587736c@Coptics.org> <20121026041628.GH5388@anubis.morrow.me.uk> Message-ID: <508A363B.3040100@hardwarefreak.com> On 10/25/2012 11:16 PM, Ben Morrow wrote: > At 1AM +0300 on 26/10/12 you (Robert JR) wrote: >> On 2012-10-26 00:15, Ben Morrow wrote: >> >>> As Stan said earlier, this is a Postfix question. The rule for > > [Looking back at the thread it wasn't Stan, it was Dennis Guhl. Sorry > about that.] I prodded him a second time, might have been off-list, and he finally posted there. So call it a team effort. ;) Wietse has already replied, and in typical fashion, asked for "concrete" evidence that Postfix was performing fcntl before dotlock, because he obviously knows better than anyone that Postfix applies a dotlock first, which you already explained here. >>> dotlocking is that you must create the .lock *before* opening the >>> file, in case whoever has it locked will be replacing the file >>> altogether; but with fcntl locking you must acquire the lock *after* >>> opening the file, since that's the way the syscall works. This means >>> that if Postfix is going to use both forms of lock, it has to >>> acquire a dotlock before it can look for a fcntl lock. >>> >>> In other words: the methods in mailbox_delivery_lock are *not* tried >>> in order, because they can't be. Dotlock is always tried first. >>> >>> You should have compatible locking settings for all your programs >>> accessing your mboxes. If Postfix is using dotlock, Dovecot should be >>> using dotlock as well. If you don't have any local programs (mail >>> clients, for instance) which require dotlocks, you should probably >>> change Postfix to just use fcntl locks. > but if that is the case you might as well configure everything to just > use fcntl locks, and forget dotlocks altogether. Yep. Postfix can use either or both. And, surprise, recommends using maildir to avoid mailbox locking entirely. > Stan's earlier point is fundamentally correct: if you can treat the > Dovecot mailstore as a black box, with mail going in through the LDA and > LMTP and mail coming out through POP and IMAP, your life will be much > easier. Traditional Unix mailbox locking strategies are *completely* > insane, and if all you are doing is delivering mail from Postfix and > reading it from Dovecot it would be better to avoid them altogether, and > switch to dbox if you can. However, if you have any other programs which > touch the mail spool (local or NFS mail clients, deliveries through > procmail) this may not be possible. And since this is a POP only server, users' MUAs should be deleting after download, so there shouldn't be much mail in these mbox files at any given time, making migration to maildir or dbox relatively simple. When using Dovecot LDA you'll eliminate the filesystem level locking problems with mbox. However, you may still have read/write contention within Dovecot, such as in your 20MB download as new mail arrives example, especially if the new message has an xx MB attachment. I don't believe Dovecot is going to start appending a new message while it's still reading out the existing 20MB of emails. Depending on how long this takes Dovecot may still issue a 4xx to Postfix, which will put the new message in the deferred queue. With maildir or dbox, reading existing mail and writing new messages occurs concurrently, as each message is a different file. -- Stan From ben at morrow.me.uk Fri Oct 26 11:11:20 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 09:11:20 +0100 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <508A2C80.6010803@hardwarefreak.com> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> <508A2C80.6010803@hardwarefreak.com> Message-ID: <20121026081120.GI5388@anubis.morrow.me.uk> At 1AM -0500 on 26/10/12 you (Stan Hoeppner) wrote: > On 10/25/2012 10:54 PM, Ben Morrow wrote: > > > > dovecot-lda runs in its own process, and does not involve the > > imap process in any way. As such it has to do locking. > > You apparently know your tools better than I do. Neither ps nor top > show a 'dovecot-lda' or similarly named process on my systems. When I > send a test message from gmail through Postfix I only see CPU or memory > activity in an imap process. When I close the MUA to end the imap > processes and then send a test message I don't see any CPU or memory > activity in any dovecot processes, only Postfix processes, including > local, and spamd. So is devecot-lda running as a sub-process or thread > of Postfix' local process? Or is it part of the 'dovecot' process, and > the message goes through so quick that top doesn't show any CPU usage by > the 'dovecot' process? Assuming you have mailbox_command = /.../dovecot-lda -a "${RECIPIENT}" or something equivalent in your Postfix configuration, dovecot-lda runs as a subprocess of local(8) under the uid of the delivered-to user. > > If I have the following in my dovecot.conf: > ... > > ... > > > I'm not sure what you mean by 'processes of [one's own] program' but > > I.e. Dovecot has its own set of processes, Postfix has its processes, > etc. With "one's one processes" I'd think it makes more sense to use > IPC and other tricks to accomplish concurrent access to a file rather > than filesystem locking features. Filesystem locking, at least if NFS is not involved, is not that expensive. Successfully acquiring a flock or fcntl lock takes only a single syscall which doesn't have to touch the disk, and any form of IPC is going to need to do that. (Even something like a shared memory region will need a mutex for synchronisation, and acquiring the mutex has to go through the kernel.) Dotlocking *is* expensive, because acquiring a dotlock is a complicated process requiring lots of syscalls, some of which have to write to disk; and any scheme involving acquiring several locks on the same file is going to be more so, especially if you can end up getting the first lock but finding you can't get the second, so then you have to undo the first and try again. More importantly, the biggest problem with mbox as a mailbox format is that any access at all has to lock the whole mailbox. If the LDA is trying to deliver a new message at the same time as an IMAP user is fetching a completely different message, or if two instances of the LDA are trying to deliver at the same time, they will be competing for the same lock even though they don't really need to be. A file-per-message format like Maildir avoids this, to the point of being mostly lockless, but that brings its own efficiency problems; the point of dbox is to find the compromise between these positions that works best. > > it's extremely common for a process to have to take locks against > > another copy of itself. All traditional Unix LDAs and MUAs do this; for > > instance, procmail will take locks in part so that if another instance > > of procmail is delivering another mail to the same user at the same time > > the mbox won't end up corrupted. > > I guess I've given MDAs w/mbox too much credit, without actually looking > at the guts. I wouldn't look too hard at the details of the various ways there are of locking and parsing mbox files, or the ways in which they can go wrong. It's enough to make anyone swear off email for life :). > Scalable databases such Oracle, db2, etc, are far more > intelligent about this, and can have many thousands of processes reading > and writing the same file concurrently, usually via O_DIRECT, not > buffered IO, so they have complete control over IO. This is > accomplished with a record lock manager and IPC, preventing more than > one process from accessing one record concurrently, but allowing massive > read/write concurrency to multiple records in a file. I'd think the > same concurrency optimization could be done with Dovecot. > > However, as Timo has pointed out, so few people use mbox these days that > he simply hasn't spent much, if any, time optimizing mbox. Implementing > some kind of lock manager and client code just for mbox IO concurrency > simply wouldn't be worth the time. Unless he's already done something > similar with mdbox. If he has, maybe that could be 'ported' to mbox as > well. But again, it's probably not worth the effort given the number of > mbox users, and the fact that nobody is complaining about mbox > performance. I'm certainly not. It works great here. The only reason for using mbox is for compatibility with other systems which use mbox, which means you have to do the locking the same way as they do (assuming you can work out what that is). If you're going to change the locking rules you might as well change the file format at the same time, both to remove the insanity and to make it actually suitable for use as an IMAP mailstore. That's what Timo did with dbox, so if you've got your systems to the point where nothing but Dovecot touches the mail files you should seriously consider switching. Ben From r.ordinas at math.univ-paris-diderot.fr Fri Oct 26 11:51:52 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Fri, 26 Oct 2012 10:51:52 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121025140852.GA15639@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> Message-ID: <508A4F28.80606@math.univ-paris-diderot.fr> Hi, Here's the doveconf -n output : # doveconf -n # 2.0.14: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.1-RELEASE-p5 amd64 auth_mechanisms = plain login auth_username_format = %Lu auth_worker_max_count = 90 default_process_limit = 1024 first_valid_gid = 1500 first_valid_uid = 1500 hostname = mailhost.mydomain.tld last_valid_gid = 1500 last_valid_uid = 1500 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * mail_gid = 1500 mail_location = maildir:~/Maildir mail_plugins = acl quota mail_log notify mail_privileged_group = mail mail_uid = 1500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/usr/local/etc/dovecot-acls:cache_secs=300 autocreate = Sent autocreate1 = Trash autocreate2 = Drafts autocreate3 = Spam autocreate4 = Faux-positif autosubscribe = Sent autosubscribe1 = Trash autosubscribe2 = Drafts autosubscribe3 = Spam autosubscribe4 = Faux-positif autosubscribe5 = INBOX mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_rule = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 quota_warning2 = storage=80%% quota-warning 80 sieve = ~/.dovecot.sieve sieve_before = /usr/local/lib/dovecot/sieve/backup-all.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at mydomain.tld protocols = imap lmtp sieve quota_full_tempfail = yes service anvil { client_limit = 3500 } service auth-worker { user = $default_internal_user } service auth { client_limit = 5500 unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = doveauth } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 172.0.0.1 port = 2525 } } service managesieve-login { inet_listener sieve_deprecated { port = 2000 } process_limit = 1024 } service managesieve { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } shutdown_clients = no ssl = required ssl_ca = On Thu, Oct 25, 2012 at 03:09:47PM +0200, Raphael Ordinas wrote: >> When sending mail to MTA (in case of sieve filter forwarding for >> example), dovecot pass a RCPT TO command just after the EHLO. He's >> missing the MAIL FROM command. >> Therefore, my MTA show me a warning like this : "improper command >> pipelining after EHLO". > Works for me with latest and greatest although I'm not using the > 'submission_host' option but pure LMTP Unix socket: > > [...] > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > > Best to show your 'doveconf -n' for more thoughts. > > Regards > Thomas From ben at morrow.me.uk Fri Oct 26 11:54:56 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Fri, 26 Oct 2012 09:54:56 +0100 Subject: [Dovecot] Public folders and groups In-Reply-To: <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> References: <1351116794.87068.YahooMailNeo@web125704.mail.ne1.yahoo.com> <50891609.9070709@softjury.de> <1351198192.82671.YahooMailNeo@web125702.mail.ne1.yahoo.com> Message-ID: <20121026085456.GJ5388@anubis.morrow.me.uk> At 1PM -0700 on 25/10/12 b m wrote: > From: Jan Phillip Greimann >> Am 25.10.2012 00:13, schrieb b m: >> >> > Currently I have dovecot working with Active Directory >> > authentication and public folders with acl. In acl I have the users >> > I want to access the public folders. It'll be easier for me to use >> > one group instead of 50 users but I can't get it to work. From where >> > does dovecot get the "group" attribute for a user? Can it read the >> > groups that a user belongs from AD? >> >> ACL groups support works by returning a comma-separated acl_groups >> extra field from userdb, which contains all the groups the user >> belongs to. >> >> It seems to be possible, I had an acl_groups field in my MySQL >> Database for this, I'am sure it is something like that in an AD too. > > No AD doesn't have such a field, but I could use some unused field to > get what I want. Let's say set "Attribute1" to "group1". The problem > is how to get that info. I guess I have to edit dovecot-ldap.conf and > put in user_attrs something like that ",=acl_groups=Attribute1". Any > suggestions? That's the wrong way around. Assuming you created an 'imapGroups' attribute containing a comma-separated list of IMAP groups, you would want to add 'imapGroups=acl_groups' to user_attrs. Alternatively, if you don't want to duplicate the information in the LDAP directory, you can use post-login scripting to set up the groups list (see http://wiki2.dovecot.org/PostLoginScripting). If you have your system set up with nss_ldap or winbind so that AD users show up as system users with their proper groups, the example on the wiki using the 'groups' command will work. Otherwise, you can pull the information directly from LDAP, something like #!/bin/sh do_ldap () { /usr/local/bin/ldapsearch -h \ "(&(objectClass=$1)($2))" $3 \ | sed -ne"s/^$3: //p" } user_dn="$(do_ldap User "sAMAccountName=$USER" dn)" ACL_GROUPS="$(do_ldap Group "member=$user_dn" cn | paste -sd, -)" export ACL_GROUPS export USERDB_KEYS="$USERDB_KEYS acl_groups" exec "$@" Obviously you will need to adjust the path and connection parameters for ldapsearch to suit your environment; also, I don't use AD, so you may need to adjust the LDAP search. (If you prefer it might be easier to do this in Perl or Python or something rather than shell.) Ben From tlx at leuxner.net Fri Oct 26 12:00:12 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 26 Oct 2012 11:00:12 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <508A4F28.80606@math.univ-paris-diderot.fr> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> Message-ID: <20121026090012.GA31929@nihlus.leuxner.net> On Fri, Oct 26, 2012 at 10:51:52AM +0200, Raphael Ordinas wrote: > service lmtp { > inet_listener lmtp { > address = 172.0.0.1 > port = 2525 > } > } Right, so you are using network sockets with LMTP. Probably does not answer the question why it is not working with the 'submission_host', but is there a reason why the redirects are not reinjected this way? > submission_host = smtp.mydomain.tld Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From busseniu at in.tum.de Fri Oct 26 12:17:44 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Fri, 26 Oct 2012 11:17:44 +0200 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 In-Reply-To: <5085593D.3080403@in.tum.de> References: <5085593D.3080403@in.tum.de> Message-ID: <508A5538.8080604@in.tum.de> Hi, On 22.10.2012 16:33, Christoph Bu?enius wrote: > . list "" INBOX.shared.%.% > > Dovecot 2.1.10 does not list any folders in response to this command. I hope this helps: I bisected this bug and found it was introduced with this changeset: http://hg.dovecot.org/dovecot-2.1/rev/a41f64348d0d changeset: 14453:a41f64348d0d user: Timo Sirainen date: Fri Apr 20 15:18:14 2012 +0300 files: src/lib-storage/list/mailbox-list-fs-iter.c description: layout=fs: Don't assume '/' hierarchy separator when finding mailbox roots. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From tlx at leuxner.net Fri Oct 26 12:31:34 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 26 Oct 2012 11:31:34 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121026090012.GA31929@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> <20121026090012.GA31929@nihlus.leuxner.net> Message-ID: <20121026093134.GB31929@nihlus.leuxner.net> On Fri, Oct 26, 2012 at 11:00:12AM +0200, Thomas Leuxner wrote: > submission_host = smtp.mydomain.tld On second thought, above probably overrides this: # doveconf -a | grep sendmail sendmail_path = /usr/sbin/sendmail ...which may be the culprit. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From r.ordinas at math.univ-paris-diderot.fr Fri Oct 26 12:59:06 2012 From: r.ordinas at math.univ-paris-diderot.fr (Raphael Ordinas) Date: Fri, 26 Oct 2012 11:59:06 +0200 Subject: [Dovecot] Small issue with "submission host" In-Reply-To: <20121026090012.GA31929@nihlus.leuxner.net> References: <50893A1B.2060205@math.univ-paris-diderot.fr> <20121025140852.GA15639@nihlus.leuxner.net> <508A4F28.80606@math.univ-paris-diderot.fr> <20121026090012.GA31929@nihlus.leuxner.net> Message-ID: <508A5EEA.2000705@math.univ-paris-diderot.fr> Actually, LMTP inet listener is only used for delivery purpose. I separated the MTA and the MDA on distinct hosts. Incomming mails are received by the MTA which proceed to some check (anti-virus, spams, and aliases) and transport them to the MDA with LMTP. Maybe I misunderstood something, but i don't see why LMTP is involve in a sieve forwarding process (or stuff like non delivery mail return) . According to comments in the "15-lda.conf" file : # Binary to use for sending mails. #sendmail_path = /usr/sbin/sendmail # If non-empty, send mails via this SMTP host[:port] instead of sendmail. submission_host = smtp.mydomain.tld If you don't use the 'submission_host' option, dovecot will forward mail with '/usr/sbin/sendmail' binary which use the forwarders you tell it to use, am i right ? Regards, Raphael Le 26/10/2012 11:00, Thomas Leuxner a ?crit : > On Fri, Oct 26, 2012 at 10:51:52AM +0200, Raphael Ordinas wrote: > >> service lmtp { >> inet_listener lmtp { >> address = 172.0.0.1 >> port = 2525 >> } >> } > Right, so you are using network sockets with LMTP. Probably does not > answer the question why it is not working with the 'submission_host', > but is there a reason why the redirects are not reinjected this way? > >> submission_host = smtp.mydomain.tld > Regards > Thomas From tss at iki.fi Fri Oct 26 13:07:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 26 Oct 2012 13:07:49 +0300 Subject: [Dovecot] Shared folders not shown if "INBOX.shared.%.%" is used with dovecot 2.1.10 In-Reply-To: <508A5538.8080604@in.tum.de> References: <5085593D.3080403@in.tum.de> <508A5538.8080604@in.tum.de> Message-ID: On 26.10.2012, at 12.17, Christoph Bu?enius wrote: > On 22.10.2012 16:33, Christoph Bu?enius wrote: >> . list "" INBOX.shared.%.% >> >> Dovecot 2.1.10 does not list any folders in response to this command. > > I hope this helps: I bisected this bug and found it was introduced with this changeset: > > http://hg.dovecot.org/dovecot-2.1/rev/a41f64348d0d I couldn't reproduce this exactly and I don't see how a41f64348d0d makes any difference .. but I did find another way to reproduce at least a similar bug. Maybe this fixes your problem too? http://hg.dovecot.org/dovecot-2.1/rev/22875bcaa952 From fabio.ferrari at unimore.it Fri Oct 26 13:24:42 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Fri, 26 Oct 2012 12:24:42 +0200 Subject: [Dovecot] Dovecot stops to work - anvil problem Message-ID: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> Hi all, we have a problem about anvil, it seems that when we have a high load the dovecot stops to work. Sometimes it is sufficient to make a dovecot reload, but sometimes we have to restart it. These are the lines related to anvil in the dovecot.log: [root at secchia ~]# grep anvil /var/log/dovecot.log | more Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files Oct 26 11:14:32 imap-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:32 imap-login: Fatal: Couldn't connect to anvil Oct 26 11:14:33 pop3-login: Error: net_connect_unix(anvil) failed: Resource temporarily unavailable Oct 26 11:14:33 pop3-login: Fatal: Couldn't connect to anvil [...] (many lines like these) Oct 26 12:01:10 pop3-login: Fatal: Couldn't connect to anvil Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused Oct 26 12:01:18 auth: Error: read(anvil-auth-penalty) failed: Connection reset by peer Oct 26 12:01:18 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Connection refused And this is the output of the doveconf -n: [root at secchia ~]# doveconf -n # 2.0.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.11.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.8 (Tikanga) xfs auth_cache_size = 1024 auth_cache_ttl = 21600 s auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no hostname = mail.unimore.it info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:/cl/mail/vhosts/sms.unimo.it/%Ln/Maildir mail_plugins = $mail_plugins quota mailbox_idle_check_interval = 60 s mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/etc/dovecot.masterusers driver = passwd-file master = yes } passdb { args = dovecot driver = pam } plugin { quota = maildir:User quota quota_exceeded_message = Quota exceeded (mailbox is full) quota_rule = *:storage=200MB quota_rule2 = *:messages=100000 quota_rule3 = INBOX.Trash:storage=+100M quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = messages=95%% quota-warning 95 %u quota_warning4 = messages=80%% quota-warning 80 %u setting_name = quota } postmaster_address = postmaster at unimore.it quota_full_tempfail = yes service anvil { client_limit = 199999 process_limit = 199999 } service auth { client_limit = 14500 unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } process_limit = 5000 } service imap { process_limit = 5000 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 } } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail user = vmail } user = dovecot } ssl_ca = References: <5085593D.3080403@in.tum.de> <508A5538.8080604@in.tum.de> Message-ID: <508A668D.4070505@in.tum.de> Hello Timo, On 26.10.2012 12:07, Timo Sirainen wrote: > but I did find another way to reproduce at least a similar bug. Maybe this fixes your problem too? http://hg.dovecot.org/dovecot-2.1/rev/22875bcaa952 That does fix the problem, thank you! Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.040 <> Boltzmannstr. 3 <> Garching From dale.gallagher at gmail.com Fri Oct 26 14:27:00 2012 From: dale.gallagher at gmail.com (Dale Gallagher) Date: Fri, 26 Oct 2012 13:27:00 +0200 Subject: [Dovecot] dovecot-lda delivery to Maildir/cur as 'seen'? Message-ID: Hi I've added a server-side feature where authenticated customers sending through our SMTP server have their outbound mail copied to their Sent folder (like Gmail does). The delivery script called by qmail calls dovecot-lda to deliver it to the user's Sent folder. The problem now, is that the Sent folder shows the mail as unread, which MUAs flag (and notify, in the case of some). I've searched the docs and mailing list, but can't find an option to tell dovecot-lda to mark the mail being delivered, as seen/read. If I've missed something, please let me know. If not, then I think it might be a good idea to add a feature to dovecot-lda permitting one to specify delivery to the ./cur subfolder of a Maildir, instead of ./new. Thanks From tony.blue.mailinglist at gmx.de Fri Oct 26 14:44:55 2012 From: tony.blue.mailinglist at gmx.de (tony.blue.mailinglist at gmx.de) Date: Fri, 26 Oct 2012 13:44:55 +0200 Subject: [Dovecot] dovecot lda - Permission denied Message-ID: <20121026114455.30440@gmx.net> Hallo, please excuse my bad english. But I am not a native speaker. I changed my cyrus to dovecot (alltogehter: fetchmail - procmail - exim4 - dovecot). But I get (I think from /usr/lib/dovecot/deliver) the following error-message in my syslog: ... Oct 25 23:37:13 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(andy) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) ... Oct 25 23:37:14 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=500(tony) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) ... Dovecot is installed as !include auth-passwdfile.conf.ext. For all users there is a entry in der /etc/dovecot/users. Usaly the user rights are set to 600. I tryed 755, but I get the same errormessage. ... service auth { unix_listener auth-userdb { mode = 0755 user = mailstore group = mailstore } ... If I try "ls /var/run/dovecot/auth-userdb -la" - i get: srwxr-xr-x 1 mailstore mailstore 0 Okt 25 23:36 /var/run/dovecot/auth-userdb How can I solve this problem? Tony From CMarcus at Media-Brokers.com Fri Oct 26 16:37:59 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 26 Oct 2012 09:37:59 -0400 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5088B691.7030100@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <5087FEED.7060007@Media-Brokers.com> <5088B691.7030100@hardwarefreak.com> Message-ID: <508A9237.7080903@Media-Brokers.com> On 2012-10-24 11:48 PM, Stan Hoeppner wrote: > Changing the process priority would not help. Indexing a large mailbox > is an IO bound, not a compute bound, operation. With Linux, changing > from the CFQ to deadline scheduler may help some with low > responsiveness. But the only real solution for such a case where iowait > is bringing the system to its knees is to acquire storage with far > greater IOPS and concurrent IO capability. I.e. a server. Ok, I get it, thanks for elaborating Stan... -- Best regards, Charles From dg at dguhl.org Fri Oct 26 17:01:41 2012 From: dg at dguhl.org (Dennis Guhl) Date: Fri, 26 Oct 2012 16:01:41 +0200 Subject: [Dovecot] dovecot-lda delivery to Maildir/cur as 'seen'? In-Reply-To: References: Message-ID: <20121026140141.GA6769@PC211.ikt.de> On Fri, Oct 26, 2012 at 01:27:00PM +0200, Dale Gallagher wrote: > Hi [..] > The problem now, is that the Sent folder shows the mail as unread, > which MUAs flag (and notify, in the case of some). I've searched the Use Sieve [1] with Imap4flags (RFC 5232) to mark the email as read. Dennis [1] http://wiki2.dovecot.org/Pigeonhole/Sieve [..] From listen at mjh.name Fri Oct 26 21:28:51 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:28:51 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087D457.6040205@sys4.de> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087D457.6040205@sys4.de> Message-ID: <20121026202851.dc0abb3d5e4a4dd5c32d2d6c@mjh.name> On Wed, 24 Oct 2012 13:43:19 +0200 Robert Schetterer wrote: > Am 24.10.2012 13:28, schrieb Milan Holz?pfel: > > The whole mdbox is 6.6 GiB large and I guess that it contains about > > 300k-600k messages. It's an archive of public mailing lists, so I could > > give access to the files. > > > > Can anybody say something about this? May the mdbox be repaired? > > perhaps this helps > > http://wiki2.dovecot.org/Tools/Doveadm/ForceResync > > however upgrading to dovecot latest might be a good idea I tried this command, but all it will do is the "rebuilding indexes" thing that Dovecot's deliver and imapd will also do. (As I mentioned, this fails.) I haven't tried a more recent version of Dovecot so far. Regards, Milan Holz?pfel -- Milan Holz?pfel From listen at mjh.name Fri Oct 26 21:29:15 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:29:15 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <5087F4B4.2060107@hardwarefreak.com> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> Message-ID: <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> On Wed, 24 Oct 2012 09:01:24 -0500 Stan Hoeppner wrote: > On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: > > > I have a problem with an incosistent mdbox: > ... > > four hours after the problem initially appeared, I did a hard reset of > > the system because it was unresponsive. > ... > > Can anybody say something about this? May the mdbox be repaired? > > If the box is truly unresponsive, i.e. hard locked, then the corrupted > indexes are only a symptom of the underlying problem, which is unrelated > to Dovecot, UNLESS, the lack of responsiveness was due to massive disk > access, which will occur when rebuilding indexes on a 6.6GB mailbox. > You need to know the difference so we have accurate information to > troubleshoot with. Thanks for your suggestion. I wasn't looking for a solution for the unresponsiveness, but I failed to make that clear. I was not patient enough to debug the unresponsiveness issue. The box was not hard locked, but any command took very look if it would at all complete. I think that it could be massive swapping, but I wouldn't expect Dovecot to be the cause. After the reboot, Dovecot would happily re-execute the failing index rebuild on each new incoming message, which suggests that Dovecot wasn't the cause for the unresponsiveness. > If the there's a kernel or hardware problem, you should see related > errors in dmesg. Please share those. The kernel had messages like INFO: task cron:2799 blocked for more than 120 seconds. in the dmesg. But again, I didn't mean to ask for a solution to this problem. Regards, Milan Holz?pfel -- Milan Holz?pfel From listen at mjh.name Fri Oct 26 21:30:24 2012 From: listen at mjh.name (Milan =?ISO-8859-1?Q?Holz=E4pfel?=) Date: Fri, 26 Oct 2012 20:30:24 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> On Wed, 24 Oct 2012 13:28:11 +0200 Milan Holz?pfel wrote: > I have a problem with an incosistent mdbox: > [...] > The problem appeared out of nowhere. [...] That's just wrong. Two minutes before the corruption occured for the first time, the machine was booted after power-off without prior shutdown. I didn't notice this until now, sorry for this. The mailbox is on XFS. As far as I remember, XFS in known for leaving NULL bytes at the end of files after a system reset. At least, I found 72 bytes of NULL in a plain text log file on XFS after such an event. Do you think this may be the source of the index corruption? Do you have any other suggestions for recovering the mailbox? Regards, Milan Holz?pfel -- Milan Holz?pfel From fxmulder at gmail.com Fri Oct 26 22:13:33 2012 From: fxmulder at gmail.com (James Devine) Date: Fri, 26 Oct 2012 13:13:33 -0600 Subject: [Dovecot] Overlapping userdb/passdbs Message-ID: I have an ldap server for which each entry includes the email address and the username portion of the email address for authentication. Authentication works by username if the username is unique among all the entries. I need to now add some users which must authenticate even if the username is not unique. I figured one way to do this would be to add a second user/pass db which puts further restrictions on the ldap query to make it unique for those users. This doesn't seem to work however as if the user is found in the first ldap query but the password does not match it does not try the second. I would use the password as part of the query but this setup requires me to allow the client to hash the password. Is there a way to do this? Or maybe I am approaching the problem wrong. From mike at alaadin.org Fri Oct 26 21:47:44 2012 From: mike at alaadin.org (Mike John) Date: Fri, 26 Oct 2012 21:47:44 +0300 Subject: [Dovecot] Changing password for users In-Reply-To: <02a35ba19c559b258dba0de278e31a4d@coptics.org> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> Message-ID: <7827e2e2d9aa524945d00575c3366400@coptics.org> On 2012-10-26 01:17, Mike John wrote: >> Hello, I am using dovecot (2.0.9) and using virtual users using >> passdb >> { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i >> make my virtual users change their passwords using web interface ? >> My >> users already uses squirrelmail to access their mail. is there a >> program to add to squirrelmail to add this function to the clients ? >> or >> should i user different separate website for password changing ? and >> what program/tool can help me with this ? Any ideas is greatly >> appreciated. Mike. Mike, > >> I don't know about forcing users to change their passwords however >> with >> Squirrelmail there are several password change plugins available >> that >> use "poppasswd" to actually c> ssword. Of course poppasswd will > probably need to be modified to go >> against your password data base, in my case it simply uses PAM. The >> version I> sion 1.8.5. Oh you probably want to restrict access to >> the > port from >> the local host only since pas > ansmitted in clear > >> quot > e>Jeff > > I know about poppassd , but it works only for /etc/passwd , > /etc/shadow, but my dovecot virtual users password files > are in different location and i do not know how to modify poppassd, > any > idea how can i do that? and is there another way other than poppassd? i have googled every where, i can not find how to modify poppassd to modify virtual users passwords at /etc/dovecot/passwords , Is there any other way ? i am sure that some one in this mailing list have virtual users and uses modified poppassd or other utils so that his clients can change their password From dave.mehler at gmail.com Fri Oct 26 23:34:46 2012 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 26 Oct 2012 16:34:46 -0400 Subject: [Dovecot] public mailbox not showing up in web client Message-ID: Hello, I'm trying to set up a public mailbox where users can receive notifications out of. I'm not getting any errors from Dovecot 2.1, but nothing is showing up in my user's web clients. In each /home/vmail/public/mailbox folder right now I just have one called testbox I have a dovecot-acl file with: user=testuser1 lr user=user1 lr etc. I'd appreciate any suggestions. Thanks. Dave. # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 5000 first_valid_uid = 5000 hostname = xxx last_valid_gid = 5000 last_valid_uid = 5000 log_path = /var/log/dovecot.error mail_gid = vmail mail_home = /home/vmail/%d/%n/home mail_location = maildir:/home/vmail/%d/%n:LAYOUT=fs mail_plugins = " acl quota zlib" mail_uid = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = yes location = maildir:/home/vmail/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace inbox { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile autocreate = Spam autosubscribe = Spam quota = dict:User quota::proxy::quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } postmaster_address = postmaster at xxx protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = xxx xxx ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_cert = References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> <7827e2e2d9aa524945d00575c3366400@coptics.org> Message-ID: <508AFACD.8050807@whyscream.net> On 26-10-12 20:47, Mike John wrote: > On 2012-10-26 01:17, Mike John wrote: > >>> Hello, I am using dovecot (2.0.9) and using virtual users using passdb >>> { args = /etc/dovecot/dovecotpasswd driver = passwd-file } How can i >>> make my virtual users change their passwords using web interface ? My >>> users already uses squirrelmail to access their mail. is there a >>> program to add to squirrelmail to add this function to the clients ? or >>> should i user different separate website for password changing ? and >>> what program/tool can help me with this ? Any ideas is greatly >>> appreciated. Mike. Mike, >> >>> I don't know about forcing users to change their passwords however with >>> Squirrelmail there are several password change plugins available that >>> use "poppasswd" to actually c> ssword. Of course poppasswd will >> probably need to be modified to go >>> against your password data base, in my case it simply uses PAM. The >>> version I> sion 1.8.5. Oh you probably want to restrict access to the >> port from >>> the local host only since pas >> ansmitted in clear >> >>> quot >> e>Jeff >> >> I know about poppassd , but it works only for /etc/passwd , >> /etc/shadow, but my dovecot virtual users password files >> are in different location and i do not know how to modify poppassd, any >> idea how can i do that? and is there another way other than poppassd? > > i have googled every where, i can not find how to modify poppassd to > modify virtual users passwords at /etc/dovecot/passwords > , Is there any other way ? i am sure that some one in this mailing list > have virtual users and uses modified poppassd or other utils so that his > clients can change their password Using a database for managing virtual users seems overkill, until you run into issues like this. I have a postgres backend for 20ish users, and I can plugin everything I want. Postfixadmin works geat, and there are many password plugins for squirrelmail/roundcube/etc that work with such a database. Disclaimer: I tried the file-based approach too, but kept building kludges for things that were a lot simpler with a database. In the end, I joined the dark side. -- Tom From jtam.home at gmail.com Sat Oct 27 01:19:31 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 26 Oct 2012 15:19:31 -0700 (PDT) Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: > From: Mike John > >> I know about poppassd , but it works only for /etc/passwd , >> /etc/shadow, but my dovecot virtual users password files >> are in different location and i do not know how to modify poppassd, >> any idea how can i do that? I downloaded and examined it; it's just a wrapper for /usr/bin/passwd, and there doesn't seem an easy way to modify it to use something other than the system password file. Maybe replace "/usr/bin/passwd" with htpasswd? > and is there another way other than poppassd? Write your own PHP script -- it couldn't be more than a few dozen lines of code for a working skeleton. Or Google "php change password htpasswd". Joseph Tam From ben at morrow.me.uk Sat Oct 27 02:09:11 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Sat, 27 Oct 2012 00:09:11 +0100 Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: <20121026230910.GK5388@anubis.morrow.me.uk> At 3PM -0700 on 26/10/12 you (Joseph Tam) wrote: > > > From: Mike John > > > >> I know about poppassd , but it works only for /etc/passwd , > >> /etc/shadow, but my dovecot virtual users password files > >> are in different location and i do not know how to modify poppassd, > >> any idea how can i do that? > > I downloaded and examined it; it's just a wrapper for /usr/bin/passwd, > and there doesn't seem an easy way to modify it to use something other > than the system password file. > > Maybe replace "/usr/bin/passwd" with htpasswd? Try pam_pwdfile with poppwd or some other poppassd that supports PAM. > > and is there another way other than poppassd? > > Write your own PHP script -- it couldn't be more than a few dozen lines > of code for a working skeleton. Or Google "php change password htpasswd". It's not as simple as you seem to think. Quite apart from getting the password-changing itself right (have you considered what happens when two users change their passwords at the same time? when Dovecot tries to read the password file at the same time as you are changing it? when the system crashes when you are halfway through rewriting the password file?), you really shouldn't be running PHP as a user with write access to a password file (even a virtual password file) in any case. Ben From rob0 at gmx.co.uk Sat Oct 27 03:26:46 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Fri, 26 Oct 2012 19:26:46 -0500 Subject: [Dovecot] Changing password for users In-Reply-To: <508AFACD.8050807@whyscream.net> References: <02a35ba19c559b258dba0de278e31a4d@coptics.org> <7827e2e2d9aa524945d00575c3366400@coptics.org> <508AFACD.8050807@whyscream.net> Message-ID: <20121027002646.GS3672@harrier.slackbuilds.org> On Fri, Oct 26, 2012 at 11:04:13PM +0200, Tom Hendrikx wrote: > Using a database for managing virtual users seems overkill, > until you run into issues like this. > > I have a postgres backend for 20ish users, and I can plugin > everything I want. Postfixadmin works geat, and there are many > password plugins for squirrelmail/roundcube/etc that work with > such a database. > > Disclaimer: I tried the file-based approach too, but kept > building kludges for things that were a lot simpler with a > database. In the end, I joined the dark side. SQLite gives me the best of both worlds: file-based stability with SQL flexibility and easy backups. There is no Postfixadmin-type solution out there yet, but if you're fine with sqlite3(1) in the console, you won't miss it. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From stan at hardwarefreak.com Sat Oct 27 03:49:32 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 19:49:32 -0500 Subject: [Dovecot] Locking /var/mail/user issue with postfix and dovecot In-Reply-To: <20121026081120.GI5388@anubis.morrow.me.uk> References: <4c9608dd97036a48885e68205922d6b2@Coptics.org> <5088B8BB.9000109@hardwarefreak.com> <50897E38.6070304@hardwarefreak.com> <20121025212403.GF5388@anubis.morrow.me.uk> <5089D891.9080207@hardwarefreak.com> <20121026035458.GG5388@anubis.morrow.me.uk> <508A2C80.6010803@hardwarefreak.com> <20121026081120.GI5388@anubis.morrow.me.uk> Message-ID: <508B2F9C.2050706@hardwarefreak.com> You are a well of accessible knowledge Ben. (How have I missed your posts in the past?) On 10/26/2012 3:11 AM, Ben Morrow wrote: > Assuming you have > > mailbox_command = /.../dovecot-lda -a "${RECIPIENT}" I'm setup for system users so it's a simpler, but yes. > or something equivalent in your Postfix configuration, dovecot-lda runs > as a subprocess of local(8) under the uid of the delivered-to user. Of course that makes sense given Postfix is doing the calling. I would have assumed this but my feeble use of tools wasn't showing anything. > Filesystem locking, at least if NFS is not involved, is not that > expensive. Successfully acquiring a flock or fcntl lock takes only a > single syscall which doesn't have to touch the disk, and any form of IPC > is going to need to do that. (Even something like a shared memory region > will need a mutex for synchronisation, and acquiring the mutex has to go > through the kernel.) Thanks for this. I was under the assumption flock/fcntl were more expensive than they are. Probably because all I'd read about them was in relation to NFS (which I don't use, but I read alot like many do). > Dotlocking *is* expensive, because acquiring a dotlock is a complicated > process requiring lots of syscalls, some of which have to write to disk; > and any scheme involving acquiring several locks on the same file is > going to be more so, especially if you can end up getting the first lock > but finding you can't get the second, so then you have to undo the first > and try again. Yeah, I knew dotlocks were the worst due to disk writes, but didn't know the other details. > More importantly, the biggest problem with mbox as a mailbox format is > that any access at all has to lock the whole mailbox. If the LDA is > trying to deliver a new message at the same time as an IMAP user is > fetching a completely different message, or if two instances of the LDA > are trying to deliver at the same time, they will be competing for the > same lock even though they don't really need to be. A file-per-message > format like Maildir avoids this, to the point of being mostly lockless, > but that brings its own efficiency problems; the point of dbox is to > find the compromise between these positions that works best. mbox locking hasn't been problem here as I split the INBOX from the user mailboxes containing IMAP folders (mbox files). We make heavy use of sieve scripts to sort on delivery, so there's not much concurrent access to any one mbox file. The efficiency issue is why I chose mbox over maildir. Users here keep a lot of (list) mail and FTS often. The load on the spindles with maildir is simply too great and would bog down all users. The IOPS benefit of mbox in this scenario outweighs any locking issues. > I wouldn't look too hard at the details of the various ways there are of > locking and parsing mbox files, or the ways in which they can go wrong. > It's enough to make anyone swear off email for life :). Heheh. > The only reason for using mbox is for compatibility with other systems > which use mbox, Not necessarily true. See above. I'm sure I'm not the only one using mbox for this reason. Dovecot is my only app hitting these mbox files. > which means you have to do the locking the same way as > they do (assuming you can work out what that is). If you're going to > change the locking rules you might as well change the file format at the > same time, both to remove the insanity and to make it actually suitable > for use as an IMAP mailstore. That's what Timo did with dbox, so if > you've got your systems to the point where nothing but Dovecot touches > the mail files you should seriously consider switching. If/when I do switch mailbox formats it'll be to mdbox so FTS doesn't drop a big hammer on the spindles. Thanks for the informative discussion Ben. -- Stan From stan at hardwarefreak.com Sat Oct 27 04:45:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 20:45:34 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <5087F4B4.2060107@hardwarefreak.com> <20121026202915.f748f4c7264a5dca71374fb5@mjh.name> Message-ID: <508B3CBE.1000004@hardwarefreak.com> On 10/26/2012 1:29 PM, Milan Holz?pfel wrote: > On Wed, 24 Oct 2012 09:01:24 -0500 > Stan Hoeppner wrote: > >> On 10/24/2012 6:28 AM, Milan Holz?pfel wrote: >> >>> I have a problem with an incosistent mdbox: >> ... >>> four hours after the problem initially appeared, I did a hard reset of >>> the system because it was unresponsive. >> ... >>> Can anybody say something about this? May the mdbox be repaired? >> >> If the box is truly unresponsive, i.e. hard locked, then the corrupted >> indexes are only a symptom of the underlying problem, which is unrelated >> to Dovecot, UNLESS, the lack of responsiveness was due to massive disk >> access, which will occur when rebuilding indexes on a 6.6GB mailbox. >> You need to know the difference so we have accurate information to >> troubleshoot with. > > Thanks for your suggestion. I wasn't looking for a solution for the > unresponsiveness, but I failed to make that clear. It's likely all related. If you have already, or will continue to, hard reset the box, you will lose inflight data in the buffer cache, which may very likely corrupt your mdbox files and/or indexes. I'm a bit shocked you'd hard reset a *slow* responding server. Especially one that appears to be unresponsive due to massive disk IO. That's a recipe for disaster... > I was not patient enough to debug the unresponsiveness issue. The box > was not hard locked, but any command took very look if it would at all > complete. I think that it could be massive swapping, but I wouldn't > expect Dovecot to be the cause. This leads me to believe your filesystem root, swap partition, and Dovecot mailbox storage are all on the same disk, or small RAID set. Is this correct? > After the reboot, Dovecot would happily re-execute the failing index > rebuild on each new incoming message, which suggests that Dovecot > wasn't the cause for the unresponsiveness. This operation is a tiny IO pattern compared to the 6.6GB re-indexing operation you mentioned before. So you can't make the simple assumption that "Dovecot wasn't the cause for the unresponsiveness". If fact Dovecot likely instigated the problem, though it likely isn't the "cause". I'll take a stab at that below. >> If the there's a kernel or hardware problem, you should see related >> errors in dmesg. Please share those. > > The kernel had messages like > > INFO: task cron:2799 blocked for more than 120 seconds. Now we're getting some meat on this plate. > in the dmesg. But again, I didn't mean to ask for a solution to this > problem. "blocked for more than 120 seconds" is a kernel warning message, not an error message. We see this quite often on the XFS list. Rarely, this is related to a kernel bug. Most often the cause of this warning is saturated IO. In this case it appears cron blocked for 120s because it couldn't read /var/cron/crontabs/[user] The most likely cause of this is that so many IO requests are piled up in the queue that it took more than 2 minutes for the hardware (disks) to complete them before servicing the cron process' IO requests. Dovecot re-indexing a 6.6GB mailbox, with other IO occurring concurrently, could easily cause this situation if you don't have sufficient spindle IOPS. I.e. this IO pattern will bring a single SATA disk or mirror pair to its knees. If you currently have everything on a single SATA disk or mirror pair, the solution for eliminating the bogging down of the system, and likely the Dovecot issues related to it, is to simply separate your root filesystem, swap, and Dovecot data files onto different physical devices. For instance, moving the root filesystem and swap to a small SSD will prevent the OS unresponsiveness, even if Dovecot is bogged down with IO to the SATA disk. With spinning rust storage, separation of root filesystem, swap, and application data to different storage IO domains is system administration 101 kind of stuff. If you're using SSD this isn't (as) critical as it's pretty hard to saturate the IO limits of an SSD. -- Stan From stan at hardwarefreak.com Sat Oct 27 05:54:21 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 26 Oct 2012 21:54:21 -0500 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> <20121026203024.5dd34fb5cca299bf99c980a1@mjh.name> Message-ID: <508B4CDD.4070508@hardwarefreak.com> On 10/26/2012 1:30 PM, Milan Holz?pfel wrote: > On Wed, 24 Oct 2012 13:28:11 +0200 > Milan Holz?pfel wrote: > >> I have a problem with an incosistent mdbox: >> [...] >> The problem appeared out of nowhere. [...] > > That's just wrong. Two minutes before the corruption occured for > the first time, the machine was booted after power-off without prior > shutdown. I didn't notice this until now, sorry for this. Ahh, more critical information. Better late than never I guess. > The mailbox is on XFS. As far as I remember, XFS in known for leaving > NULL bytes at the end of files after a system reset. At least, I found > 72 bytes of NULL in a plain text log file on XFS after such an event. > Do you think this may be the source of the index corruption? Very possibly. > Do you have any other suggestions for recovering the mailbox? Other than restoring from a backup, I do not. Others might. But I will offer this suggestion: Never run a server without a properly functioning UPS and shutdown scripts. The system in question isn't a laptop is it? I'm trying to ascertain how many server 'rules' you're breaking before making any more assumptions or giving any more advice. -- Stan From bernics.gabor at penta.hu Sat Oct 27 10:52:29 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Sat, 27 Oct 2012 09:52:29 +0200 Subject: [Dovecot] mail open slowly Message-ID: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> Hello, I have a dovecot IMAP server (relative small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) with 100 Gbyte maildirs. Server works fine but sometimes I can open mails slowly (5-10 sec), typical with new mails. It's intresting when I open an another old mail (with 0 sec wait), after new mail open quickly. Is it dovecot mail indexes or I/O problem? Load and CPU use is small typical 0.10, 10%, I see small IO wait. Debian 6.0, Dovecot 1.2.15, fsync and nmap is disable Best Regards, Gabor From rs at sys4.de Sat Oct 27 12:36:03 2012 From: rs at sys4.de (Robert Schetterer) Date: Sat, 27 Oct 2012 11:36:03 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> Message-ID: <508BAB03.9050709@sys4.de> Am 27.10.2012 09:52, schrieb Bernics G?bor | Penta Uni? Zrt.: > > > Hello, > > I have a dovecot IMAP server (relative small hardware: HP > Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) > with 100 Gbyte maildirs. that seems ok > > Server works fine but sometimes I can open > mails slowly (5-10 sec), typical with new mails. what mailbox type, how much mail in that mailbox how many concurent cons, imap or pop3 ? > > It's intresting when I > open an another old mail (with 0 sec wait), after new mail open > quickly. may client cached > > Is it dovecot mail indexes or I/O problem? perhaps this , perhaps other > > Load and CPU use > is small typical 0.10, 10%, I see small IO wait. > > Debian 6.0, Dovecot > 1.2.15, fsync and nmap is disable show config and logs > > Best Regards, > > Gabor > > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From bernics.gabor at penta.hu Sat Oct 27 14:58:27 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Sat, 27 Oct 2012 13:58:27 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <508BAB03.9050709@sys4.de> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> Message-ID: <5f981c03d54f00df233f82495df72022@penta.hu> Thank you I use dovecot LDA (+sieve) with maildir. conf: http://pastebin.com/9fhYD58g logs: http://pastebin.com/CXct3B6k connections: http://pastebin.com/v24iRz60 "It's intresting when I open an another old mail (with 0 sec wait), after new mail open quickly. may client cached" it's possible. 2012-10-27 11:36 id?pontban Robert Schetterer ezt ?rta: > Am 27.10.2012 09:52, schrieb Bernics G?bor | Penta Uni? Zrt.: > >> Hello, I have a dovecot IMAP server (relative small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with 2x 7200 SATA disk) with 100 Gbyte maildirs. > > that seems ok > >> Server works fine but sometimes I can open mails slowly (5-10 sec), typical with new mails. > > what mailbox type, how much mail in that mailbox > how many concurent cons, imap or pop3 ? > Is it dovecot mail indexes or I/O problem? > > perhaps this , perhaps other From rs at sys4.de Sat Oct 27 15:55:04 2012 From: rs at sys4.de (Robert Schetterer) Date: Sat, 27 Oct 2012 14:55:04 +0200 Subject: [Dovecot] mail open slowly In-Reply-To: <5f981c03d54f00df233f82495df72022@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> Message-ID: <508BD9A8.8010101@sys4.de> Am 27.10.2012 13:58, schrieb Bernics G?bor | Penta Uni? Zrt.: > > > Thank you > > I use dovecot LDA (+sieve) with maildir. > > conf: in general you should upgrade to 2.1.x with lmtp for better performance use auth cache http://wiki.dovecot.org/Authentication/Caching an example you find in your config > > > http://pastebin.com/9fhYD58g > > logs: > > http://pastebin.com/CXct3B6k i dont see imap here, do more verbose logging > > > connections: > > http://pastebin.com/v24iRz60 to much info for this stage how much traffic concurent in fail stage would be interesting any chance measure iops of the storage ? what mail client ? if tb do debug http://wiki.dovecot.org/Debugging/Thunderbird how much mail does this mailbox have > > "It's intresting when I > open an another old mail (with 0 sec wait), after new mail open quickly. > > > may client cached" > > it's possible. > > 2012-10-27 11:36 id?pontban Robert > Schetterer ezt ?rta: > >> Am 27.10.2012 09:52, schrieb Bernics G?bor | > Penta Uni? Zrt.: >> >>> Hello, I have a dovecot IMAP server (relative > small hardware: HP Microserver with 6 Gbyte RAM, linux soft RAID1 with > 2x 7200 SATA disk) with 100 Gbyte maildirs. >> >> that seems ok >> >>> > Server works fine but sometimes I can open mails slowly (5-10 sec), > typical with new mails. >> >> what mailbox type, how much mail in that > mailbox >> how many concurent cons, imap or pop3 ? >> Is it dovecot mail > indexes or I/O problem? >> >> perhaps this , perhaps other > > > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From stan at hardwarefreak.com Sat Oct 27 21:27:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 27 Oct 2012 13:27:50 -0500 Subject: [Dovecot] mail open slowly In-Reply-To: <5f981c03d54f00df233f82495df72022@penta.hu> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> Message-ID: <508C27A6.30207@hardwarefreak.com> On 10/27/2012 6:58 AM, Bernics G?bor | Penta Uni? Zrt. wrote: > I use dovecot LDA (+sieve) with maildir. > > conf: > > http://pastebin.com/9fhYD58g Next time simply paste "dovecot -n" output into your email. Assuming Dovecot is the only program accessing the maildirs, try: maildir_very_dirty_syncs=yes That may help some. It may not have been a factor in this case, but note that when anyone is doing a full text search on a large mailbox on this hardware with maildir you will see latency, and it is unavoidable. Neither a single 7.2K SATA spindle nor md/RAID1 pair of them, has enough seek capacity to service all the sector requests in a timely fashion. Also, I noticed you disabled fsync. This is a very very bad idea for a mail server. If you lose power, or suffer a kernel/hardware/etc crash, you lose the Linux buffer cache contents. Thus, you may lose emails that haven't been flushed to disk, and possibly get index file corruption if mmap'd pages haven't been flushed. Running with fsync disabled is like having sex with a Bangkok prostitute without a condom while juggling chainsaws while driving drunk at 250kph at night without headlights. fsync does hurt write performance to a degree, especially with maildir storage, but will likely be invisible on a small server with few users/light load. And it will prevent potentially severe problems with file corruption and/or lost emails. -- Stan From stephan at rename-it.nl Sun Oct 28 00:03:20 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 27 Oct 2012 23:03:20 +0200 Subject: [Dovecot] Problem with sieve. dovecot 2.0.17 In-Reply-To: <5088E74C.9030006@ngtech.co.il> References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> <5088E74C.9030006@ngtech.co.il> Message-ID: <508C4C18.5010401@rename-it.nl> On 10/25/2012 9:16 AM, Eliezer Croitoru wrote: > My only problem is that it will put the file in the folder but will > not mark the folder with the new file until I actually check the > folder manually. > it's not that much hustle but if there is a way to solve it I will be > more then happy to hear about it. This is most likely a client problem. Have you configured your client to check that folder? Regards, Stephan. From dave at boostpro.com Sat Oct 27 23:00:16 2012 From: dave at boostpro.com (David Abrahams) Date: Sat, 27 Oct 2012 16:00:16 -0400 Subject: [Dovecot] When are search indexes updated? Message-ID: I noticed that occasionally searching in my huge archive mailbox can be really slow, so I tried doveadm index on it and it seemed to do a lot of work, which seemed strange given, for example, that dovecot-lda says it keeps Dovecot index files up-to-date. Then I thought, "maybe these are different files than the search indices." If so, that's not entirely clear from the docs and Wiki. So, questions: * When are search indexes updated? * Are they updated incrementally? * If not, why not? * If so, why would a mailbox's index drift out-of-date, as mine had? BTW, I'm using the clucene search backend. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost From stan at hardwarefreak.com Sun Oct 28 04:46:34 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 27 Oct 2012 21:46:34 -0500 Subject: [Dovecot] When are search indexes updated? In-Reply-To: References: Message-ID: <508C9C8A.8000309@hardwarefreak.com> On 10/27/2012 3:00 PM, David Abrahams wrote: > > I noticed that occasionally searching in my huge archive mailbox can be > really slow, so I tried doveadm index on it and it seemed to do a lot of > work, which seemed strange given, for example, that dovecot-lda says it > keeps Dovecot index files up-to-date. Then I thought, "maybe these are > different files than the search indices." If so, that's not entirely > clear from the docs and Wiki. So, questions: Mailbox and search indexes are separate. Look in your mailbox directory and you'll see them, such as on 1.2.x with mbox: $ la /home/stan/mail/.imap/1-Dovecot total 3.4M drwx------ 2 stan stan 135 Oct 25 21:39 . drwx------ 51 stan stan 4.0K Apr 13 2012 .. -rw------- 1 stan stan 44K Oct 27 13:28 dovecot.index -rw------- 1 stan stan 1.2M Oct 27 21:23 dovecot.index.cache -rw------- 1 stan stan 18K Oct 27 21:23 dovecot.index.log -rw------- 1 stan stan 1.1M May 20 06:32 dovecot.index.search -rw------- 1 stan stan 1.1M May 20 06:32 dovecot.index.search.uids I've not full text searched this folder for quite some time, thus the search indexes are not current, and the next FTS of this mail folder will take much more time than if the FTS indexes were current. > * When are search indexes updated? When the index is stale. > * Are they updated incrementally? > * If not, why not? > * If so, why would a mailbox's index drift out-of-date, as mine had? When a sufficient number of messages are added to an IMAP folder the FTS index becomes stale. This index is not updated in real time. This is why Timo and others recommend cron'ing a script to index folders regularly that are searched regularly. This keeps the indexes up to date and keeps searches fast. If you don't do this or search often, your indexes become stale. Then each time you do an FTS search the first thing that happens is an FTS re-indexing of the mail folder. Only then does it display the search results. > BTW, I'm using the clucene search backend. I've not used Lucene, but I believe the default behavior is similar to the Dovecot 1.2.x FTS indexer. -- Stan From claude.xavier at gmail.com Sun Oct 28 11:28:28 2012 From: claude.xavier at gmail.com (Xavier Claude) Date: Sun, 28 Oct 2012 10:28:28 +0100 Subject: [Dovecot] How to activate antispam plugin Message-ID: <3146470.D7UsahtfX5@coruscant> Hello, I'm using dovecot 2.1.7 from the Debian backports package and I'm trying to get working the antispam plugin with dspam. I have followed the documentation http://wiki2.dovecot.org/Plugins/Antispam but it does not seem to work. The /var/log/dspam folder is empty even after I put mail in the Spam folder. How can I see if the plugin is working and what config option am I missing ? Thank in advance for your help. Here is my config from dovecot -n: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-xenU-6887-i386 i686 Debian 6.0.6 mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } plugin { antispam_backend = dspam antispam_dspam_args = --mode=teft;--deliver=;--user;%u antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = move antispam_spam = Spam antispam_trash = trash;Corbeille;Trash;Deleted Items; Deleted Messages sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap pop3" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } ssl_cert = References: <5086B568.1010905@ngtech.co.il> <5086F2BB.7010704@rename-it.nl> <5088E74C.9030006@ngtech.co.il> <508C4C18.5010401@rename-it.nl> Message-ID: <508D4F1F.4030603@ngtech.co.il> On 10/27/2012 11:03 PM, Stephan Bosch wrote: > > This is most likely a client problem. Have you configured your client to > check that folder? Yes unless there there is a special thing I dont know yet about in Thunderbird. Thanks, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il From afodis.pinon at hotmail.fr Mon Oct 29 11:01:03 2012 From: afodis.pinon at hotmail.fr (Boris PINON) Date: Mon, 29 Oct 2012 10:01:03 +0100 Subject: [Dovecot] Active Directory 2003 user database and passwords with special characters Message-ID: Hello everybody, As explained in the topic, i have troubles with authentication of my users. First of all, sorry for my poor english... I'm running dovecot v1.2.15 on a Debian 6 64bits server up to date. My users database is an Active Directory 2003 (it's important to know that because Active Directory can't retrieve users passwords, you have to bind LDAP with a domain administrator). So, i'm using userdb ldap for authenticated my users and it works ! BUT... When an user having a password with special characters like " ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ", authentication does not work. And in my log file : MY_SERVER | mail/info | dovecot | 2012/08/27 10:42:14 | auth(default): cache(my_user,192.168.7.127): plain(RU0975?*) != 'RU0975??*' As you can see, the character ? has been replaced by ??. My dovecot.conf : protocols = imap imaps pop3 pop3s managesieve shutdown_clients = yes protocol imap { listen = 192.168.7.1:143 ssl_listen = 192.168.7.1:993 mail_plugins = quota imap_quota autocreate imap_client_workarounds = outlook-idle delay-newmail tb-extra-mailbox-sep } protocol pop3 { listen = 192.168.7.1:110 ssl_listen = 192.168.7.1:995 mail_plugins = quota pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_lock_session = no pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol managesieve { listen = 192.168.7.1:4190 login_executable = /usr/lib/dovecot/managesieve-login mail_executable = /usr/lib/dovecot/managesieve managesieve_implementation_string = dovecot } protocol lda { mail_plugins = sieve quota postmaster_address = postmaster at contoso.fr hostname = webmail.contoso.fr sendmail_path = /usr/sbin/sendmail quota_full_tempfail = no auth_socket_path = /var/run/dovecot/auth-master } log_timestamp = "%Y-%m-%d %H:%M:%S " syslog_facility = mail mail_debug = no auth_debug = no auth_debug_passwords = no ssl = required ssl_cert_file = /etc/ssl/certs/webmail.contoso.fr.pem ssl_key_file = /etc/ssl/private/webmail.contoso.fr.key ssl_ca_file = /etc/ssl/certs/VERYSIGN.pem ssl_verify_client_cert = no mail_location = maildir:%h mail_full_filesystem_access = no mail_uid = 500 mail_gid = 8 mail_privileged_group = mail first_valid_uid = 500 last_valid_uid = 500 first_valid_gid = 8 last_valid_gid = 8 login_greeting = Webmail CONTOSO login_process_size = 256 login_process_per_connection = no login_processes_count = 2 login_max_processes_count = 128 login_max_connections = 512 max_mail_processes = 1024 mail_process_size = 256 mail_max_keyword_length = 50 disable_plaintext_auth = yes auth_failure_delay = 2 auth_process_size = 256 auth_username_format = %Lu auth default { mechanisms = plain login auth_cache_size = 2048 passdb ldap { args = /etc/dovecot/dovecot-ldap.conf } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } user = vmail count = 1 socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail group = mail } client { path = /var/run/dovecot/auth-client mode = 0666 user = vmail group = mail } } } dict { } plugin { quota = maildir:User quota quota_warning = bytes=80%% /usr/lib/dovecot/quota-warning 80 quota_warning2 = bytes=95%% /usr/lib/dovecot/quota-warning 95 quota_warning3 = bytes=99%% /usr/lib/dovecot/quota-warning 99 sieve=dovecot.sieve sieve_dir=~/.Sieve sieve_extensions=+imapflags autocreate = Spam autocreate2 = Trash autosubscribe = Spam autosubscribe2 = Trash autosubscribe3 = Sent autosubscribe4 = Drafts } And my dovecot-ldap.conf : # My domain controller uris = ldap://192.168.1.1:3268 dn = CN=ServerOperator,CN=Users,DC=contoso,DC=fr dnpass = MyPassword debug_level = 0 auth_bind = yes ldap_version = 3 base = CN=Users,DC=contoso,DC=fr deref = never scope = subtree user_attrs = mailDirectory=home=/var/spool/mail/%$,mailQuota=quota_rule=*:bytes=%$,=quota_rule2=Trash:storage=100%% user_filter = (&(|(sAMAccountName=%n)(mailAcceptingGeneralID=%u)(mail=%u))(!(|(mailDrop="*|*")(mailDrop="*:*")(mailDrop="*/*")(userParams=noMail)))) pass_filter = (&(sAMAccountName=%n)(!(|(mailDrop="*|*")(mailDrop="*:*")(mailDrop="*/*")(userParams=noMail)))) default_pass_scheme = CRYPT Does anyone else have this problem? If yes, how to solve? Thank you in advance. From tss at iki.fi Mon Oct 29 16:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:18:22 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <20121016231856.GA10851@daniel.localdomain> References: <20121016231856.GA10851@daniel.localdomain> Message-ID: <0CECDB89-90BF-4A2F-97AC-713344F24996@iki.fi> On 17.10.2012, at 2.18, Daniel Parthey wrote: > doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A > shows the following message in the log when iterating the 49th user: > > Oct 17 00:47:17 10.129.3.233 dovecot: doveadm: Error: purge: invalid option -- 'e' > Oct 17 00:47:17 10.129.3.233 dovecot: doveadm(someuser at example-ll.org): Error: doveadm purge: Client sent unknown parameter: ? > > Any ideas on how this error gets triggered? Not sure. There's no valid 'e' option anywhere. I guess one of the non-option parameters begin with "-e" and it think it's an option. But I can't really think of how that would happen with purge either. So it would be helpful to look at what exactly the doveadms are talking to each others. Could you get the network traffic from them? Or strace -s 1000 doveadm purge should show it somewhere too. From tss at iki.fi Mon Oct 29 16:27:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:27:20 +0200 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <20121018060354.GA2528@leningrad.koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> Message-ID: On 18.10.2012, at 9.03, Levent Dane wrote: >> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html > > in Code/dovecot-virtual: > Archive > inthread refs keyword code not deleted I still couldn't reproduce with this. > I tried to take coredump but i didn't compile with debug flags. > http://pastebin.com/CMbiYJeK I think the problem here mainly is that gdb doesn't work very nicely across multiple execs (imap executes doveconf which executes imap again). You can avoid that by getting a core dump the regular way or making the $base_dir/config socket 0666 permissions. > If you can't reproduce this error. Tomorrow, I'll compile with debug flags. A proper gdb backtrace would definitely be the easiest way to solve this. BTW. Is it only STATUS (UNSEEN) that crashes, or also if you simply SELECT the mailbox? From bernics.gabor at penta.hu Mon Oct 29 16:29:22 2012 From: bernics.gabor at penta.hu (=?UTF-8?Q?Bernics_G=C3=A1bor_=7C_Penta_Uni=C3=B3_Zrt=2E?=) Date: Mon, 29 Oct 2012 15:29:22 +0100 Subject: [Dovecot] mail open slowly In-Reply-To: <508C27A6.30207@hardwarefreak.com> References: <6e5049ac68dc35f2fbd95c86d5c15714@penta.hu> <508BAB03.9050709@sys4.de> <5f981c03d54f00df233f82495df72022@penta.hu> <508C27A6.30207@hardwarefreak.com> Message-ID: Thanks a lot to everybody First step I will upgrade to dovecot2. I will write my experiences. "Running with fsync disabled is like having sex with a Bangkok prostitute without a condom while juggling chainsaws while driving drunk at 250kph at night without headlights." :-) Gabor 2012-10-27 20:27 id?pontban Stan Hoeppner ezt ?rta: > On 10/27/2012 6:58 AM, Bernics G?bor | Penta Uni? Zrt. wrote: > >> I use dovecot LDA (+sieve) with maildir. conf: http://pastebin.com/9fhYD58g [1] > > Next time simply paste "dovecot -n" output into your email. > > Assuming Dovecot is the only program accessing the maildirs, try: > > maildir_very_dirty_syncs=yes > > That may help some. > > It may not have been a factor in this case, but note that when anyone is > doing a full text search on a large mailbox on this hardware with > maildir you will see latency, and it is unavoidable. Neither a single > 7.2K SATA spindle nor md/RAID1 pair of them, has enough seek capacity to > service all the sector requests in a timely fashion. > > Also, I noticed you disabled fsync. This is a very very bad idea for a > mail server. If you lose power, or suffer a kernel/hardware/etc crash, > you lose the Linux buffer cache contents. Thus, you may lose emails > that haven't been flushed to disk, and possibly get index file > corruption if mmap'd pages haven't been flushed. > > Running with fsync disabled is like having sex with a Bangkok prostitute > without a condom while juggling chainsaws while driving drunk at 250kph > at night without headlights. > > fsync does hurt write performance to a degree, especially with maildir > storage, but will likely be invisible on a small server with few > users/light load. And it will prevent potentially severe problems with > file corruption and/or lost emails. Links: ------ [1] http://pastebin.com/9fhYD58g From tss at iki.fi Mon Oct 29 16:38:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:38:58 +0200 Subject: [Dovecot] Dovecot quota postgres dictionary problems In-Reply-To: <1350632450161-38234.post@n4.nabble.com> References: <1350632450161-38234.post@n4.nabble.com> Message-ID: <794F0BC6-CF9D-4BE7-A0C9-FC0D93E72166@iki.fi> On 19.10.2012, at 10.40, tmihalicek wrote: > I have a strange errors in .err log file, but the postgres seem to be filling > with quota changes, i will also put configs in > > Oct 19 09:23:52 mailstore-node-01 dovecot: imap(test at example.net): Error: > read(/var/run/dovecot/dict) failed: Timeout after 30 seconds > Oct 19 09:24:22 mailstore-node-01 dovecot: imap(test at example.net): Error: > read(/var/run/dovecot/dict) failed: Timeout after 30 seconds dict process is taking too long to give results back. Is PostgreSQL too heavily loaded? > Oct 19 09:23:21 mailstore-node-01 dovecot: imap(test at example.net): Panic: > file dict-client.c: line 270 (client_dict_finish_transaction): assertion > failed: (dict->async_commits > 0) http://hg.dovecot.org/dovecot-2.1/rev/67e9cb0b06ec should fix this crash. From tss at iki.fi Mon Oct 29 16:42:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:42:57 +0200 Subject: [Dovecot] LDA without lookup as non-root? In-Reply-To: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> References: <1350679409.31412.YahooMailNeo@web39302.mail.mud.yahoo.com> Message-ID: On 19.10.2012, at 23.43, E.B. wrote: > I'm having some problems getting LDA to work without > userdb lookups and have a few related questions. This system has all > users in MySQL, each user with unique UID/GID, no local users at all. > Installation is from apt-get. > > > 1) If LDA is invoked without > lookups, is it correct to assume that the "service auth" and "service > auth-worker" can be completely removed from dovecot master > configuration? (I have tried commenting them out and logging into IMAP, > which seems to work, not sure if anyone else needs the auth service) If you remove them the defaults are simply used. > 2) > If LDA is invoked without lookups, will I be unable to use Dovecot > quota plugin? Does it need to have a user lookup to get quota info? > (haven't added quota support, need to take this one step at a time) You can give quota info also via either environment variables or via -o plugin/quota_rule=xx parameter. > 3) The interesting part -- I am invoking LDA from Maildrop. See: > http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So > when invoked, Maildrop has already dropped to the destination UID/GID > and the needed paths are available in the environment. However, using > as many permutations of calling LDA as I can think of (based on http://wiki2.dovecot.org/LDA ), I always get this: > > (command line usage error. Command output: lda: Fatal: Couldn't lookup our username (uid=2500) ) Set USER environment. > 3.5) > Related question, my users have separate homedir and maildir, both > paths are looked up by Maildrop. I think I need to call LDA with > "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? As long as the home and mail directories point to the same ones as they are when logging in via IMAP/POP3. From tss at iki.fi Mon Oct 29 16:45:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:45:06 +0200 Subject: [Dovecot] Configuring Dovecot & Snarf plugin for the first time In-Reply-To: <50835541.8000808@bubble.org> References: <50835541.8000808@bubble.org> Message-ID: On 21.10.2012, at 4.52, Jeffrey Ross wrote: > However whenever I enable the snarf plugin using the example on the wiki page my email is not loaded and when I remove my configuration for snarf my email re-appears. Based upon what I can tell the snarf plugin is either not loading (but I see it listed in the logs) or simply not working (which is probably because its not configured properly). .. > plugin { > snarf = = /snarf/INBOX > } Looks like you have one too many "="? From tss at iki.fi Mon Oct 29 16:47:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:47:25 +0200 Subject: [Dovecot] trash plugin not doing it's job In-Reply-To: References: Message-ID: <189B7E53-0495-4D2E-A845-6CEE1304898D@iki.fi> On 18.10.2012, at 11.05, Jan-Frode Myklebust wrote: > I enabled the trash plugin yesterday, adding "trash" to mail_plugins, > and configuring the plugin setting "trash = > /etc/dovecot/dovecot-trash.conf.ext". > > > But I still see users with lots of files in INBOX.Trash getting > bounced because of quota exceeded: .. > # 2.0.14: /etc/dovecot/dovecot.conf There are several fixes to Trash plugin in v2.1. I think it's simply somewhat broken in v2.0. From tss at iki.fi Mon Oct 29 16:49:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 16:49:58 +0200 Subject: [Dovecot] Auth caching and password changes In-Reply-To: <508526C2.8030403@um.es> References: <508149CC.9070004@um.es> <508526C2.8030403@um.es> Message-ID: <1B474730-A7EF-4607-9A1E-4DD215518E6B@iki.fi> On 22.10.2012, at 13.58, Angel L. Mateo wrote: > My question now is there any way to configure authentication so a mechanism is only use when connections coming from a set of IPs? local/remote {} blocks were supposed to provide this. They don't currently work for auth process settings though. From tss at iki.fi Mon Oct 29 17:04:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:04:25 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <5087FF4F.8050103@papaya-cms.com> References: <5087FF4F.8050103@papaya-cms.com> Message-ID: <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> On 24.10.2012, at 17.46, Alexander Weber wrote: > if address :is "to" "mantis-admin@<*>" > { > fileinto "/home/shared/.automail.Bugtracker/"; > } Use mailbox name, not filesystem path: fileinto "shared/automail/Bugtracker"; From tss at iki.fi Mon Oct 29 17:08:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:08:42 +0200 Subject: [Dovecot] Dovecot sends BYE while fetching X-GM-MSGID In-Reply-To: <1698191351159403@web29e.yandex.ru> References: <1698191351159403@web29e.yandex.ru> Message-ID: On 25.10.2012, at 13.03, Loshkovskyi Andrii wrote: > While using the following set of commands, I am having the error as below: > > FETCH 7 (X-GM-MSGID) > A15 FETCH 7 (X-GM-MSGID) > A15 BAD Error in IMAP command FETCH: Unknown parameter X-GM-MSGID > > Can I somehow disable such errors so that Dovecot won't send BYE on X-GM-MSGID but just proceed with following emails? BYE or BAD? It shouldn't send BYE unless you send 20 consecutive BAD commands. From weber at papaya-cms.com Mon Oct 29 17:11:25 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Mon, 29 Oct 2012 16:11:25 +0100 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> Message-ID: <508E9C9D.8060601@papaya-cms.com> Am 29.10.2012 16:04, schrieb Timo Sirainen: > On 24.10.2012, at 17.46, Alexander Weber wrote: > >> if address :is "to" "mantis-admin@<*>" >> { >> fileinto "/home/shared/.automail.Bugtracker/"; >> } > > Use mailbox name, not filesystem path: > > fileinto "shared/automail/Bugtracker"; > error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. sieve: info: started log at Oct 29 16:10:03. error: msgid=<*>: failed to store into mailbox 'shared/automail/Bugtracker/': Invalid mailbox name. nope, didn't work :/ From tss at iki.fi Mon Oct 29 17:18:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:18:42 +0200 Subject: [Dovecot] Dovecot stops to work - anvil problem In-Reply-To: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> References: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> Message-ID: <51972B14-6973-4510-870D-956F858FC76B@iki.fi> On 26.10.2012, at 13.24, FABIO FERRARI wrote: > Hi all, > > we have a problem about anvil, it seems that when we have a high load the > dovecot stops to work. Sometimes it is sufficient to make a dovecot > reload, but sometimes we have to restart it. > > Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files This is the problem. > And these are the limit settings in the OS: > * soft nofile 131072 > * hard nofile 131072 > > Have someone had the same problem? The OS limits are ok. But you need to make sure that the dovecot processes have enough fds in ulimit. You can check the limits with: cat /proc//limits The "Max open files" soft limit is what you're most likely hitting. Use "ulimit -n 10000" or something before running dovecot binary. And make sure that it changes the limit in the proc. Many init scripts change the ulimit internally. From tss at iki.fi Mon Oct 29 17:18:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:18:46 +0200 Subject: [Dovecot] Can Dovecot authenticate against an external email server? In-Reply-To: <5087035B.7060208@perkel.com> References: <5087035B.7060208@perkel.com> Message-ID: On 23.10.2012, at 23.51, Marc Perkel wrote: > Just wondering if anyone has done this. > > I have a spam filtering service where I am now storing spam for users I filter for. It's a filter and forward service so I don't control the recipient's email server. > > What I would like to do somehow is have the user enter their email address and password and then look up their POP/IMAP server from a database and try to authenticate from it. If sucessful then the user will be able to access their stored spam using Dovecot and Squirrelmail. http://wiki2.dovecot.org/PasswordDatabase/IMAP From tss at iki.fi Mon Oct 29 17:20:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:20:15 +0200 Subject: [Dovecot] dovecot lda - Permission denied In-Reply-To: <20121026114455.30440@gmx.net> References: <20121026114455.30440@gmx.net> Message-ID: <157A66BA-69AB-45AE-927C-21F827B1736B@iki.fi> On 26.10.2012, at 14.44, tony.blue.mailinglist at gmx.de wrote: > Oct 25 23:37:13 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=501(andy) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) > ... > Oct 25 23:37:14 gustav dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=500(tony) egid=100(users) missing +w perm: /var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755) > ... > > Dovecot is installed as !include auth-passwdfile.conf.ext. For all users there is a entry in der /etc/dovecot/users. > > Usaly the user rights are set to 600. I tryed 755, but I get the same errormessage. 0755 is basically the same as 0600 for sockets, since you disabled writes for others. Use 0777 to give everyone permissions. From tss at iki.fi Mon Oct 29 17:22:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:22:22 +0200 Subject: [Dovecot] Overlapping userdb/passdbs In-Reply-To: References: Message-ID: <98C16420-1D9E-4F37-86D7-9FB91438B843@iki.fi> On 26.10.2012, at 22.13, James Devine wrote: > I have an ldap server for which each entry includes the email address and > the username portion of the email address for authentication. > Authentication works by username if the username is unique among all the > entries. I need to now add some users which must authenticate even if the > username is not unique. I figured one way to do this would be to add a > second user/pass db which puts further restrictions on the ldap query to > make it unique for those users. This doesn't seem to work however as if > the user is found in the first ldap query but the password does not match > it does not try the second. I would use the password as part of the query > but this setup requires me to allow the client to hash the password. Is > there a way to do this? Or maybe I am approaching the problem wrong. You'd need to update this patch: http://dovecot.org/patches/2.0/auth-multi-password-2.0.diff It worked for v1.1 and maybe for v1.2. I never included it mainly because I never had time to check if it had any security issues. From jk at jkart.de Mon Oct 29 17:23:29 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 16:23:29 +0100 Subject: [Dovecot] Out of memory/Managesieve Message-ID: <508E9F71.8050208@jkart.de> Hello, I have here a problem with managesieve. With the login about webmail (roundcube) comes here in the log: --snip dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) --snap I've increased of 2048M and the same above. then with 4096 M happens the following --snip dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) --snap How can one solve then this? Any ideas greatly appreciated. Thanks. -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Backup interessiert niemanden - Auf Restore kommt es an! From tss at iki.fi Mon Oct 29 17:26:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:26:49 +0200 Subject: [Dovecot] When are search indexes updated? In-Reply-To: <508C9C8A.8000309@hardwarefreak.com> References: <508C9C8A.8000309@hardwarefreak.com> Message-ID: <6DC094E4-2D06-4146-A4C2-1717614E30E4@iki.fi> On 28.10.2012, at 4.46, Stan Hoeppner wrote: >> * When are search indexes updated? > > When the index is stale. > >> * Are they updated incrementally? >> * If not, why not? >> * If so, why would a mailbox's index drift out-of-date, as mine had? > > When a sufficient number of messages are added to an IMAP folder the FTS > index becomes stale. This index is not updated in real time. This is > why Timo and others recommend cron'ing a script to index folders > regularly that are searched regularly. This keeps the indexes up to > date and keeps searches fast. If you don't do this or search often, > your indexes become stale. Then each time you do an FTS search the > first thing that happens is an FTS re-indexing of the mail folder. Only > then does it display the search results. Otherwise correct, but "re-indexing" is the wrong word. No already indexed mails are reindexed. Only new mails are added to the index. From tss at iki.fi Mon Oct 29 17:31:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:31:42 +0200 Subject: [Dovecot] Rebuilding indexes fails on inconsistent mdbox In-Reply-To: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> References: <20121024132811.7cf18fdf7343b4dd961b2858@mjh.name> Message-ID: <394FADB5-4E6C-4179-BF30-244390964FA4@iki.fi> On 24.10.2012, at 14.28, Milan Holz?pfel wrote: > Oct 24 10:43:23 two dovecot: imap-login: Login: user=<...>, method=PLAIN, rip=..., lip=..., mpid=4977, TLS > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-24 10:43:23] bytes=115/53726 > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: Inconsistency in map index (467,31960 != 467,36768) > Oct 24 10:43:23 two dovecot: imap(listen at mjh.name): Warning: mdbox .../mdbox/storage: rebuilding indexes The above problems aren't too bad, since Dovecot fixes itself. > Oct 24 10:45:19 two dovecot: imap(listen at mjh.name): Panic: file mdbox-storage-rebuild.c: line 773 (rebuild_update_refcounts): assertion failed: (map_uid < msgs[i]->map_uid) Now this is a bug. > Dovecot 2.0.19-0ubuntu1 But the bug may have already been fixed in v2.1. > The whole mdbox is 6.6 GiB large and I guess that it contains about > 300k-600k messages. It's an archive of public mailing lists, so I could > give access to the files. I'd try first with a recent 2.1 version and if that doesn't fix the crash the easiest way for me to fix it would be to get the files. If you put the files through http://dovecot.org/tools/mdbox-obfuscate.pl they should compress pretty nicely. From tss at iki.fi Mon Oct 29 17:34:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:34:28 +0200 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <508E9C9D.8060601@papaya-cms.com> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> <508E9C9D.8060601@papaya-cms.com> Message-ID: <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> On 29.10.2012, at 17.11, Alexander Weber wrote: >> fileinto "shared/automail/Bugtracker"; > > error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. "shared/automail/Bugtracker" isn't the same as "shared/.automail.Bugtracker/". Anyway that specific error message comes from the trailing '/'. From jk at jkart.de Mon Oct 29 17:34:45 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 16:34:45 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508E9F71.8050208@jkart.de> References: <508E9F71.8050208@jkart.de> Message-ID: <508EA215.6000303@jkart.de> am 29.10.12 16:23 schrieb Jim Knuth : > Hello, > > I have here a problem with managesieve. With the login about > webmail (roundcube) comes here in the log: > > --snip > dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): > Out of memory > dovecot: managesieve-login: Fatal: master: service(managesieve-login): > child 10157 returned error 83 (Out of memory (service > managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) > --snap > I've increased of 2048M and the same above. > then with 4096 M happens the following > > --snip > dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: > Bad file descriptor > dovecot: managesieve-login: Fatal: master: service(managesieve-login): > child 9777 killed with signal 6 (core dumps disabled) > --snap > > How can one solve then this? > Any ideas greatly appreciated. Thanks. > Sorry, I've forgotten. OS Debian stable and Dovecot 2.1. -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Nicht Absicht unterstellen, wenn auch Dummheit ausreicht! From tss at iki.fi Mon Oct 29 17:41:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 17:41:06 +0200 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508E9F71.8050208@jkart.de> References: <508E9F71.8050208@jkart.de> Message-ID: On 29.10.2012, at 17.23, Jim Knuth wrote: > I have here a problem with managesieve. With the login about > webmail (roundcube) comes here in the log: You can always easily reproduce this? Can you get the network traffic logs between Roundcube and Dovecot and reproduce it by sending those same commands manually? > --snip > dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory > dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) Looks like there's a bug somewhere.. Doesn't it log a "raw backtrace"? > --snap > I've increased of 2048M and the same above. > then with 4096 M happens the following > > --snip > dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor > dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) > --snap Probably related to the first error. It would be helpful to get gdb backtraces from both of them, although from the first one you couldn't without patching + recompiling Dovecot. But from the second one I think you can get a core dump with: service managesieve-login { executable = managesieve-login -D } Then you can do something like: gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core bt full From weber at papaya-cms.com Mon Oct 29 17:54:51 2012 From: weber at papaya-cms.com (Alexander Weber) Date: Mon, 29 Oct 2012 16:54:51 +0100 Subject: [Dovecot] [dovecot} Invalid mailbox name. In-Reply-To: <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> References: <5087FF4F.8050103@papaya-cms.com> <5C98CBE0-ACCF-4B36-BF46-97BAD3CD7DF8@iki.fi> <508E9C9D.8060601@papaya-cms.com> <54B8418B-6C5A-4414-BE38-E2214D37B845@iki.fi> Message-ID: <508EA6CB.9010600@papaya-cms.com> Am 29.10.2012 16:34, schrieb Timo Sirainen: > On 29.10.2012, at 17.11, Alexander Weber wrote: > >>> fileinto "shared/automail/Bugtracker"; >> >> error: msgid=<*>: failed to store into mailbox 'shared/.automail.Bugtracker/': Invalid mailbox name. > > "shared/automail/Bugtracker" isn't the same as "shared/.automail.Bugtracker/". > > Anyway that specific error message comes from the trailing '/'. > I've tried every combination of this shared subfolder ( pretty wired :S ) but nevermind - i try procmail with sieve after procmail. thanks anyway :) From tss at iki.fi Mon Oct 29 18:53:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 18:53:00 +0200 Subject: [Dovecot] Save/restore IMAP session state Message-ID: <1351529580.13571.93.camel@hurina> Some future Dovecot version will have "imap-idle" processes where IDLEing IMAP connections get moved, so the system wouldn't waste so much memory for all the IDLEing imap processes. A week ago I thought I'd see how easy it would be to implement this. I got a basic proof of concept working as a "X-STATE" command. Save the state: a x-state * STATE AQDLW45QdwAAAAMAAABuAQAAAAAAAFAcffYAPHnpFctbjlDbYQAAcEmzCwAA a OK State exported. Restore the state: b x-state AQDLW45QdwAAAAMAAABuAQAAAAAAAFAcffYAPHnpFctbjlDbYQAAcEmzCwAA b OK State imported. This could also be used to implement quick session state restoring for webmails (as suggested by Michael Slusarz). For getting the imap-idle process there would have to be code that: * triggers the session saving when process is IDLEing * figures out what filesystem paths the imap-idle should be looking at (i.e. paths to selected mailbox's dovecot.index.log file and maybe for e.g. maildir new/) * send the session state string, paths and imap connection fd to imap-idle process via UNIX socket * implement the actual imap-idle process * implement a way for imap-idle process to send back the state and connection fd to restore the imap process The patch is ugly and still missing many things. Anyway I thought I'd include it here just in case someone was really eager to continue implementing it. :) I'm not sure when I'll have time for it. A full patch would probably have to have some session_save()/session_restore() functions in lib-storage API. But a quick and dirty way is possible to implement for v2.1 as well, as long as some IMAP extensions aren't used (most importantly rfc5267). -------------- next part -------------- A non-text attachment was scrubbed... Name: imap-state.diff Type: text/x-patch Size: 11305 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 18:57:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 18:57:32 +0200 Subject: [Dovecot] Save/restore IMAP session state In-Reply-To: <1351529580.13571.93.camel@hurina> References: <1351529580.13571.93.camel@hurina> Message-ID: <567ADA17-F5E6-48DF-9E9D-601267C568FE@iki.fi> On 29.10.2012, at 18.53, Timo Sirainen wrote: > The patch is ugly and still missing many things. Anyway I thought I'd > include it here just in case someone was really eager to continue > implementing it. :) I'm not sure when I'll have time for it. Oh, and of course I forgot one file out of the patch. Here's an updated one. -------------- next part -------------- A non-text attachment was scrubbed... Name: imap-state2.diff Type: application/octet-stream Size: 12948 bytes Desc: not available URL: From guallar at easternrad.com Mon Oct 29 19:57:37 2012 From: guallar at easternrad.com (Josep L. Guallar-Esteve) Date: Mon, 29 Oct 2012 13:57:37 -0400 Subject: [Dovecot] INBOX permissios woes Message-ID: <9cc05811b75ed0f7235dd86d0e5c1dfd@easternrad.com> Hello, I have a dovecot system that uses winbind authentication against Active Directory. I set it up by following the directions in the wiki. That works great. When a new user receives an email, the inbox is created with permissions 600 (rw- --- --) and ownership user:mail , even though I did chmod 02770 /var/spool/mail. And then, when dovecot tries to access the inbox, it throws the error: Oct 29 13:47:59 imap-login: Info: Login: user=, method=PLAIN, rip=10.0.0.6, lip=10.0.0.26, mpid=29047, secured Oct 29 13:47:59 imap(user1): Error: stat(/var/mail/user1) failed: Permission denied Oct 29 13:47:59 imap(user1): Error: stat(/var/mail/user1) failed: Permission denied Accessing users' Sent, Trash, creating new folders.... all that works fine. I've been looking at the documentation, reading the wiki, searching on google, asking on IRC. If you have any hint or documentation that I've must have overlooked, please let me know. Here's my dovecot information: [josep at testmail ]$ dovecot --version 2.0.9 [josep at testmail ]$ dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.11.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain ntlm login auth_use_winbind = yes auth_username_format = %Lu base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log listen = * log_path = /var/log/dovecot.log mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail mbox_write_locks = fcntl passdb { driver = pam } protocols = imap service auth { unix_listener auth-userdb { mode = 0600 } } ssl_cert = References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> Message-ID: <20121029181700.GA4240@leningrad.koli.be> On 10/29, Timo Sirainen wrote: >On 18.10.2012, at 9.03, Levent Dane wrote: > >>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >> >> in Code/dovecot-virtual: >> Archive >> inthread refs keyword code not deleted > >I still couldn't reproduce with this. I think the problem is mail-search.c is corrupting the index files. >> I tried to take coredump but i didn't compile with debug flags. >> http://pastebin.com/CMbiYJeK > >I think the problem here mainly is that gdb doesn't work very nicely across multiple execs (imap executes doveconf which executes imap again). You can avoid that by getting a core dump the regular way or making the $base_dir/config socket 0666 permissions. I compiled with -ggdb flag. I'm getting this informations /var/log/messages: http://pastebin.com/bpkvp4Ak and from gdb: http://pastebin.com/HY0mVYBS I'm using mutt for imap access. When I pressed '%' key which runs function, the dovecot got seqfault. >> If you can't reproduce this error. Tomorrow, I'll compile with debug flags. > >A proper gdb backtrace would definitely be the easiest way to solve this. > >BTW. Is it only STATUS (UNSEEN) that crashes, or also if you simply SELECT the mailbox? I tried simple SELECT and it still crashed. As I said, the problem is dovecot.index files. Somehow, mail-search.c corrupts this file. -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 From tss at iki.fi Mon Oct 29 20:23:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 20:23:14 +0200 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <20121029181700.GA4240@leningrad.koli.be> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> <20121029181700.GA4240@leningrad.koli.be> Message-ID: <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> On 29.10.2012, at 20.17, Levent Dane wrote: > On 10/29, Timo Sirainen wrote: >> On 18.10.2012, at 9.03, Levent Dane wrote: >> >>>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >>> >>> in Code/dovecot-virtual: >>> Archive >>> inthread refs keyword code not deleted >> >> I still couldn't reproduce with this. > > I think the problem is mail-search.c is corrupting the index files. Not that itself, but yeah looks like if virtual plugin assert-crashes in mail-search.c it leaves the indexes so that the next access will segfault. > I compiled with -ggdb flag. I'm getting this informations > /var/log/messages: http://pastebin.com/bpkvp4Ak > and from gdb: http://pastebin.com/HY0mVYBS Better backtrace than last time, but still no debug information in the backtrace. Maybe that got stripped somewhere between compiling and installing? You can check with "file ..../imap" to see if it's there. Also backtrace from both the mail-search.c assert crash and the segfault would be useful. From calestyo at scientia.net Mon Oct 29 22:31:48 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 21:31:48 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? Message-ID: <1351542708.3435.25.camel@fermat.scientia.net> Hi. For mbox, http://wiki2.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata lists a numer of mail headers: - X-IMAPbase - X-IMAP - X-UID - Status - X-Status - X-Keywords - Content-Length that are treated specially by dovecot. It also suggests, that these should be stripped by the LDA (I guess in order that someone sending you such mail cannot set the status or keywords, or even "attack you" by setting a bogus Content-Length). I wondered, when using maildir, are there any headers that dovecote would treat specially, too? And which I therefore should strip? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 22:39:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 22:39:51 +0200 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? Message-ID: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. From tss at iki.fi Mon Oct 29 22:40:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 22:40:46 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351542708.3435.25.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> Message-ID: <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> On 29.10.2012, at 22.31, Christoph Anton Mitterer wrote: > For mbox, > http://wiki2.dovecot.org/MailboxFormat/mbox#Dovecot.27s_Metadata lists a > numer of mail headers: > - X-IMAPbase > - X-IMAP > - X-UID > - Status > - X-Status > - X-Keywords > - Content-Length > that are treated specially by dovecot. > > It also suggests, that these should be stripped by the LDA (I guess in > order that someone sending you such mail cannot set the status or > keywords, or even "attack you" by setting a bogus Content-Length). Right. > I wondered, when using maildir, are there any headers that dovecote > would treat specially, too? > And which I therefore should strip? No. Maildir metadata is stored elsewhere. From calestyo at scientia.net Mon Oct 29 22:54:09 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 21:54:09 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste Message-ID: <1351544049.3435.47.camel@fermat.scientia.net> Hi. I recently mentioned in several posts, that I'd tended to use mbox rather than maildir, because you don't loose so much space (due to always allocating full blocks per maildir file and thus per mail). I made some tests of my archive, which consists of some 3,4 million mails at a total of 42GB). Most of these mails are probably normal sized, but there are also some with bigger attachments. For those who are interested here are the results: I used a 53687091200 B image file (via loop device) and tested ext4 only. btrfs is IMHO not yet ready, I have had often issues with XFS (corruptions), reiser4 is more or less dead and reiser3 is said to have issues (see e.g. its wikipedia article, even though it has that mode for small files which would fit nicely). As you see the number of mails increased a bit, cause I tested over several days... but this is only a very small increase so it shouldn't change the numbers a lot. 1) Original mbox archives (right now in Evolution) mbox exact space: 38122676224 (does not include meta-data) mbox guess space: 44625670144 (includes Evolution meta-data which is several GBs) mbox num mails: 3412999 (occurances of From_ lines) In the following: - image file, 1B-blocks, Used_begin, Used_end, Available_begin, Available_end result out of df -B 1 - mdir exact used space is the sum of du -B 1 for each regular file (i.e. each mdir file) - mdir guess used space du -B 1 on the root dir of the filesystem - mdir num mails: find . type -f | wc -l on the root dir of the filesystem 2) EXT4 with 4096 blocks: image file: 53687091200 1B-blocks: 52844687360 Used_begin: 188555264 Used_end: 45198778368 Available_begin: 49971777536 Available_end: 2444972032 mdir exact used space: 44810866688 mdir guess used space: 45010243584 mdir num mails: 3423296 delta: 6.688190464 G delta / mail: 1953 B 3) EXT4 with 2048 blocks: image file: 53687091200 1B-blocks: 50324295680 Used_begin: 82857984 Used_end: 41598846976 Available_begin: 47557083136 Available_end: 6041094144 mdir exact used space: 41323991040 mdir guess used space: 41516007424 mdir num mails: 3425033 delta: 3.201314816 G delta / mail: 934 B 4) EXT4 with 1024 blocks: image file: 53687091200 1B-blocks: 50314834944 Used_begin: 38287360 Used_end: 39909360640 Available_begin: 47592193024 Available_end: 7721119744 mdir exact used space: 39683908608 mdir guess used space: 39871086592 mdir num mails: 3425033 delta: 1.561232384 G delta / mail: 455 B As you can see, the delta per mail is rather close to the statistically expected values of 2048B, 1024B and 512B. In the end I probably changed my opinion. ~7GB of wasted block space for all my mails is actually quite a lot, but in days of cheap disk space it's acceptable. And with mbox one has IMHO the major disadvantage that mailservers (including dovecot) store some meta-data _in_ it (i.e. in the mails themselves) , which I don't like a lot. I still think about reports that mbox is much faster with full text search (which sounds reasonable)... but therefore one needs probably and database backend anyway. HTH, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 23:00:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:00:56 +0200 Subject: [Dovecot] v2.2.alpha1 released Message-ID: <1351544456.13571.102.camel@hurina> http://dovecot.org/releases/2.2/alpha/dovecot-2.2.alpha1.tar.gz http://dovecot.org/releases/2.2/alpha/dovecot-2.2.alpha1.tar.gz.sig I wanted to start stabilizing v2.2 release some months ago already, but I somehow got stuck adding more and more features. Now it looks like all of the necessary API changes are done, so everything I'm planning on near future can still be added to v2.2 without major changes. The redesigned dsync hasn't had much testing yet, so avoid running it with important mails. Would be nice if people started testing and reporting any bugs. I was going to create some kind of a test suite for testing all the possible syncing combinations and also doing some random stress testing, but I haven't had time for that yet. The new dsync supports doing very fast syncs by saving the resulting state and giving it as command line parameter to the next sync. So for example doveadm sync -s "" > new-state saves the state and doveadm sync -s `cat new-state` continues from the saved state. The replicator code doesn't yet support this. * When creating home directories, the permissions are copied from the parent directory if it has setgid-bit set. For full details, see http://wiki2.dovecot.org/SharedMailboxes/Permissions * "doveadm auth" command was renamed to "doveadm auth test" * IMAP: ID command now advertises server name as Dovecot by default. It was already trivial to guess this from command replies. + Implemented IMAP MOVE and BINARY extensions + Implemented IMAP CATENATE, URLAUTH and URLAUTH=BINARY extensions (by Stephan Bosch). + Implemented IMAP NOTIFY extension. Requires mailbox_list_index=yes to be enabled. + Redesigned and rewritten dsync. The new design makes the syncing faster, more reliable and more featureful. The new dsync protocol isn't backwards compatible with old dsync versions (but is designed to be forwards compatible with future versions). + All mailbox formats now support per-user message flags for shared mailboxes by using a private index. It can be enabled by adding :INDEXPVT= to mail location. This should be used instead of :INDEX also for Maildir/mbox to improve performance. + Improved mailbox list indexes. They should be usable now, although still disabled by default. + Added LAYOUT=index. The mailbox directories are created using their GUIDs in the filesystem, while the actual GUID <-> name mapping exists only in the index. + LMTP proxy: Implemented XCLIENT extension for passing remote IP address through proxy. From pw at wk-serv.de Mon Oct 29 23:05:42 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 29 Oct 2012 22:05:42 +0100 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <5083D963.3000700@wk-serv.de> References: <5083D963.3000700@wk-serv.de> Message-ID: <508EEFA6.1020506@wk-serv.de> Hi, no one here who is able to reply to my questions? Regards Patrick Patrick Westenberg schrieb: > Hi everyone, > > short version: > Is there no built in failover mechanism for the director service to > handle a backend failure? > > Long version: > I have a frontend server running the director service and two backends. > Due to maintenance I had to shut down one of the backends which caused > connection errors for the users being directed to this backend. > > I was very surprised as I expected the director to redirect these users > to the remaining backend. > > Am I wrong or is the director not working as expected? > > Regards > Patrick > > > > # 2.1.6: /usr/local/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 > auth_mechanisms = plain login > director_mail_servers = 172.17.1.1 172.17.1.2 > director_servers = 172.17.1.3 172.17.1.4 > lmtp_proxy = yes > log_path = /var/log/dovecot.log > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > protocols = imap pop3 lmtp sieve > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-userdb { > user = dovecot > } > } > service director { > fifo_listener login/proxy-notify { > mode = 0666 > } > inet_listener { > address = 172.17.1.3 > port = 9090 > } > unix_listener director-userdb { > mode = 0600 > } > unix_listener login/director { > mode = 0666 > } > } > service imap-login { > executable = imap-login director > } > service lmtp { > inet_listener lmtp { > address = 172.17.1.3 > port = 24 > } > } > service managesieve-login { > executable = managesieve-login director > inet_listener sieve { > port = 4190 > } > } > service pop3-login { > executable = pop3-login director > } > ssl_cert = ssl_key = protocol !smtp { > passdb { > args = proxy=y nopassword=y starttls=any-cert > driver = static > } > } > protocol smtp { > passdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > userdb { > args = /usr/local/etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > } > protocol lmtp { > auth_socket_path = director-userdb > } From tss at iki.fi Mon Oct 29 23:06:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:06:53 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351544049.3435.47.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> Message-ID: <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> On 29.10.2012, at 22.54, Christoph Anton Mitterer wrote: > I recently mentioned in several posts, that I'd tended to use mbox > rather than maildir, because you don't loose so much space (due to > always allocating full blocks per maildir file and thus per mail). .. > In the end I probably changed my opinion. > ~7GB of wasted block space for all my mails is actually quite a lot, but > in days of cheap disk space it's acceptable. > And with mbox one has IMHO the major disadvantage that mailservers > (including dovecot) store some meta-data _in_ it (i.e. in the mails > themselves) , which I don't like a lot. > I still think about reports that mbox is much faster with full text > search (which sounds reasonable)... but therefore one needs probably and > database backend anyway. There is of course mdbox also, which gives the best of both mbox and maildir (and some of its own new annoyances). From calestyo at scientia.net Mon Oct 29 23:09:11 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:09:11 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> Message-ID: <1351544951.3435.61.camel@fermat.scientia.net> Hi Timo. On Mon, 2012-10-29 at 22:40 +0200, Timo Sirainen wrote: > > I wondered, when using maildir, are there any headers that dovecote > > would treat specially, too? > > And which I therefore should strip? > > No. Maildir metadata is stored elsewhere. Great... and I expect that this ("no headers from the maildir files are interpreted") applies also, when one "imports" mails the first time. With import I don't mean via IMAP, but plainly moving e.g. a maildir++ tree under dovecots mail location. Then dovecot usually starts to generate all it's metadata,... and I expect that things like status and keywords are left simply unset... and things like UID and UIDVALIDITY are freshly initialised and not tried to be converted from the maildir files, right? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From tss at iki.fi Mon Oct 29 23:11:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:11:15 +0200 Subject: [Dovecot] No failover from director to backend? In-Reply-To: <508EEFA6.1020506@wk-serv.de> References: <5083D963.3000700@wk-serv.de> <508EEFA6.1020506@wk-serv.de> Message-ID: <6DFB1CD2-5FE6-405A-B2A8-545938A11F98@iki.fi> People already replied and pointed to poolmon. There is no built-in failure handling, because it's not possible to implement it in a way that works well for everyone. Although I think poolmon could also itself use a bit of tweaking. For example if all hosts became very heavily loaded, poolmon would now probably drop all of them immediately if one if its check connections failed. On 29.10.2012, at 23.05, Patrick Westenberg wrote: > Hi, > > no one here who is able to reply to my questions? > > Regards > Patrick > > > > Patrick Westenberg schrieb: >> Hi everyone, >> >> short version: >> Is there no built in failover mechanism for the director service to >> handle a backend failure? >> >> Long version: >> I have a frontend server running the director service and two backends. >> Due to maintenance I had to shut down one of the backends which caused >> connection errors for the users being directed to this backend. >> >> I was very surprised as I expected the director to redirect these users >> to the remaining backend. >> >> Am I wrong or is the director not working as expected? >> >> Regards >> Patrick >> >> >> >> # 2.1.6: /usr/local/etc/dovecot/dovecot.conf >> # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 >> auth_mechanisms = plain login >> director_mail_servers = 172.17.1.1 172.17.1.2 >> director_servers = 172.17.1.3 172.17.1.4 >> lmtp_proxy = yes >> log_path = /var/log/dovecot.log >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> protocols = imap pop3 lmtp sieve >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0666 >> user = postfix >> } >> unix_listener auth-userdb { >> user = dovecot >> } >> } >> service director { >> fifo_listener login/proxy-notify { >> mode = 0666 >> } >> inet_listener { >> address = 172.17.1.3 >> port = 9090 >> } >> unix_listener director-userdb { >> mode = 0600 >> } >> unix_listener login/director { >> mode = 0666 >> } >> } >> service imap-login { >> executable = imap-login director >> } >> service lmtp { >> inet_listener lmtp { >> address = 172.17.1.3 >> port = 24 >> } >> } >> service managesieve-login { >> executable = managesieve-login director >> inet_listener sieve { >> port = 4190 >> } >> } >> service pop3-login { >> executable = pop3-login director >> } >> ssl_cert = > ssl_key = > protocol !smtp { >> passdb { >> args = proxy=y nopassword=y starttls=any-cert >> driver = static >> } >> } >> protocol smtp { >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> } >> protocol lmtp { >> auth_socket_path = director-userdb >> } > From tss at iki.fi Mon Oct 29 23:13:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:13:36 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351544951.3435.61.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> Message-ID: <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> On 29.10.2012, at 23.09, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 22:40 +0200, Timo Sirainen wrote: >>> I wondered, when using maildir, are there any headers that dovecote >>> would treat specially, too? >>> And which I therefore should strip? >> >> No. Maildir metadata is stored elsewhere. > > Great... and I expect that this ("no headers from the maildir files are > interpreted") applies also, when one "imports" mails the first time. > > With import I don't mean via IMAP, but plainly moving e.g. a maildir++ > tree under dovecots mail location. Yeah. > Then dovecot usually starts to generate all it's metadata,... and I > expect that things like status and keywords are left simply unset... and > things like UID and UIDVALIDITY are freshly initialised and not tried to > be converted from the maildir files, right? Flags are stored in the maildir filenames, so they're always preserved. Keywords, UIDs, UIDVALIDITY etc is preserved if you copy the dovecot-* files with it (which is a good idea to do). From calestyo at scientia.net Mon Oct 29 23:15:30 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:15:30 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> Message-ID: <1351545330.3435.66.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:06 +0200, Timo Sirainen wrote: > There is of course mdbox also, which gives the best of both mbox and maildir (and some of its own new annoyances). Thanks, Timo,... I forgot to mention that. For me _personally_ two things speak against using it: a) To be honest, "you must not lose the dbox index files, they can't be regenerated without data loss"[0] made me a bit scared ;-) b) ext* has no integrity checking (by hash sums) so I used to create my own that puts SHA512 hashes into the inodes of files (as USER_XATTRS). This of course, works only when you have a storage format where files don't change anymore once written,... which can't work with formats having multiple mails per file. Thanks, Chris. btw: What are the actual advantages of sdbox over maildir? [0] http://wiki2.dovecot.org/MailboxFormat/dbox -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Mon Oct 29 23:20:27 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:20:27 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> Message-ID: <1351545627.3435.71.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:13 +0200, Timo Sirainen wrote: > > Great... and I expect that this ("no headers from the maildir files > are > > interpreted") applies also, when one "imports" mails the first time. > > > > With import I don't mean via IMAP, but plainly moving e.g. a maildir > ++ > > tree under dovecots mail location. > > Yeah. So that means: From a "security" point of view, when using maildir (!) there's no need to remove such headers, cause dovcote ignores them (on maildir) always. Right?! I just wondered because when I looked through my mail archive (currently as mentioned, under Evolution)... many emails already had X-UID and X-IMAP* headers.... (set by the remote side, not by Evolution) ... and in no case these should be able to mess around in my dovecot :) > UIDs, UIDVALIDITY etc is preserved if you copy the dovecot-* files > with it (which is a good idea to do). I'll have a question on that too, but ask it under a separate mail in a few minutes,.. cause it doesn't fit this thread anymore ;) Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From limon at koli.be Mon Oct 29 23:26:48 2012 From: limon at koli.be (Levent Dane) Date: Mon, 29 Oct 2012 16:26:48 -0500 Subject: [Dovecot] Problems with Virtual and mail-search.c In-Reply-To: <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> References: <456733b1b04e92265fbd9ba8e005132c@koli.be> <22F1A715-73EE-4F2B-9ECE-CA84B69939A7@iki.fi> <20121018060354.GA2528@leningrad.koli.be> <20121029181700.GA4240@leningrad.koli.be> <0029F8DC-E9A8-4FB1-A2F8-1A3631823157@iki.fi> Message-ID: <20121029212648.GA4292@leningrad.koli.be> On 10/29, Timo Sirainen wrote: >On 29.10.2012, at 20.17, Levent Dane wrote: > >> On 10/29, Timo Sirainen wrote: >>> On 18.10.2012, at 9.03, Levent Dane wrote: >>> >>>>> I can't reproduce this. What contents do you have in dovecot-virtual files? Also doveconf -n output and gdb backtrace would be helpful: http://dovecot.org/bugreport.html >>>> >>>> in Code/dovecot-virtual: >>>> Archive >>>> inthread refs keyword code not deleted >>> >>> I still couldn't reproduce with this. >> >> I think the problem is mail-search.c is corrupting the index files. > >Not that itself, but yeah looks like if virtual plugin assert-crashes in mail-search.c it leaves the indexes so that the next access will segfault. > >> I compiled with -ggdb flag. I'm getting this informations >> /var/log/messages: http://pastebin.com/bpkvp4Ak >> and from gdb: http://pastebin.com/HY0mVYBS > >Better backtrace than last time, but still no debug information in the backtrace. Maybe that got stripped somewhere between compiling and installing? You can check with "file ..../imap" to see if it's there. > >Also backtrace from both the mail-search.c assert crash and the segfault would be useful. I think I get correct backtrace. I attached this mail and uploaded pastebin. http://pastebin.com/L41e6AXY -- Levent Dane 832 356 7771 4604 Spruce St, Bellaire, TX 77401 -------------- next part -------------- Oct 29 16:21:40 widder dovecot: imap(limon): Panic: file mail-search.c: line 90 (mail_search_args_init_sub): assertion failed: (arg->value.keywords == NULL) Oct 29 16:21:40 widder dovecot: imap(limon): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x451b1) [0xb76911b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4521f) [0xb769121f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7660d4e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x3b845) [0xb770a845] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_search_result_update_flags+0xe3) [0xb77320d3] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_sync_search_results_update+0x69) [0xb77394f9] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mailbox_sync_deinit+0x1f5) [0xb7738855] -> /usr/lib/dovecot/lib20_fts_plugin.so(+0xa0a6) [0xb74970a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x3a) [0xb770f7fa] -> /usr/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0xbf2) [0xb7487ac2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x39) [0xb770f769] -> dovecot/imap(imap_sync_init+0x54) [0x8060294] -> dovecot/imap() [0x8052262] -> dovecot/imap(cmd_idle+0xc3) [0x80523f3] -> dovecot/imap(command_exec+0x3d) [0x80591cd] -> dovecot/imap() [0x805815f] -> dovecot/imap() [0x8058230] -> dovecot/imap(client_handle_input+0x12d) [0x805847d] -> dovecot/imap(client_input+0x5f) [0x8058daf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x42) [0xb769ff92] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd3) [0xb76a0f43] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0xb769fa30] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2d) [0xb768880d] -> dovecot/imap(main+0x2b1) [0x8061c71] -> /lib/libc.so.6(__libc_start_main+0xe7) [0xb74de573] -> dovecot/imap() [0x804fa51] Oct 29 16:21:40 widder dovecot: imap(limon): Fatal: master: service(imap): child 8060 killed with signal 6 (core dumped) -------------- next part -------------- #0 0xf57fe416 in __kernel_vsyscall () No symbol table info available. #1 0xb74f1a1a in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb74f3014 in abort () from /lib/libc.so.6 No symbol table info available. #3 0xb76911c5 in default_fatal_finish (type=, status=) at failures.c:191 backtrace = 0x8df75a8 "/usr/lib/dovecot/libdovecot.so.0(+0x451b1) [0xb76911b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4521f) [0xb769121f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7660d4e] -> /usr/lib/dovecot/libdo"... #4 0xb769121f in i_internal_fatal_handler (ctx=0xbfaff584, format=0xb7767320 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfaff5a4 "?v\267Z") at failures.c:649 status = 0 #5 0xb7660d4e in i_panic (format=0xb7767320 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = 0xbfaff5a4 "?v\267Z" #6 0xb770a845 in mail_search_args_init_sub (args=, arg=0x8e6cee8, change_uidsets=false, search_saved_uidset=0x0) at mail-search.c:90 thread_args = keywords = {0x8e6cf40 "lists", 0x0} __FUNCTION__ = "mail_search_args_init_sub" #7 0xb77320d3 in search_result_update_search (changed_uids_arr=0x8e87030, result=0x8e58ed0) at index-search-result.c:69 search_ctx = changed_uids = 0x8e46b30 next_uid = 29224 ret = t = mail = changed_count = 1 changed_idx = 0 #8 index_search_result_update_flags (result=0x8e58ed0, uids=0x8e87030) at index-search-result.c:131 search_arg = {next = 0x8e6cee8, type = SEARCH_UIDSET, value = {subargs = 0x0, seqset = {arr = { buffer = 0x8e740e8, element_size = 8}, v = 0x8e740e8, v_modifiable = 0x8e740e8}, str = 0x0, time = 0, size = 0, flags = 0, search_flags = 0, date_type = 0, thread_type = MAIL_THREAD_NONE, keywords = 0x0, modseq = 0x0, search_args = 0x0, search_result = 0x0, mailbox_glob = 0x0}, context = 0x0, hdr_field_name = 0x0, match_not = 0, match_always = 0, nonmatch_always = 0, fuzzy = 0, result = 0} ret = 0 __FUNCTION__ = "index_search_result_update_flags" #9 0xb77394f9 in search_result_update (result=0x8e58ed0, ctx=0x8e87010) at index-sync-search.c:75 No locals. #10 index_sync_search_results_update (ctx=0x8e87010) at index-sync-search.c:88 results = 0x9060740 i = count = 3 #11 0xb7738855 in index_mailbox_sync_deinit (_ctx=0x8e87010, status_r=0xbfaffa3c) at index-sync.c:386 ctx = 0x8e87010 sync_rec = {seq1 = 3077094660, seq2 = 148987872, type = 0} delayed_expunges = false ret = 0 #12 0xb74970a6 in fts_sync_deinit (ctx=0x8e87010, status_r=0xbfaffa3c) at fts-storage.c:584 box = 0x9060580 fbox = 0x9060898 flist = 0x8e16060 ret = 0 #13 0xb770f7fa in mailbox_sync_deinit (_ctx=0xbfaffa40, status_r=0xbfaffa3c) at mail-storage.c:1347 ctx = box = 0x9060580 errormsg = error = ret = #14 0xb7487ac2 in virtual_sync_backend_box_sync (sync_flags=, bbox=0x8e632a8, ctx=0x8e8dda8) at virtual-sync.c:973 uidmap = sync_rec = {seq1 = 22114, seq2 = 22114, type = MAILBOX_SYNC_TYPE_FLAGS} idx1 = vuid = sync_ctx = 0x0 sync_status = {sync_delayed_expunges = 0} idx2 = vseq = 149273152 #15 virtual_sync_backend_box (bbox=0x8e632a8, ctx=0x8e8dda8) at virtual-sync.c:1067 sync_flags = status = {messages = 3077174859, recent = 148890672, unseen = 152726112, uidvalidity = 3215980904, uidnext = 3077353460, first_unseen_seq = 149046960, first_recent_uid = 149450720, last_cached_seq = 543664, highest_modseq = 13217152038154990465, keywords = 0x8df63a0, permanent_flags = 3077174635, nonpermanent_modseqs = 0, permanent_keywords = 0, allow_new_keywords = 1} ret = #16 virtual_sync_backend_boxes (ctx=0x8e8dda8) at virtual-sync.c:1399 bboxes = 0x9079798 i = count = 1 #17 virtual_sync (flags=0, mbox=0x8e62e18) at virtual-sync.c:1496 ctx = 0x8e8dda8 index_sync_flags = ret = #18 virtual_storage_sync_init (box=0x8e62e18, flags=0) at virtual-sync.c:1516 mbox = 0x8e62e18 sync_ctx = ret = #19 0xb770f769 in mailbox_sync_init (box=0x8e62e18, flags=0) at mail-storage.c:1324 _data_stack_cur_id = 4 ctx = #20 0x08060294 in imap_sync_init (client=0x8e17628, box=0x8e62e18, imap_flags=0, flags=0) at imap-sync.c:142 ctx = 0x8e5ba40 __FUNCTION__ = "imap_sync_init" #21 0x08052262 in idle_sync_now (box=, ctx=0x8e17eb8) at cmd-idle.c:145 __FUNCTION__ = "idle_sync_now" #22 0x080523f3 in cmd_idle (cmd=0x8e17e30) at cmd-idle.c:276 client = 0x8e17628 ctx = 0x8e17eb8 #23 0x080591cd in command_exec (cmd=0x8e17e30) at imap-commands.c:148 hook = 0x8dff260 ret = #24 0x0805815f in client_command_input (cmd=0x8e17e30) at imap-client.c:682 client = 0x8e17628 command = __FUNCTION__ = "client_command_input" #25 0x08058230 in client_command_input (cmd=0x8e17e30) at imap-client.c:733 client = 0x8e17628 command = __FUNCTION__ = "client_command_input" #26 0x0805847d in client_handle_next_command (remove_io_r=, client=0x8e17628) at imap-client.c:774 size = 12 #27 client_handle_input (client=0x8e17628) at imap-client.c:786 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #28 0x08058daf in client_input (client=0x8e17628) at imap-client.c:825 cmd = output = 0x8e16bfc bytes = 12 __FUNCTION__ = "client_input" #29 0xb769ff92 in io_loop_call_io (io=0x8f49090) at ioloop.c:379 ioloop = 0x8dfe400 t_id = 2 #30 0xb76a0f43 in io_loop_handler_run (ioloop=0x8dfe400) at ioloop-epoll.c:213 ctx = 0x8dfe5e0 events = event = 0x8dfe620 list = 0x8e16c90 io = tv = {tv_sec = 1791, tv_usec = 756031} events_count = 148991120 msecs = 1 ret = 1 i = j = call = #31 0xb769fa30 in io_loop_run (ioloop=0x8dfe400) at ioloop.c:398 No locals. #32 0xb768880d in master_service_run (service=0x8dfe330, callback=0x80612f0 ) at master-service.c:543 No locals. #33 0x08061c71 in main (argc=1, argv=0x8dfe1c0) at main.c:389 set_roots = {0x80645e0, 0x0} login_set = {auth_socket_path = 0x8df6060 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x8061720 , failure_callback = 0x8061430 } service_flags = storage_service_flags = username = c = From tss at iki.fi Mon Oct 29 23:39:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:39:33 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351545627.3435.71.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> Message-ID: <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> On 29.10.2012, at 23.20, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:13 +0200, Timo Sirainen wrote: >>> Great... and I expect that this ("no headers from the maildir files >> are >>> interpreted") applies also, when one "imports" mails the first time. >>> >>> With import I don't mean via IMAP, but plainly moving e.g. a maildir >> ++ >>> tree under dovecots mail location. >> >> Yeah. > So that means: From a "security" point of view, when using maildir (!) > there's no need to remove such headers, cause dovcote ignores them (on > maildir) always. Right?! Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). > I just wondered because when I looked through my mail archive (currently > as mentioned, under Evolution)... many emails already had X-UID and > X-IMAP* headers.... (set by the remote side, not by Evolution) ... and > in no case these should be able to mess around in my dovecot :) If you migrated from mbox format it could have brought those headers to maildir. They're anyway not used for anything by Dovecot. From tss at iki.fi Mon Oct 29 23:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:42:28 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351545330.3435.66.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> Message-ID: <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: > btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. From jk at jkart.de Mon Oct 29 23:43:08 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 22:43:08 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> Message-ID: <508EF86C.5070202@jkart.de> am 29.10.12 16:41 schrieb Timo Sirainen : > On 29.10.2012, at 17.23, Jim Knuth wrote: > >> I have here a problem with managesieve. With the login about >> webmail (roundcube) comes here in the log: > > You can always easily reproduce this? Can you get the network traffic logs between Roundcube and Dovecot and reproduce it by sending those same commands manually? > >> --snip >> dovecot: managesieve-login: Fatal: pool_system_realloc(4294967296): Out of memory >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 10157 returned error 83 (Out of memory (service managesieve-login { vsz_limit=1024 MB }, you may need to increase it)) > > Looks like there's a bug somewhere.. Doesn't it log a "raw backtrace"? > >> --snap >> I've increased of 2048M and the same above. >> then with 4096 M happens the following >> >> --snip >> dovecot: managesieve-login: Panic: epoll_ctl(add, 61538840) failed: Bad file descriptor >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) >> --snap > > Probably related to the first error. It would be helpful to get gdb backtraces from both of them, although from the first one you couldn't without patching + recompiling Dovecot. But from the second one I think you can get a core dump with: > > service managesieve-login { > executable = managesieve-login -D > } > > Then you can do something like: > > gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/managesieve-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/managesieve-login...done. (no debugging symbols found)...done. /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. (gdb) > bt full bt full No stack. (gdb) > -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning. [Churchill] From tss at iki.fi Mon Oct 29 23:46:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 29 Oct 2012 23:46:05 +0200 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508EF86C.5070202@jkart.de> References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> Message-ID: On 29.10.2012, at 23.43, Jim Knuth wrote: > ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core > /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. You'll of course need to have the core file first. Instead of: >> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) It should say (core dumped). Besides the executable change, you'll need to run "ulimit -c unlimited" just before dovecot binary. From calestyo at scientia.net Mon Oct 29 23:52:54 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:52:54 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> Message-ID: <1351547574.3435.74.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:39 +0200, Timo Sirainen wrote: > Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). Great... I think it would worth adding all this to: http://wiki2.dovecot.org/MailboxFormat/Maildir Is the wiki open for public editing (after registering an account)? Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Mon Oct 29 23:54:42 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 22:54:42 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> Message-ID: <1351547682.3435.76.camel@fermat.scientia.net> On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: > > btw: What are the actual advantages of sdbox over maildir? > > * Not moving files from new/ to cur/ directory > * Not renaming files when changing message flags > * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) > > Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. Oh that's quite some advantage... And I guess the interior of the files is the same? I.e. just the plain mail without any changes or quoting? For sdbox, does that part with "loosing the indexes means game over" ;) , too? Thanks, Chris -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jk at jkart.de Mon Oct 29 23:58:42 2012 From: jk at jkart.de (Jim Knuth) Date: Mon, 29 Oct 2012 22:58:42 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> Message-ID: <508EFC12.4000509@jkart.de> am 29.10.12 22:46 schrieb Timo Sirainen : > On 29.10.2012, at 23.43, Jim Knuth wrote: > >> ~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core >> /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. > > You'll of course need to have the core file first. Instead of: > >>> dovecot: managesieve-login: Fatal: master: service(managesieve-login): child 9777 killed with signal 6 (core dumps disabled) > > It should say (core dumped). Besides the executable change, you'll need to run "ulimit -c unlimited" just before dovecot binary. > If I run "ulimit -c unlimited" no problems more with Managesieve Login over Roundcube: Oct 29 22:50:46 srv1 dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=9354, secured, session= Oct 29 22:50:46 srv1 dovecot: managesieve(web1p1): Disconnected: Logged out bytes=120/177 Oct 29 22:53:16 srv1 dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=9418, secured, session= Oct 29 22:53:16 srv1 dovecot: managesieve(web1p1): Disconnected: Logged out bytes=44/145 But the same: srv1:~# gdb /usr/lib/dovecot/managesieve-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/managesieve-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/managesieve-login...done. (no debugging symbols found)...done. /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. (gdb) bt full No stack. (gdb) q -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Ein Tag an dem Du nicht l?chelst, ist ein verlorener Tag. (Charly Chaplin) From tss at iki.fi Tue Oct 30 00:05:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 00:05:42 +0200 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351547682.3435.76.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <1351547682.3435.76.camel@fermat.scientia.net> Message-ID: On 29.10.2012, at 23.54, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: >>> btw: What are the actual advantages of sdbox over maildir? >> >> * Not moving files from new/ to cur/ directory >> * Not renaming files when changing message flags >> * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) >> >> Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. > > Oh that's quite some advantage... > > And I guess the interior of the files is the same? I.e. just the plain > mail without any changes or quoting? Yes, but it's in dbox format so it contains also some extra metadata (not in the mail headers). > For sdbox, does that part with "loosing the indexes means game > over" ;) , too? You'll lost message flags then. Both sdbox and mdbox keep dovecot.index.backup files and repairing tries very hard to preserve everything from the indexes it sees, so I don't think it's a big concern as long as the system behaves properly. From tss at iki.fi Tue Oct 30 00:08:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 00:08:28 +0200 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <1351547574.3435.74.camel@fermat.scientia.net> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> Message-ID: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> On 29.10.2012, at 23.52, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:39 +0200, Timo Sirainen wrote: >> Right. The only special case is X-UIDL: header, which is used for POP3 UIDLs but only if pop3_reuse_xuidl=yes (which isn't really recommended nowadays as there are other ways to do it). > > Great... I think it would worth adding all this to: > http://wiki2.dovecot.org/MailboxFormat/Maildir Well, that isn't really maildir-specific. It's pop3 specific that is done with all mailbox formats. pop3_reuse_xuidl setting's comments should probably warn about the possibility of receiving unwanted X-UIDL headers in new mails. > Is the wiki open for public editing (after registering an account)? Yes. You don't even need to register. From calestyo at scientia.net Tue Oct 30 00:16:55 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 23:16:55 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> Message-ID: <1351549015.3435.80.camel@fermat.scientia.net> On Tue, 2012-10-30 at 00:08 +0200, Timo Sirainen wrote: > > Great... I think it would worth adding all this to: > > http://wiki2.dovecot.org/MailboxFormat/Maildir > > Well, that isn't really maildir-specific. It's pop3 specific that is > done with all mailbox formats. pop3_reuse_xuidl setting's comments > should probably warn about the possibility of receiving unwanted > X-UIDL headers in new mails. No I meant _everything_.. i.e. that dovecote never interprets these message headers when using maildir... unless for that one case when using POP3 on maildir and when pop3_reuse_xuidl is set to yes. I'll make some chances and post you the diff links here, so you can check this and correct if something's wrong. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From janfrode at tanso.net Tue Oct 30 00:26:29 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 29 Oct 2012 23:26:29 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> +1 Better to be lenient, than to confuse users by accepting some but not other messages. I believe most larger mail providers has a max message size of around 64MB or less, so allowing the final message to exceed quota by about that sounds reasonable to me. -jf From calestyo at scientia.net Tue Oct 30 00:31:20 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Mon, 29 Oct 2012 23:31:20 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> Message-ID: <1351549880.3435.81.camel@fermat.scientia.net> I think it should be configurable by how much (either a fixed space or relative to the quota) the last mail may be larger than the quota.... but then... +1 as well :) Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From calestyo at scientia.net Tue Oct 30 01:13:45 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 00:13:45 +0100 Subject: [Dovecot] does dovecot interpret any mail headers specially with maildir? In-Reply-To: <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> References: <1351542708.3435.25.camel@fermat.scientia.net> <4CA5B70B-04CE-4288-9624-CDEFBDA2DE2C@iki.fi> <1351544951.3435.61.camel@fermat.scientia.net> <0E5B57D8-05F4-4459-A983-5DB8C0F60C26@iki.fi> <1351545627.3435.71.camel@fermat.scientia.net> <4189DF4F-E808-447B-9702-3FB511829668@iki.fi> <1351547574.3435.74.camel@fermat.scientia.net> <2CD68F26-C6E1-4DDA-ADBD-1C081700EEAB@iki.fi> Message-ID: <1351552425.3435.83.camel@fermat.scientia.net> Please have a look at: http://master.wiki2.dovecot.org/MailboxFormat/mbox?action=diff&rev2=17&rev1=16 http://master.wiki2.dovecot.org/MailboxFormat/Maildir?action=diff&rev2=45&rev1=44 whether it's correct. Oh and... I'd assume that everything I've added for maildir also applies to the dbox formats? If so, I'd add the text there, too. Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From bob at computerisms.ca Tue Oct 30 01:23:16 2012 From: bob at computerisms.ca (Bob Miller) Date: Mon, 29 Oct 2012 16:23:16 -0700 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <1351552996.2097.57.camel@worklian> +1 to one last mail, though it would be nice if the over percentage could be configurable... -- Computerisms Bob Miller 867-334-7117 / 867-633-3760 http://computerisms.ca On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. > From sven at svenhartge.de Tue Oct 30 01:36:08 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 30 Oct 2012 00:36:08 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail > that is over 1MB, Dovecot rejects the mail. But smaller mails aren't > rejected probably for days. So user might not even realize that they > didn't receive one of the mails. Also having a user "almost over > quota" is a rather strange state I think. > So what do you think about v2.2 allowing delivery of one last mail > even if it brings the user over quota? Except add a limit that if the > message size is as much as the user's entire quota limit it wouldn't > be added (or 50% or ..?). Also IMAP wouldn't allow this, since user > would get an error anyway. I could make this also optional, but if > nobody really wants to keep the old behavior there's really no point > in adding the option. Yes, please add this new option. If possible with configurable limit. I'd rather have a user go directly over quota with one final mail than have a situation where half the mails get delivered and the other half is rejected. >From a 1st level support stand point this new behavior is easier to explain than the way it is now. By looking into my new crytal ball I can see the following happening: A user with 300KBytes under his quota gets a mail with 500KBytes in size. This of course bounces. He is then called by the sender who complains about the full mailbox. The user then sends himself a test mail (Subject: Test, Body: Test) which is delivered, because it is rather small and fits inside the few bytes left. The user then is confused. (And I have to use some of my precious time to explain to the user the inner workings of the mail system. ;)) So I'd very much appreciate such an option. Gr??e, Sven. -- Sigmentation fault. Core dumped. From noel.butler at ausics.net Tue Oct 30 01:43:30 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 30 Oct 2012 09:43:30 +1000 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <1351554210.7884.5.camel@tardis> On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? +1 only if configurable, and with an additional configurable quota percentage value option for those that do enable the function. In 99.9% of cases I could never see a service provider wanting this, but some small private businesses perhaps might see a benefit in it. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From sven at svenhartge.de Tue Oct 30 01:48:32 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 30 Oct 2012 00:48:32 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1351554210.7884.5.camel@tardis> Message-ID: Noel Butler wrote: > On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: >> So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? > +1 only if configurable, and with an additional configurable quota > percentage value option for those that do enable the function. > In 99.9% of cases I could never see a service provider wanting this, > but some small private businesses perhaps might see a benefit in it. If your user quota is 1GiB (which is not big, if you look at todays user quotas even at freemail providers) and the max mail size 30MiB, then a users max mailbox size would then be 1054MiB. Not an unreasonable price to pay for an easier to understand error condition, IMHO. Gr??e, Sven. -- Sigmentation fault. Core dumped. From calestyo at scientia.net Tue Oct 30 02:16:05 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 01:16:05 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <1351547682.3435.76.camel@fermat.scientia.net> Message-ID: <1351556165.3435.88.camel@fermat.scientia.net> On Tue, 2012-10-30 at 00:05 +0200, Timo Sirainen wrote: > > And I guess the interior of the files is the same? I.e. just the plain > > mail without any changes or quoting? > Yes, but it's in dbox format so it contains also some extra metadata (not in the mail headers). Yeah of course... but the important point here is the "not in the mail headers" part :) So I've added the following changes, please double check :) http://master.wiki2.dovecot.org/MailboxFormat/dbox?action=diff&rev2=30&rev1=29 > > For sdbox, does that part with "loosing the indexes means game > > over" ;) , too? > You'll lost message flags then. Both sdbox and mdbox keep > dovecot.index.backup files and repairing tries very hard to preserve > everything from the indexes it sees, so I don't think it's a big > concern as long as the system behaves properly. Yeah... sounds not too bad... :) Off topic: Have you ever thought about adding a "real" DB backend? Nothing against dbox... :) ... and I have no performance comparison of dbox with what could be done with a DBMS... but the advantage of the later would be that you get all fancy features from database systems for free... like fast indexing, online replication, etc. p.. One might even reuse something like AOX for this. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From noel.butler at ausics.net Tue Oct 30 02:27:58 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 30 Oct 2012 10:27:58 +1000 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <1351554210.7884.5.camel@tardis> Message-ID: <1351556878.7884.20.camel@tardis> On Tue, 2012-10-30 at 00:48 +0100, Sven Hartge wrote: > Noel Butler wrote: > > On Mon, 2012-10-29 at 22:39 +0200, Timo Sirainen wrote: > > >> So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? > > > +1 only if configurable, and with an additional configurable quota > > percentage value option for those that do enable the function. > > > In 99.9% of cases I could never see a service provider wanting this, > > but some small private businesses perhaps might see a benefit in it. > > If your user quota is 1GiB (which is not big, if you look at todays user > quotas even at freemail providers) and the max mail size 30MiB, then a > users max mailbox size would then be 1054MiB. > > Not an unreasonable price to pay for an easier to understand error > condition, IMHO. > Sven , That's nice when it's one or ten, but you need to look at the big picture, what about 300K users, all doing the same. Also, as to mail sizes, in decades gone by with dialup it was 5mb, now days with DSL, Cable, FTTN etc, many that I know of use 50mb mail sizes because that takes mere seconds now days. Don't forget, in some countries, hardware is still incredibly (criminally) overpriced, a 600G drive from HP in the U.S. is about 350 odd last time I looked, probably lot cheaper now, in this country (AU), the same drive today is still around 800, and that was when our dollar was 1.07 to the U.S. 1.00, even with taxes and customs and transport, some so and so's are still making an absolute massive killing in profits. Of course the more appropriate way would be like most of us do now, send the warning messages, if the users can not be bothered to keep an eye on their quota or act when they get mailbox almost/now full warnings, why is it our problem :) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From calestyo at scientia.net Tue Oct 30 02:42:25 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 01:42:25 +0100 Subject: [Dovecot] mbox2mdir... what about UIDs/etc? (was: how to best import Evolution/Thunderbird mail into dovecot?) In-Reply-To: References: <1350429674.3360.27.camel@fermat.scientia.net> <20121017145144.GA777@PC211.ikt.de> Message-ID: <1351557745.3435.106.camel@fermat.scientia.net> Hi again :) In the meantime I made some checks[0] on how much storage one looses by using maildir (compared to mbox)... and decided that it's much but I can live with it. This of course doesn't solve my problems that I have a possibly a mix of different mbox subformats, a mix of different mail status formats (Thunderbird and Evolution)... and some 17k mails that suffered from From_ line corruption (due to Evolution, getmail and postfix either incorrectly quoting them or even intentionally using mboxo)... so I'll still need some scripting in the end. Which I'll base upon mb2md[1] respectively it's Dovecot-izsed version[2]. I diffed the two, and it seems the only differences are that the later handles the following in addition: 1) keywords (via X-IMAP, X-IMAPbase and X-Keywords) 2) UIDs, UIDVALITIDYs and UIDLASTs (via the X-IMAP, X-IMAPbase and X-UID mail headers of the mboxes 3) ,S= and ,W= tags (Guess that's it right?) Now I have some questions: to 1) I never used keywords on mails myself so far,... so if any X-Keywords headers exist, these were sent from remote. So I guess I _really want_ to ignore them (and not let remote people set my local keywords), right? to 2) I haven't had time yet to read into the IMAP4 RFC (though I'll need to do so soon),... but AFAIU the UIDs, UIDVALITIDYs and UIDLASTs are used for the server/clients to identify which message they talk about and avoid unnecessary reloading and to assure statuses are set on the right message, etc. All mails that I migrate were only used locally by one client. So I guess I can fully ignore any UID/UIDVALITIDY/UIDLAST preservation, right? So in principle I can use plain mb2md (without the dovecot mods)... and simply convert all my mboxes to maildir, put them in the dovecot mail (having the mails in the ../new dirs) location and start dovecot, right? Now will dovecot itself assign fresh consecutive UIDs to all maildir files? Or will I get into troubles? to 3) If dovecot can make use of these,.. I'm happy with having them set, but analogous to (2): If I use plain mb2md (without the dovecot mods)... and simply convert all my mboxes to maildir, put them in the dovecot mail (having the mails in the ../new dirs) location and start dovecot.... Can I make dovecot to calculate these fields by itself when it loads? Thanks, Chris. [0] http://dovecot.org/pipermail/dovecot/2012-October/069130.html [1] http://batleth.sapienti-sat.org/projects/mb2md/ [2] http://dovecot.org/tools/mb2md.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5450 bytes Desc: not available URL: From jtam.home at gmail.com Tue Oct 30 04:09:12 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 29 Oct 2012 19:09:12 -0700 (PDT) Subject: [Dovecot] Changing password for users In-Reply-To: References: Message-ID: Ben Morrow wrote: >> Maybe replace "/usr/bin/passwd" with htpasswd? > > Try pam_pwdfile with poppwd or some other poppassd that supports PAM. That's it! I was trying to remember the name of this PAM module. >>> and is there another way other than poppassd? >> >> Write your own PHP script -- it couldn't be more than a few dozen lines >> of code for a working skeleton. Or Google "php change password htpasswd". > > It's not as simple as you seem to think. Quite apart from getting the > password-changing itself right (have you considered what happens when > two users change their passwords at the same time? when Dovecot tries to > read the password file at the same time as you are changing it? when the > system crashes when you are halfway through rewriting the password > file?), you really shouldn't be running PHP as a user with write access > to a password file (even a virtual password file) in any case. I did consider it, and you're right, it is tricky to get it absolutely right. If robusteness and security was of utmost importance, I would abandon PHP too. I was scaling the solution to the OP's technical ability and apparent size of their operation -- if poppwd passes muster, this wouldn't be too far off. Joseph Tam From tony.blue.mailinglist at gmx.de Tue Oct 30 07:33:22 2012 From: tony.blue.mailinglist at gmx.de (tony.blue.mailinglist at gmx.de) Date: Tue, 30 Oct 2012 06:33:22 +0100 Subject: [Dovecot] dovecot-lda not correct folder Message-ID: <508F66A2.7010809@gmx.de> Hello, i use dovecot with maildir. The maildir-folder looks like this: vmail/mail/user1/cur vmail/mail/user1/.Sent vmail/mail/user1/.optionalfolder Procmail should put some definded mails in the "optionalfolder" My uses configuration does not do this: ... DELIVERMAIL="/usr/lib/dovecot/dovecot-lda" IMAP="$DELIVERMAIL -e -d $LOGNAME -m INBOX" ZUSATZORDNER="$DELIVERMAIL -e -d $LOGNAME -m .optionalfolder" ... dovecot-lda puts the mails for the optionalfolder always in the .cur (INBOX). What?s the correct dovecot-lda parameter to put the mails in the optionalfolder? Thank you! Tony From slusarz at curecanti.org Tue Oct 30 09:19:07 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 30 Oct 2012 01:19:07 -0600 Subject: [Dovecot] Save/restore IMAP session state In-Reply-To: <1351529580.13571.93.camel@hurina> References: <1351529580.13571.93.camel@hurina> Message-ID: <20121030011907.Horde.5xjiGoF5lbhQj39rg9FXuZA@bigworm.curecanti.org> Quoting Timo Sirainen : > A week ago I thought I'd see > how easy it would be to implement this. I got a basic proof of concept > working as a "X-STATE" command. [snip] > This could also be used to implement quick session state restoring for > webmails (as suggested by Michael Slusarz). Wow. We must have some sort of crazy mind-meld going on: I have been working on this concept the last few days with the idea of generating some sort of draft proposal to provoke further discussion going forward. I would say great minds think alike, but that would be giving myself too much credit. So this provided the necessary motivation to finish the draft concept. Now that I have (somewhat) figured out the RFC 2629 XML format for doing this kind of thing, hopefully this has is presented in a somewhat coherent format. The draft, which is significantly more comprehensive than your suggestion, can be found here: https://raw.github.com/slusarz/horde-sandbox/master/imap-state-draft/draft-imap-state-00.txt Not sure if this should remain the forum for discussing this concept, or if we should move to private messages (or even to the imap-protocol list). Let me know your thoughts on this (or anyone else with an interest). michael From crohmann at netcologne.de Tue Oct 30 09:53:06 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 30 Oct 2012 08:53:06 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <508F8762.4040109@netcologne.de> On 29.10.2012 21:39, Timo Sirainen wrote: > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. Great idea. This makes being over quota a stable state and makes it easier for users to understand their "problem". Regards Christian From zybi at talex.pl Tue Oct 30 11:17:03 2012 From: zybi at talex.pl (=?UTF-8?B?QXJ0dXIgWmFwcnphxYJh?=) Date: Tue, 30 Oct 2012 10:17:03 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <508F9B0F.30108@talex.pl> Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option. > This will finally make possible to reject RCPT TO: before the message size is known instead of accepting the message and sending a bounce later (bouncing SPAM is not good). -- Talex Sp??ka Akcyjna z siedzib? w Poznaniu adres: ul. Karpia 27d, 61-619 Pozna? NIP 782-00-21-045 zarejestrowana w S?dzie Rejonowym Pozna? ? Nowe Miasto i Wilda w Poznaniu VIII Wydzia? Gospodarczy - KRS pod nr 000048779 kapita? zak?adowy: 3.000.092,00 PLN (w ca?o?ci wp?acony) Uwaga: Niniejsza wiadomo??, w szczeg?lno?ci jej tre?? oraz za??czniki, mo?e by? poufna. W przypadku, gdy nie jest Pan/Pani zamierzonym jej adresatem, informujemy, ?e wszelkie rozpowszechnianie, dystrybucja lub powielanie powy?szej wiadomo?ci jest zabronione. Jednocze?nie prosimy o powiadomienie nadawcy oraz niezw?oczne usuni?cie powy?szej wiadomo?ci wraz z za??cznikami. Dzi?kujemy, Talex S.A. w Poznaniu. Confidentiality Notice: This email, particularly its content and any attached files, may be confidential. If you are not an intended recipient, any disclosure, distribution and reproduction of this message is prohibited. In this case please notify the sender immediately and then delete this message and any attachments. Thank you, Talex S.A., Poznan. From Ralf.Hildebrandt at charite.de Tue Oct 30 11:42:36 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 30 Oct 2012 10:42:36 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> Message-ID: <20121030094236.GG25787@charite.de> * Jan-Frode Myklebust : > > > +1 > > Better to be lenient, than to confuse users by accepting some but not other messages. Amen to that! +1 -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From cr at sys4.de Tue Oct 30 12:11:14 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 11:11:14 +0100 Subject: [Dovecot] copymail deleted Message-ID: Hi, I had enabled an option in dovecot. mail_attachment_dir = /var/mail/virtual/copymail/attachments After a while I checked /var/mail/virtual and did some cleanup. I did not remember that copymail was specified in dovecot and erased it. Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: file_istream.stat(/var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6) failed: No such file or directory Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: istream-concat: Failed to get size of stream /var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: read() failed: Invalid argument (FETCH for mailbox INBOX UID 196) Oct 30 10:56:05 mx0 dovecot: imap(hidden): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-30 10:56:05] in=150 out=950 I have Bacula and have restored most of the stuff, but obviously not all files. That is not too important. But I do not know, how to tell dovecot that it may "forget" about files that produce a "No such file or directory" error. Can I do some "rescan/rebuild" in dovecot? Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Tue Oct 30 12:19:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 12:19:31 +0200 Subject: [Dovecot] copymail deleted In-Reply-To: References: Message-ID: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> On 30.10.2012, at 12.11, Christian R??ner wrote: > Oct 30 10:56:05 mx0 dovecot: imap(hidden): Error: file_istream.stat(/var/mail/virtual/copymail/attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6) failed: No such file or directory > > I have Bacula and have restored most of the stuff, but obviously not all files. That is not too important. But I do not know, how to tell dovecot that it may "forget" about files that produce a "No such file or directory" error. > > Can I do some "rescan/rebuild" in dovecot? Currently you can't in any easy way. The easiest fix for now I think would be to write a script that reads through dbox files, parses the attachment metadata lines and recreates the missing files with the original size (e.g. sparse-0-filled). The dbox parsing can be done easily with: doveadm dump m.1 | grep ^msg.ext-ref The format is: 1*( ) If the options="-" then the byte count is the final size. If options="B" then byte count is the base64-encoded size while the original file has to be base64-decoded size. From ef at math.uni-bonn.de Tue Oct 30 12:42:36 2012 From: ef at math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) Date: Tue, 30 Oct 2012 11:42:36 +0100 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <20121030104235.GL15430@trav.math.uni-bonn.de> Sounds like a reasonable idea, but one has to keep in mind that file system quotas never work that way. So that change would make quota=fs behave differently from the rest. So it should at least be configurable, I think. From sheng-wei.lim at proximityjobs.com Tue Oct 30 11:51:55 2012 From: sheng-wei.lim at proximityjobs.com (sheng-wei.lim) Date: Tue, 30 Oct 2012 17:51:55 +0800 Subject: [Dovecot] Problem about SSL for Dovecot. Message-ID: <000001cdb684$34e16f20$9ea44d60$@proximityjobs.com> Hi All, With the below setting (cropped), the ssl certificate(signed by godaddy) don?t seems to work. It will still ask me if I want to accept this certificate. I have use the same certificate for apache host and postfix it works without any prompt. dovecot version : 2.0.19 # OS: Linux 3.2.0-32-generic-pae i686 Ubuntu 12.04.1 LTS Dovecot config: ssl = required ssl_cert = References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> Message-ID: <508FB360.5090704@Media-Brokers.com> On 2012-10-29 5:42 PM, Timo Sirainen wrote: > On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: > >> btw: What are the actual advantages of sdbox over maildir? > * Not moving files from new/ to cur/ directory > * Not renaming files when changing message flags > * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a lot with this) > > Basically less disk I/O and making it possible to have mailboxes with a huge number of messages without everything slowing down horribly. > I had been wanting to ask about this too... So... what are the disadvantages? -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Oct 30 13:03:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 30 Oct 2012 07:03:02 -0400 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <1351544049.3435.47.camel@fermat.scientia.net> References: <1351544049.3435.47.camel@fermat.scientia.net> Message-ID: <508FB3E6.6030304@Media-Brokers.com> On 2012-10-29 4:54 PM, Christoph Anton Mitterer wrote: > In the end I probably changed my opinion. > ~7GB of wasted block space for all my mails is actually quite a lot, but > in days of cheap disk space it's acceptable. > And with mbox one has IMHO the major disadvantage that mailservers > (including dovecot) store some meta-data_in_ it (i.e. in the mails > themselves) , which I don't like a lot. > I still think about reports that mbox is much faster with full text > search (which sounds reasonable)... but therefore one needs probably and > database backend anyway. What makes the most sense for me is to use mbox (or mdbox) for longer term storage that you may be offloading to slower storage systems, and use maildir (or sdbox) for the new mails... Would work great as long as you have a reliable method for archiving older mails out to your slower storage. This is what I plan on doing someday... -- Best regards, Charles From simon.buongiorno at gmail.com Tue Oct 30 13:11:37 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Tue, 30 Oct 2012 07:11:37 -0400 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <20121030094236.GG25787@charite.de> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> <86FAD77C-0932-4DB2-9747-B14770C8E547@tanso.net> <20121030094236.GG25787@charite.de> Message-ID: On Oct 30, 2012 5:43 AM, "Ralf Hildebrandt" wrote: > > * Jan-Frode Myklebust : > > > > > > +1 > > > > Better to be lenient, than to confuse users by accepting some but not other messages. > > Amen to that! +1 Surely the answer is that as soon as any mail is rejected an over-quota message is injected? That way, the quota remains as it currently is, but the user will a) be aware that he's over or nearly over quota, b) that a mail was rejected for being too big (if you inject the right over-quota message). Simon From calestyo at scientia.net Tue Oct 30 13:30:29 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 12:30:29 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <508FB360.5090704@Media-Brokers.com> References: <1351544049.3435.47.camel@fermat.scientia.net> <9D29C5F7-A6BC-4D74-AAA9-14675035D09C@iki.fi> <1351545330.3435.66.camel@fermat.scientia.net> <22F4E090-F572-40F4-8B69-D48E48856815@iki.fi> <508FB360.5090704@Media-Brokers.com> Message-ID: <1351596629.7808.5.camel@heisenberg.scientia.net> On Tue, 2012-10-30 at 07:00 -0400, Charles Marcus wrote: > So... what are the disadvantages? I (but I'm no expert) would guess that it's a dovecot-only format. No support from most other tools,... I'd guess you cannot use e.g. maildrop with it, or can you? I personally was always a bit worried, when meta-data is put in the mail... now AFAIU dbox does _not_ do this... and you can cleanly extract each unmodified mail from the dbox fail (single or multi), right? Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From calestyo at scientia.net Tue Oct 30 13:31:40 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Tue, 30 Oct 2012 12:31:40 +0100 Subject: [Dovecot] mbox vs. maildir storage block waste In-Reply-To: <508FB3E6.6030304@Media-Brokers.com> References: <1351544049.3435.47.camel@fermat.scientia.net> <508FB3E6.6030304@Media-Brokers.com> Message-ID: <1351596700.7808.6.camel@heisenberg.scientia.net> On Tue, 2012-10-30 at 07:03 -0400, Charles Marcus wrote: > What makes the most sense for me is to use mbox (or mdbox) for longer > term storage that you may be offloading to slower storage systems, and > use maildir (or sdbox) for the new mails... Was also something I thought about... still the more I think about it, the more I hate, that with mbox meta-data is stored in the mails. > Would work great as long as you have a reliable method for archiving > older mails out to your slower storage. I still hope for some DB backend ;) Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From cgregoir99 at yahoo.com Tue Oct 30 14:01:47 2012 From: cgregoir99 at yahoo.com (Christian Gregoire) Date: Tue, 30 Oct 2012 12:01:47 +0000 (GMT) Subject: [Dovecot] POP3 Proxy : user format not accepted Message-ID: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Hello, I want to use Dovecot as a POP3 proxy (http://wiki.dovecot.org/HowTo/ImapProxy). All is working fine on my sample platform, except that I have plenty (several thousands) of users that login using local_part#domain, instead of local_part at domain, which is an old setting on my POP3 server. And in that case, Dovecot returns 'Authentication failed'. Here is my proxy table : mysql> select * from tbl_proxy; +--------------------+-------------+--------------------+ | user ? ? ? ? ? ? ? | host ? ? ? ?| destuser ? ? ? ? ? | +--------------------+-------------+--------------------+ | christian at mydom.fr | 10.10.100.1 | christian at mydom.fr | | christian#mydom.fr | 10.10.100.1 | christian at mydom.fr | +--------------------+-------------+--------------------+ If I login on the Dovecot proxy with the '@' version, everything is fine : root : ~> telnet?10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian at mydom.fr +OK pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) And MySQL logs show the query : 121030 12:55:28 ? ? 3 Query ? ? SELECT NULL AS password, host, destuser, 'Y' AS nologin, 'Y' AS nodelay, 'Y' AS proxy, 'Y' AS nopassword FROM tbl_proxy WHERE user = 'christian at mydom.fr' If I login on the Dovecot proxy with the '#' version, it fails : root : ~> telnet 10.10.100.24 110 Trying 10.10.100.24... Connected to 10.10.100.24 (10.10.100.24). Escape character is '^]'. +OK Dovecot ready. user christian#mydom.fr +OK pass azerty42 -ERR Authentication failed. And nothing shows up the the MySQL logs. If I login directly on the POP server with the same credentials, no problem : root : ~> telnet?10.10.100.1 110 Trying 10.10.100.1... Connected to?10.10.100.1?(10.10.100.1). Escape character is '^]'. +OK Welcome to POP3 Server V 2.06. Authenticate yourself. user christian#mydom.fr +OK Password required for christian#mydom.fr pass azerty42 +OK christian#mydom.fr has 3 messages (3561 octets) Any idea ? Is the '#' not internally supported in the user login by Dovecot ? Thanks in advance. Christian From tss at iki.fi Tue Oct 30 15:08:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 15:08:37 +0200 Subject: [Dovecot] POP3 Proxy : user format not accepted In-Reply-To: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> References: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Message-ID: On 30.10.2012, at 14.01, Christian Gregoire wrote: > Any idea ? Is the '#' not internally supported in the user login by Dovecot ? See auth_username_chars setting. From chris at dotchristopher.com Tue Oct 30 15:03:55 2012 From: chris at dotchristopher.com (Chris Smith) Date: Tue, 30 Oct 2012 14:03:55 +0100 Subject: [Dovecot] Dovecot does not update acl_shared_dict file Message-ID: <20121030140355.Horde.LhzrQUVMXLlQj9A7c15yx4A@www.dotchristopher.com> Hi all, Firstly, thanks for all your effort with this software. Much appreciated. I am having a slight issues trying to enable reading of other users mailboxes. The docs are a little sparse for those that aren't mailadmin heros, I wonder if anyone could please help me see where I am going wrong. I would like to allow some users to list and read the mailboxes of others. E.g: When User1 logs in, they are presented with a list of their own folders, and those of User2, User3, etc to which they have (e.g. read/list) access I followed the instructions on the wiki (for Version 1.X)- there are two concepts here: 1. Grant some [e.g. list/read] access on a mailbox folder to a particular user 2. Enable the acl_shared_dict to allow dovecot to track (and display to IMAP clients) the folders to which they have access. This will take the form of a BDB file: /var/lib/dovecot/shared-mailboxes.db This file can only be updated by using the SETACL command. *This is where I have the problem*. The SETACL command does nothing. No matter how hard I try, I cannot get dovecot to update this file. The only indication I have that the file exists and is readable by dovecot is the disappearance from the logs of the line: No acl_shared_dict setting - shared mailbox listing is disabled So I can see that dovecot knows the file is there because it does not complain any more. The file is there (because I created it myself, although it is empty: size = 0), and in a moment of weakness I made sure it could be updated (this will be changed back!): ls -l /var/lib/dovecot/shared-mailboxes.db -rwxrwxrwx 1 dovecot dovecot 0 2012-10-30 12:27 /var/lib/dovecot/shared-mailboxes.db But if I try to update the ACL, absolutely nothing happens: [ > command ] [ < reply ] > telnet localhost 143 < * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=CRAM-MD5] Email server > a login [User2] [pass] < a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH ACL RIGHTS=texk] Logged in > a SETACL Inbox [User 1] rl < a OK Setacl complete. At this time, in the dovecot log (with debug enabled): 2012-10-30 13:55:24 IMAP([User2]): Info: Namespace : Using permissions from /home/mailboxes/[domain]/[User1]: mode=0770 gid=-1 2012-10-30 13:55:24 IMAP([User2]): Info: acl vfile: reading file /home/mailboxes/[domain]/[User1]/dovecot-acl 2012-10-30 13:55:24 IMAP([User2]): Info: acl vfile: reading file /home/mailboxes/[domain]/[USer1]/dovecot-acl Can anyone please help me track down what I'm doing wrong here?! It's driving me mad! Thanks for your time if you've read this far!! As requested: dovecot --version 1.2.9 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot info_log_path: /var/log/dovecot.info log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps imap listen: 127.0.0.1:143 ssl_listen: 37.235.54.98 ssl_cert_file: /etc/ssl/dovecot.crt ssl_key_file: /etc/ssl/private/dovecot.key login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_greeting: Email server valid_chroot_dirs: /var/spool/vmail mail_location: maildir:/home/mailboxes/%d/%n mail_debug: yes mbox_write_locks: fcntl dotlock mail_plugins: acl imap_acl lda: postmaster_address: [ valid at ddress ] auth default: mechanisms: plain cram-md5 verbose: yes passdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: passwd-file args: /etc/dovecot/users socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix master: path: /var/spool/postfix/private/auth-master mode: 384 user: postfix group: postfix plugin: acl: vfile acl_shared_dict: file:/var/lib/dovecot/shared-mailboxes.db acl: vfile:/etc/dovecot/acls From cr at sys4.de Tue Oct 30 15:28:22 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 14:28:22 +0100 Subject: [Dovecot] copymail deleted In-Reply-To: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> Message-ID: <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> > The format is: > > 1*( ) > > If the options="-" then the byte count is the final size. If options="B" then byte count is the base64-encoded size while the original file has to be base64-decoded size. Ok, so far I have "grep'ed" this here: msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 1443213 550635 B76 56/f2/56f25e225385902f3fc5185dc3d0103f59b34d14-b134401e794009503a0400002cb72ff6 1994019 477177 B76 c4/36/c436874b56cf3cd105e82f9243c7eac53c467f32-b234401e794009503a0400002cb72ff6 2561522 1075531 B76 77/af/77af1045a783308dbbf2f8a464c5136a0407e720-b334401e794009503a0400002cb72ff6 3715582 1195635 B76 99/33/99339b17a21ce052cd8f47f1d88c6e869cc1650b-b434401e794009503a0400002cb72ff6 4966686 715386 B76 fe/df/fedf23091720d3fa649af3bd6537e66304b8061a-b534401e794009503a0400002cb72ff6 5805913 788086 B76 ab/36/ab36f53a443f1855bc13caaba9e01e9464b2921f-b634401e794009503a0400002cb72ff6 6684258 906273 B76 10/70/1070d21039bc3f305bb948315a01344eefb2a465-b734401e794009503a0400002cb72ff6 7590707 204613 B76 39/44/394402c057791482f79351363f025ae0a7caf1b0-b834401e794009503a0400002cb72ff6 7795492 1349911 B76 41/bd/41bd01b4880065e5136cafbd1d191a1f8a1ead55-b934401e794009503a0400002cb72ff6 9271435 1504539 B76 c6/71/c671c1367e843741a2cc8f083a37231522d37640-ba34401e794009503a0400002cb72ff6 10877759 357555 B76 58/f5/58f582d2644025b843cf991f5cf783d27f9d90c9-bb34401e794009503a0400002cb72ff6 11826037 890683 B76 82/da/82dabbe06f269e7c79417db3b570246a648d2139-bc34401e794009503a0400002cb72ff6 msg.ext-ref = 118947 317624 B76 ad/9b/ad9be52e11433cd0337cda13bf0a458fd0fd948d-df905c0cd33d0950ae7800002cb72ff6 436770 139669 B76 78/15/781526d896a0530a5e76ebce65f2eb690d102dd3-e0905c0cd33d0950ae7800002cb72ff6 576610 457829 B76 61/3a/613a70c8515c572a04211fb0c63828d9c9acfb70-e1905c0cd33d0950ae7800002cb72ff6 1107667 410786 B76 7f/6b/7f6b7ee9b08a73600d98e8583aae343a90e76b96-e2905c0cd33d0950ae7800002cb72ff6 1611186 816686 B76 ff/ff/ffff9362c5356d8bedb17bd56edf0524bd0ae7b3-e3905c0cd33d0950ae7800002cb72ff6 2516232 643918 B76 4f/aa/4faa153fada5ceea79016cf2eadc1d05110f3f2e-e4905c0cd33d0950ae7800002cb72ff6 3291363 1036359 B76 e6/f3/e6f342bf28e8edfd3214666aaa52f0c067bae22b-e5905c0cd33d0950ae7800002cb72ff6 4418344 668813 B76 20/78/2078c98fb9bcadeeaa49bc38dc31548142fc71b1-e6905c0cd33d0950ae7800002cb72ff6 5154786 502218 B76 40/f4/40f4af3ad2077493caa34faabb201531609b50c4-e7905c0cd33d0950ae7800002cb72ff6 5782912 628591 B76 cc/a9/cca98a2a325f1be9a398d62890836cf11f267c4b-e8905c0cd33d0950ae7800002cb72ff6 6518382 526201 B76 17/47/1747a90b58c50c3d01da7f3a6601f7073cd5b163-e9905c0cd33d0950ae7800002cb72ff6 7140759 517776 B76 04/af/04afe7deb8e6ee99153433d2845da417e54cd042-ea905c0cd33d0950ae7800002cb72ff6 7769983 2317979 B76 05/13/0513bcfceff303125f233ad2c01c5ba2ed96c6a2-eb905c0cd33d0950ae7800002cb72ff6 10214312 3097649 B76 35/e4/35e46902b3e6473b9689a92acd71e58fb7165a8f-ec905c0cd33d0950ae7800002cb72ff6 msg.ext-ref = 75027 1291257 B76 b9/dc/b9dcd6899ae65e5c11b122d7bfc3be9fefc21024-5df010068b3f0950c27d00002cb72ff6 1441078 1131344 B76 f6/e6/f6e63f000d6501be472629747448057b122104c1-5ef010068b3f0950c27d00002cb72ff6 2572595 2218094 B76 93/96/9396c5eaeac2615119e55c67fa8f010332ba0fd3-5ff010068b3f0950c27d00002cb72ff6 4790862 2211695 B76 cc/a5/cca5607fb739306f3628a19575dc41432f74a22d-60f010068b3f0950c27d00002cb72ff6 7002730 2614603 B76 66/10/661002c8039997174e34b9ef31d0e693a556eebe-61f010068b3f0950c27d00002cb72ff6 9617506 2760312 B76 8c/65/8c656fe835af26c175337cd318daca8ae8e00369-62f010068b3f0950c27d00002cb72ff6 12377991 2341764 B76 19/c8/19c83e0bf1284e74e49feecaf95506266201551d-63f010068b3f0950c27d00002cb72ff6 15209343 406758 B76 b6/62/b66216837cc48422e22e7a9a22631f840a49ef78-64f010068b3f0950c27d00002cb72ff6 15616301 136877 B76 06/9f/069f5ab86dc9e8e9972f3f5c0dda03c1f3103730-65f010068b3f0950c27d00002cb72ff6 15753350 971075 B76 a7/7c/a77c36690ff0f0f774b82efaf15f93535ba027e9-66f010068b3f0950c27d00002cb72ff6 16849194 1197333 B76 4f/28/4f2881be6d0e8a7f53c0e226c0dbb148b05674c7-67f010068b3f0950c27d00002cb72ff6 18168424 850768 B76 92/72/9272e1ea7ceb79df6222686bf157f957fa9851c1-68f010068b3f0950c27d00002cb72ff6 19019393 135641 B76 60/fd/60fdcd7851c8f0a21f342aaafce9e49a3e00e1aa-69f010068b3f0950c27d00002cb72ff6 19155207 897179 B76 63/59/6359abf4f9e806e3990e0d6590e519924c838fa5-6af010068b3f0950c27d00002cb72ff6 20169966 1022612 B76 f8/65/f8654367f5df050d23565644e83c8c50abb69c39-6bf010068b3f0950c27d00002cb72ff6 But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. Thanks a lot Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From tss at iki.fi Tue Oct 30 15:42:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 30 Oct 2012 15:42:49 +0200 Subject: [Dovecot] copymail deleted In-Reply-To: <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> Message-ID: On 30.10.2012, at 15.28, Christian R??ner wrote: > msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 > But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. They are all base64 yes, the B76 means that all the encoded lines will be 76 chars long. So the file size above needs to be 1282212, divided by 77 (76+LF) = 16652 full lines and 8 bytes over. Base64 encodes 3 byte blocks into 4 byte chars, so the original data has (16652*76+8)/4*3 = 949170 bytes (or 1-2 bytes less, but that makes no difference because it's padded anyway). So if you create /attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 that is 949170 bytes long, and do the same for the rest of the attachments, you should be able to read this mail without errors. You can easily create the files without wasting space with: dd if=/dev/zero of=foo bs=1 seek=949169 count=1 From cr at sys4.de Tue Oct 30 16:44:01 2012 From: cr at sys4.de (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Tue, 30 Oct 2012 15:44:01 +0100 Subject: [Dovecot] copymail deleted In-Reply-To: References: <2B52CF76-2638-45C8-BD75-1773EAB99D0E@iki.fi> <62B9745B-844F-4A83-8B87-F5DED1389180@sys4.de> Message-ID: <09B9ED24-9319-48A7-85D4-0FF7D12F6296@sys4.de> Hi, >> msg.ext-ref = 83713 1282212 B76 6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 > >> But I did not understand the base64 explanation. Sorry :) For me it seems all "options" are B-prefixed. So they are all base64? But which value is now the size and how do I create the missing files now? Using dd? Can you give me an example from the output above? That would help me. > > They are all base64 yes, the B76 means that all the encoded lines will be 76 chars long. So the file size above needs to be 1282212, divided by 77 (76+LF) = 16652 full lines and 8 bytes over. Base64 encodes 3 byte blocks into 4 byte chars, so the original data has (16652*76+8)/4*3 = 949170 bytes (or 1-2 bytes less, but that makes no difference because it's padded anyway). > > So if you create /attachments/6a/50/6a506530265ef7c9feb396410eaf6946036e9a79-b034401e794009503a0400002cb72ff6 that is 949170 bytes long, and do the same for the rest of the attachments, you should be able to read this mail without errors. > > You can easily create the files without wasting space with: > dd if=/dev/zero of=foo bs=1 seek=949169 count=1 Thanks. I have calculated both other files and recreated zero padded files. Now I am going to watch the log file and see, if errors are gone. One last question: If the user now opens a mail, where the attachments are broken and he/she removes the mail, are the created hand-made files be removed automatically? Thanks in advance Kind regards -Christian R??ner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From dmiller at amfes.com Tue Oct 30 17:00:19 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 30 Oct 2012 08:00:19 -0700 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 Message-ID: I'm compiling as I normally do. The config line for Dovecot is: configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap followed by make & make install Then a 'configure' for Pigeonhole, followed by make, yields: libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sieve-dump sieve-dump.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_new_null at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_peer_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_load_verify_locations at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_rsa_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_use_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_desc_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_new_index at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_client_CA_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_ext_d2i at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_accept at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_cert at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_server_method at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OBJ_txt2nid at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_write at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_num at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_get_write_guarantee at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_cipher_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_push at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_get_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_alert_type_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_COMP_get_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_by_id at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_write at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `i2d_DHparams at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_ENTRY_get_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_info_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_get_subject_name at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_init at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_clear_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_load_error_strings at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_RSA at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CIPHER_get_bits at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_INFO_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_set_flags at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_new at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_dup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_index_by_NID at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `d2i_DHparams at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_text_by_NID at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `OPENSSL_add_all_algorithms_noconf at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_DSA at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_cleanup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_get_entry at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_type at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_library_init at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_cipher at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_cipher_list at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_connect at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_error_string_n at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_CTX_get_ex_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_verify at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_bio_pair at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSLv23_client_method at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_use_certificate at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_length at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_get_cert_store at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_set_tmp_dh_callback at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_set_default_ciphers at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_PrivateKey at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_new_mem_buf at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_ex_data_X509_STORE_CTX_idx at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_pop_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_value at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `GENERAL_NAME_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_version at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RAND_bytes at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_new at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_ctrl_pending at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_set_bio at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ASN1_STRING_data at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_load_builtin_engines at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_read_bio_X509 at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_cleanup at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_STORE_add_crl at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `BIO_read at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ENGINE_finish at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_read at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_NAME_oneline at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_free_strings at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `EVP_PKEY_free at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_current_compression at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_verify_cert_error_string at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_CTX_ctrl at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `ERR_peek_last_error at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `RSA_generate_key at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_state_string_long at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `PEM_X509_INFO_read_bio at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `DH_generate_parameters at OPENSSL_1.0.0' /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `X509_free at OPENSSL_1.0.0' collect2: ld returned 1 exit status make[3]: *** [sieve-dump] Error 1 make[3]: *** Waiting for unfinished jobs.... -- Daniel From rob0 at gmx.co.uk Tue Oct 30 18:23:18 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 30 Oct 2012 11:23:18 -0500 Subject: [Dovecot] POLL: v2.2 to allow one mail over quota? In-Reply-To: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> References: <9BCD2E19-0836-4885-9459-97692F8327B1@iki.fi> Message-ID: <20121030162318.GB3672@harrier.slackbuilds.org> On Mon, Oct 29, 2012 at 10:39:51PM +0200, Timo Sirainen wrote: > Currently if user is 1MB under quota and someone tries to deliver > mail that is over 1MB, Dovecot rejects the mail. But smaller mails > aren't rejected probably for days. So user might not even realize > that they didn't receive one of the mails. Also having a user > "almost over quota" is a rather strange state I think. > > So what do you think about v2.2 allowing delivery of one last mail > even if it brings the user over quota? Except add a limit that if > the message size is as much as the user's entire quota limit it > wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, > since user would get an error anyway. I could make this also > optional, but if nobody really wants to keep the old behavior > there's really no point in adding the option. I think the thing to do is to adjust the admin's thinking about it. Yes, if the current mailstore is under quota, by all means, you should accept the next email up to the maximum size the server accepts. No exception, just take it. You control $quota and $maxMsg. Set your quota with that in mind, where $(($quota - 1 + $maxMsg)) total is something you can live with. That said, I have been fortunate to never have to set up a quota. Storage is cheap. An occasional cron job can point out individual users who might be beyond what you'd consider reasonable, and to those users, apply a LART. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From cgregoir99 at yahoo.com Tue Oct 30 22:57:36 2012 From: cgregoir99 at yahoo.com (Christian Gregoire) Date: Tue, 30 Oct 2012 20:57:36 +0000 (GMT) Subject: [Dovecot] POP3 Proxy : user format not accepted In-Reply-To: References: <1351598507.54968.YahooMailNeo@web172404.mail.ir2.yahoo.com> Message-ID: <1351630656.22457.YahooMailNeo@web172405.mail.ir2.yahoo.com> Indeed, I set its value to empty to allow all characters and it now works. Thanks a lot Timo. ________________________________ De?: Timo Sirainen ??: Christian Gregoire ; Dovecot Mailing List Envoy? le : Mardi 30 octobre 2012 14h08 Objet?: Re: [Dovecot] POP3 Proxy : user format not accepted On 30.10.2012, at 14.01, Christian Gregoire wrote: > Any idea ? Is the '#' not internally supported in the user login by Dovecot ? See auth_username_chars setting. From stephan at rename-it.nl Tue Oct 30 23:46:02 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 30 Oct 2012 22:46:02 +0100 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: References: Message-ID: <50904A9A.8030703@rename-it.nl> On 10/30/2012 4:00 PM, Daniel L. Miller wrote: > I'm compiling as I normally do. The config line for Dovecot is: > > configure --with-ldap --with-ssl --with-bzlib --with-zlib > --with-stemmer --with-lucene --with-ldap > > followed by make & make install > > Then a 'configure' for Pigeonhole, followed by make, yields: What kind of system are you compiling this on? Regards, Stephan. From cliff at clamjuice.org Wed Oct 31 00:29:21 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Tue, 30 Oct 2012 18:29:21 -0400 Subject: [Dovecot] Unable to get Managesieve working Message-ID: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> I have Roundcube webmail (v. 0.8.2) running with the managesieve plugin (v. 5.1). I am able to create sieve rules without any problems in Roundcube, but incoming mail is not being processed with the rule I specify. I suspect there is something that isn't configured correctly in dovecot, but unfortunately I am unable to find a resolution online and my very limited knowledge of dovecot isn't allowing me to solve this problem on my own. I am including my dovecot config below. Please keep my knowledge level in mind when suggesting options and let me know if there is any other information I can provide to help troubleshoot the problem. Thanks for any assistance! # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key ssl_cipher_list: ALL:!LOW:!SSLv2 login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_location: maildir:~/mail:LAYOUT=fs:INBOX=~/mail/ mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve auth default: mechanisms: plain login passdb: driver: pam passdb: driver: pam userdb: driver: passwd userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: sieve: ~/.dovecot.sieve sieve_dir: ~/mail/sieve From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 01:37:09 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 00:37:09 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> Message-ID: <20121030233709.GA14111@daniel.localdomain> Hi Cliff, Cliff Dunn wrote: > I have Roundcube webmail (v. 0.8.2) running with the managesieve > plugin (v. 5.1). I am able to create sieve rules without any > problems in Roundcube, but incoming mail is not being processed with > the rule I specify. managesieve is the service for managing your sieve rules, it does not process any emails. In order to actually sort mails into folders, you need to add the "sieve" plugin to your mail_plugins list. http://wiki.dovecot.org/LDA/Sieve/Dovecot protocol lda { .. # Support for dynamically loadable plugins. mail_plugins is a space separated # list of plugins to load. mail_plugins = sieve # ... other plugins like quota } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 02:09:57 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 01:09:57 +0100 Subject: [Dovecot] lmtp out of memory - raw backtrace Message-ID: <20121031000957.GA15191@daniel.localdomain> Hi, Our setup: - 4 hosts with director and mailbox instance - delivery via director lmtp into mailbox lmtp - mailbox format: mdbox - storage on NFS - OS: Linux 2.6.32-44-server x86_64 Ubuntu 10.04.4 LTS - Dovecot 2.1.10 - Pigeonhole 0.3.3 We're getting strange "out of memory" lmtp errors/backtrace with dovecot 2.1.10 accompanied by high load caused caused by a lot of lmtp deliveries to one user. First action would be to increase vsz_limit to a higher value, but I just want to make sure there is no bug - before blindly increasing this limit. I think it should not crash and corrupt mdbox, even if memory limit is reached: Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Fatal: pool_system_realloc(16777216): Out of memory Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4271a) [0x7f6dcbae971a] -> /usr/lib/dovecot/libdovecot.so.0(+0x42766) [0x7f6dcbae 9766] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f6dcbabd158] -> /usr/lib/dovecot/libdovecot.so.0(+0x53690) [0x7f6dcbafa690] -> /usr/lib/dovecot/libdovecot.so.0(+0x3e6f5) [0x7f6dcbae56f5] -> /usr/lib/dove cot/libdovecot.so.0(buffer_write+0x7c) [0x7f6dcbae5e7c] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8033) [0x7f6dcbdda033] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_record_map_move_to_private+0x3 c) [0x7f6dcbdda4ec] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_get_atomic_map+0x18) [0x7f6dcbde9c88] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb7cf9) [0x7f6dcbde9cf9] -> /usr/lib/dovecot/lib dovecot-storage.so.0(mail_index_sync_record+0x7e6) [0x7f6dcbdea626] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x23c) [0x7f6dcbdeae4c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_m ap+0xa8) [0x7f6dcbddc2b8] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb42da) [0x7f6dcbde62da] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb458a) [0x7f6dcbde658a] -> /usr/lib/dovecot/libdovecot-storage.so.0( mail_index_sync_begin_to+0x4f) [0x7f6dcbde6ecf] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1e) [0x7f6dcbde6f4e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_atomic_lock+0x5e) [0x 7f6dcbd6115e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_transaction_save_commit_pre+0x46) [0x7f6dcbd64fb6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9d4f3) [0x7f6dcbdcf4f3] -> /usr/lib/dovecot/libdov ecot-storage.so.0(mail_index_transaction_commit_full+0x9f) [0x7f6dcbddd97f] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x8a) [0x7f6dcbdcf18a] -> /usr/lib/dovecot/modules/lib10_quota_plug in.so(+0xba7f) [0x7f6dca4eca7f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x3d) [0x7f6d Oct 29 20:14:56 10.129.3.249 dovecot: lmtp(26698, user1 at example.org): Fatal: master: service(lmtp): child 26698 returned error 83 (Out of memory (service lmtp { vsz_limit=256 MB }, you may need to increase it)) Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1 at example.org): Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 01:15:49 10.129.3.249 dovecot: lmtp(17927, user1 at example.org): Error: mmap_anon(216690688) failed: Cannot allocate memory Oct 30 10:32:27 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: Corrupted dbox file /mail/dovecot/example.org/user1/mail/storage/m.3577 (around offset=1844402): EOF reading msg header (got 0/30 bytes) Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Cron/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/postfix/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:44 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/Lists/Dovecot/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:33:46 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: /mail/dovecot/example.org/user1/mail/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: 2ZCXNsuej1CcUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHO029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21404, user1 at example.org): Error: 2ZCXNsuej1CcUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: PFBjN8uej1CdUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHP029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:32:59] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: spYhE82ej1CiUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHQ029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: sOscMs2ej1CrUwAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300932.q9U9URHS029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:01] Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21410, user1 at example.org): Error: spYhE82ej1CiUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21405, user1 at example.org): Error: PFBjN8uej1CdUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:16 10.129.3.249 dovecot: lmtp(21419, user1 at example.org): Error: sOscMs2ej1CrUwAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: +jXqE+uej1AiVAAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300933.q9U9URHi029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21538, user1 at example.org): Error: +jXqE+uej1AiVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: Timeout (180s) while waiting for lock for transaction log file /mail/dovecot/example.org/user1/mail/storage/dovecot.map.index.log Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: CxDMM+uej1AnVAAAk4785w: sieve: mailbox: deliver: session= msgid=<201210300933.q9U9URHl029943 at common.example-hosting.net> from=MAILER-DAEMON at common.example-hosting.net: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-10-30 10:33:31] Oct 30 10:36:46 10.129.3.249 dovecot: lmtp(21543, user1 at example.org): Error: CxDMM+uej1AnVAAAk4785w: sieve: script /mail/dovecot/example.org/user1/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /mail/dovecot/example.org/user1/.dovecot.sieve.log may reveal additional details) Regards Daniel -- https://plus.google.com/103021802792276734820 From cliff at clamjuice.org Wed Oct 31 02:17:56 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Tue, 30 Oct 2012 20:17:56 -0400 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <20121030233709.GA14111@daniel.localdomain> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> Message-ID: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Ok, so when I add the mail_plugins = sieve I get: sudo service dovecot restart Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found from directory /usr/lib/dovecot/modules/imap Error: imap dump-capability process returned 89 Fatal: Invalid configuration in /etc/dovecot/dovecot.conf failed I am assuming something is missing here? On 2012-10-30 19:37, Daniel Parthey wrote: > Hi Cliff, > > Cliff Dunn wrote: >> I have Roundcube webmail (v. 0.8.2) running with the managesieve >> plugin (v. 5.1). I am able to create sieve rules without any >> problems in Roundcube, but incoming mail is not being processed with >> the rule I specify. > > managesieve is the service for managing your sieve rules, > it does not process any emails. > > In order to actually sort mails into folders, you need to add > the "sieve" plugin to your mail_plugins list. > > http://wiki.dovecot.org/LDA/Sieve/Dovecot > > protocol lda { > .. > # Support for dynamically loadable plugins. mail_plugins is a space > separated > # list of plugins to load. > mail_plugins = sieve # ... other plugins like quota > } > > Regards > Daniel From tss at iki.fi Wed Oct 31 02:20:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 31 Oct 2012 02:20:34 +0200 Subject: [Dovecot] lmtp out of memory - raw backtrace In-Reply-To: <20121031000957.GA15191@daniel.localdomain> References: <20121031000957.GA15191@daniel.localdomain> Message-ID: On 31.10.2012, at 2.09, Daniel Parthey wrote: > - Dovecot 2.1.10 > - Pigeonhole 0.3.3 > > We're getting strange "out of memory" lmtp errors/backtrace with dovecot 2.1.10 > accompanied by high load caused caused by a lot of lmtp deliveries to one user. > > First action would be to increase vsz_limit to a higher value, but I just > want to make sure there is no bug - before blindly increasing this limit. Recent changes in hg should help with this for LMTP. I'll probably release 2.1.11 somewhat soon. > I think it should not crash and corrupt mdbox, even if memory limit is reached: It should crash, because it reached the enforced vsz limit. If you don't want it to crash you can disable the limit, at the cost of potentially eating all of your memory. > Oct 30 10:32:27 10.129.3.249 dovecot: mailbox: mail: imap(user1 at example.org): : Error: Corrupted dbox file /mail/dovecot/example.org/user1/mail/storage/m.3577 (around offset=1844402): EOF reading msg header (got 0/30 bytes) This does look like something that should be fixed. It's not exactly a corruption so much as finding a partially written mail during rescan (because of the crash), but still it can probably be avoided. From daniel.parthey at informatik.tu-chemnitz.de Wed Oct 31 02:45:12 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 31 Oct 2012 01:45:12 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Message-ID: <20121031004511.GA15928@daniel.localdomain> Cliff Dunn wrote: > >In order to actually sort mails into folders, you need to add > >the "sieve" plugin to your mail_plugins list. > > > >http://wiki.dovecot.org/LDA/Sieve/Dovecot > > > >protocol lda { > > mail_plugins = sieve > >} > Ok, so when I add the mail_plugins = sieve I get: > sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found > from directory /usr/lib/dovecot/modules/imap > Error: imap dump-capability process returned 89 > Fatal: Invalid configuration in /etc/dovecot/dovecot.conf > failed > > I am assuming something is missing here? I guess you are missing the sieve plugin completely, did you install the dovecot-sieve plugin? Where did you get sieve from and how did you install it? Did you follow all the compile instructions at http://wiki.dovecot.org/LDA/Sieve/Dovecot or did you install some package? What do the following commands say? # find /usr/lib/dovecot | grep sieve # apt-cache policy dovecot-sieve # dpkg -l "dovecot*" Regards Daniel -- https://plus.google.com/103021802792276734820 From dmiller at amfes.com Wed Oct 31 03:11:50 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 30 Oct 2012 18:11:50 -0700 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: <50904A9A.8030703@rename-it.nl> References: <508FEB83.5010209@amfes.com> <50904A9A.8030703@rename-it.nl> Message-ID: On 30.10.2012 14:46, Stephan Bosch wrote: > On 10/30/2012 4:00 PM, Daniel L. Miller wrote: > >> I'm compiling as I normally do. The config line for Dovecot is: configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap followed by make & make install Then a 'configure' for Pigeonhole, followed by make, yields: > > What kind of system are you compiling this on? AMD Opteron 4180, Ubuntu Precision, Linux 3.2.0-25, 64-bit. libssl-dev 1.0.1c-3ubuntu1 libc6 2.15 gcc 4.6.3 -- Daniel From calestyo at scientia.net Wed Oct 31 03:13:03 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 02:13:03 +0100 Subject: [Dovecot] maildir S= and W= Message-ID: <1351645983.24721.0.camel@fermat.scientia.net> Hi. Even new mails delivered by my MDA don't get the ,S= and ,W= fields set... (but when I "upload" a mail via IMAP to dovecot, they are set) Is there some place in dovecot where I need to enable this? Or would it be the MDA that has to calculate and set this already when placing a file in ./new? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5113 bytes Desc: not available URL: From calestyo at scientia.net Wed Oct 31 03:50:23 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 02:50:23 +0100 Subject: [Dovecot] maildir and end-of-line encoding Message-ID: <1351648223.24721.4.camel@fermat.scientia.net> Hi. I just wondered, the following: My MDA may get mails that use LF or CR/LF end of line encodings and deliver them into maildirs. I couldn't find any information about, whether one should or must convert all into one format, cause AFAIK at least on the IMAP side, CR/LF is always used? How does this work on the maildir/backend side of dovcot? Can it work with both and simply automatically convert LF into CR/LF? Thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5113 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 09:20:43 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 08:20:43 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351645983.24721.0.camel@fermat.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> Message-ID: <5090D14B.9080805@skye.it> Il 31/10/2012 02:13, Christoph Anton Mitterer ha scritto: > Hi. > > Even new mails delivered by my MDA don't get the ,S= and ,W= fields > set... > (but when I "upload" a mail via IMAP to dovecot, they are set) > > > Is there some place in dovecot where I need to enable this? Or would it > be the MDA that has to calculate and set this already when placing a > file in ./new? The MDA should calculate and set this, dovecot always add these informations, as you can see when upload file via IMAP. Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From skdovecot at smail.inf.fh-brs.de Wed Oct 31 11:08:10 2012 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 31 Oct 2012 10:08:10 +0100 (CET) Subject: [Dovecot] backtrace for non-existant %{ldap:attr} on login Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I'm fetching the user and auth data from LDAP, this is the string: pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,mailUidNumber=userdb_uid,mailGidNumber=userdb_gid,mailLocationDovecot=userdb_mail,uid=userdb_user,=userdb_quota_rule=*:bytes=%{ldap:mailQuotaBytes}, =userdb_quota_rule2=Trash:bytes=+%{ldap:mailQuotaBytesTrash} If mailQuotaBytesTrash or mailQuotaBytes is not present, the LOGIN process does not work: 1 login user pwd 1 NO [UNAVAILABLE] Temporary authentication failure. [mailsrv2:2012-10-31 08:56:51] * OK Waiting for authentication process to respond.. If I add those two attributes, the user can login successfully. dovecot-2.1.10/sbin/dovecot --version 2.1.10 (9cdeab12f3e1) The log entries: 2012-10-31 09:56:51 auth: Panic: pool_data_stack_realloc(): stack frame changed 2012-10-31 09:56:51 auth: Error: Raw backtrace: /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x4857a) [0x7f2c0528c57a] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x485c6) [0x7f2c0528c5c6] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f2c0525feaf] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x58f2e) [0x7f2c0529cf2e] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x442f5) [0x7f2c052882f5] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x68) [0x7f2c05288728] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(str_vprintfa+0x6d) [0x7f2c052a796d] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(str_printfa+0x88) [0x7f2c052a7ac8] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x42682e] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(var_expand_with_funcs+0x6fb) [0x7f2c052ac48b] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb](db_ldap_result_iterate_next+0x12f) [0x42734f] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x428974] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x428f5f] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x4282a1] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f2c05298756] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f2c0529978f] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f2c052986f8] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f2c052840d3] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb](main+0x376) [0x41bba6] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f2c04449c8d] -> dovecot2.1/auth [0 wait, 1 passdb, 0 userdb]() [0x40cf89] 2012-10-31 09:56:51 auth: Fatal: master: service(auth): child 15865 killed with signal 6 (core dumps disabled) 2012-10-31 09:56:51 imap-login: Warning: Auth connection closed with 1 pending requests (max 0 secs, pid=15869, EOF) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUJDqe2oxLS8a3A9mAQI+YQf/Qd4IIeM35Hmmpl1IMcQwJFK4854G5Dku yK+GsWhE2gxI6KaLO6DSI/kpN79qhQRkHsUAHzoPiZ7kQpZprNaEP/CIPkTzw//i HyC2Odpfa8fWUOqtH5Cp6X5spF6hQa4mmQyzgXguF9bdjZkNu4vQ78wRuQGG4eHi BOkNL0b93DsN3NSIoXDpiAiJn1aleTLe7mYkfsGewjb+AN+FpR4hLbf32yRRn8J0 Fkn8agEijixGMXEBD4ZprTbX9NbUr92YPfRycMnA2A00MUUlv/iKlqXRpMLafRjL bnHL/QE80xHoKKJUR96/RjruciIolAtlyYvhTU9ibRpLYm5Hcd9bZg== =iPvg -----END PGP SIGNATURE----- From fabio.ferrari at unimore.it Wed Oct 31 13:12:41 2012 From: fabio.ferrari at unimore.it (FABIO FERRARI) Date: Wed, 31 Oct 2012 12:12:41 +0100 Subject: [Dovecot] Dovecot stops to work - anvil problem In-Reply-To: <51972B14-6973-4510-870D-956F858FC76B@iki.fi> References: <50e548774960a3a57cf060470783c9ed.squirrel@webmail2.unimore.it> <51972B14-6973-4510-870D-956F858FC76B@iki.fi> Message-ID: <0d40f13a4256adae8f084c385dcd0fd3.squirrel@webmail2.unimore.it> Thank you very much for your help, I cross mi fingers but it seems that this was the problem. Fabio Ferrari > On 26.10.2012, at 13.24, FABIO FERRARI wrote: > >> Hi all, >> >> we have a problem about anvil, it seems that when we have a high load >> the >> dovecot stops to work. Sometimes it is sufficient to make a dovecot >> reload, but sometimes we have to restart it. >> >> Oct 26 11:13:55 anvil: Error: net_accept() failed: Too many open files > > This is the problem. > >> And these are the limit settings in the OS: >> * soft nofile 131072 >> * hard nofile 131072 >> >> Have someone had the same problem? > > The OS limits are ok. But you need to make sure that the dovecot processes > have enough fds in ulimit. You can check the limits with: > > cat /proc//limits > > The "Max open files" soft limit is what you're most likely hitting. Use > "ulimit -n 10000" or something before running dovecot binary. And make > sure that it changes the limit in the proc. Many init scripts change the > ulimit internally. > > From tss at iki.fi Wed Oct 31 13:26:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 31 Oct 2012 13:26:11 +0200 Subject: [Dovecot] Pigeonhole 3.3 broken against Dovecot 2.1.10 In-Reply-To: References: Message-ID: On 30.10.2012, at 17.00, Daniel L. Miller wrote: > I'm compiling as I normally do. The config line for Dovecot is: > > configure --with-ldap --with-ssl --with-bzlib --with-zlib --with-stemmer --with-lucene --with-ldap > > followed by make & make install > > Then a 'configure' for Pigeonhole, followed by make, yields: > libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sieve-dump sieve-dump.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot > libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -o .libs/sievec sievec.o -Wl,--export-dynamic ../../src/lib-sieve/.libs/libdovecot-sieve.so ../../src/lib-sieve-tool/.libs/libsieve-tool.a /usr/local/lib/dovecot/libdovecot-storage.so /usr/local/lib/dovecot/libdovecot-lda.so -L/usr/local/lib/dovecot /usr/local/lib/dovecot/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot > /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `sk_new_null at OPENSSL_1.0.0' > /usr/local/lib/dovecot/libdovecot-storage.so: undefined reference to `SSL_get_error at OPENSSL_1.0.0' I think this is a Dovecot bug, fixed by: http://hg.dovecot.org/dovecot-2.1/rev/7d931927e4ac You could also do this by adding -lssl -lcrypto manually to the installed dovecot-config and running configure again for pigeonhole. From calestyo at scientia.net Wed Oct 31 13:41:51 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 12:41:51 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <5090D14B.9080805@skye.it> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> Message-ID: <1351683711.7825.0.camel@heisenberg.scientia.net> On Wed, 2012-10-31 at 08:20 +0100, Alessio Cecchi wrote: > The MDA should calculate and set this, dovecot always add these > informations, as you can see when upload file via IMAP. Ah thanks,... do you know whether it's possible to have them set by maildrop? I couldn't find anything on this. Cheers, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 14:04:27 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 13:04:27 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351683711.7825.0.camel@heisenberg.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> Message-ID: <509113CB.7020402@skye.it> Il 31/10/2012 12:41, Christoph Anton Mitterer ha scritto: > On Wed, 2012-10-31 at 08:20 +0100, Alessio Cecchi wrote: >> The MDA should calculate and set this, dovecot always add these >> informations, as you can see when upload file via IMAP. > Ah thanks,... do you know whether it's possible to have them set by > maildrop? I couldn't find anything on this. My maildrop (2.4) version, working with qmail and vpopmail, add S= by default. Probably you are running an old version without Maildir++ support: http://www.inter7.com/courierimap/README.maildirquota.html ============== Delivering to a Maildir++ Delivering to a Maildir++ is like delivering to a Maildir, with the following exceptions: Follow the usual Maildir conventions for naming the filename used to store the message, except that append ,S=nnnnn to the name of the file, where nnnnn is the size of the file. This eliminates the need to stat() most messages when calculating the quota. If the size of the message is not known at the beginning, append ,S=nnnnn when renaming the message from tmp to new. ============== Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From cliff at clamjuice.org Wed Oct 31 14:47:10 2012 From: cliff at clamjuice.org (Cliff Dunn) Date: Wed, 31 Oct 2012 08:47:10 -0400 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <20121031004511.GA15928@daniel.localdomain> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> <20121031004511.GA15928@daniel.localdomain> Message-ID: <34eb54a518ab2a0cdff5709b9e8bfac6@mail.clamjuice.org> I didn't follow those instructions as I assumed that it would be installed with Dovecot from the Debian repositories. See output of commands below. And thanks again for the help! # find /usr/lib/dovecot | grep sieve /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.so /usr/lib/dovecot/modules/lda/lib90_sieve_plugin.la /usr/lib/dovecot/managesieve /usr/lib/dovecot/managesieve-login # apt-cache policy dovecot-sieve N: Unable to locate package dovecot-sieve # dpkg -l "dovecot*" Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-============================-============================-======================================================================== un dovecot (no description available) ii dovecot-common 1:1.2.15-7 secure mail server that supports mbox and maildir mailboxes ii dovecot-imapd 1:1.2.15-7 secure IMAP server that supports mbox and maildir mailboxes On 2012-10-30 20:45, Daniel Parthey wrote: > Cliff Dunn wrote: >> >In order to actually sort mails into folders, you need to add >> >the "sieve" plugin to your mail_plugins list. >> > >> >http://wiki.dovecot.org/LDA/Sieve/Dovecot >> > >> >protocol lda { >> > mail_plugins = sieve >> >} >> Ok, so when I add the mail_plugins = sieve I get: >> sudo service dovecot restart >> Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found >> from directory /usr/lib/dovecot/modules/imap >> Error: imap dump-capability process returned 89 >> Fatal: Invalid configuration in /etc/dovecot/dovecot.conf >> failed >> >> I am assuming something is missing here? > > I guess you are missing the sieve plugin completely, > did you install the dovecot-sieve plugin? > Where did you get sieve from and how did you install it? > > Did you follow all the compile instructions at > http://wiki.dovecot.org/LDA/Sieve/Dovecot or > did you install some package? > > What do the following commands say? > > # find /usr/lib/dovecot | grep sieve > # apt-cache policy dovecot-sieve > # dpkg -l "dovecot*" > > Regards > Daniel From stephan at rename-it.nl Wed Oct 31 15:03:23 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 31 Oct 2012 14:03:23 +0100 Subject: [Dovecot] Unable to get Managesieve working In-Reply-To: <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> References: <5156445d59b6876846d7550ac32f647e@mail.clamjuice.org> <20121030233709.GA14111@daniel.localdomain> <91ac06a291da8cf523a8cbd4b0177139@mail.clamjuice.org> Message-ID: <5091219B.6000709@rename-it.nl> Op 10/31/2012 1:17 AM, Cliff Dunn schreef: > Ok, so when I add the mail_plugins = sieve I get: > sudo service dovecot restart > Restarting IMAP/POP3 mail server: dovecotFPlugin sieve not found from > directory /usr/lib/dovecot/modules/imap > Error: imap dump-capability process returned 89 > Fatal: Invalid configuration in /etc/dovecot/dovecot.conf > failed > > I am assuming something is missing here? You should only put the mail_plugins=sieve inside de protocol lda {} section. Adding the Sieve plugin to IMAP makes no sense. Regards, Stephan. From calestyo at scientia.net Wed Oct 31 15:55:32 2012 From: calestyo at scientia.net (Christoph Anton Mitterer) Date: Wed, 31 Oct 2012 14:55:32 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <509113CB.7020402@skye.it> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> <509113CB.7020402@skye.it> Message-ID: <1351691732.8425.1.camel@heisenberg.scientia.net> On Wed, 2012-10-31 at 13:04 +0100, Alessio Cecchi wrote: > maildrop (2.4) Ah thanks... yeah I had an old version.. > add S= by > default. > http://www.inter7.com/courierimap/README.maildirquota.html AFAIU that... ,W= is however not set, right? :( thanks, Chris. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5165 bytes Desc: not available URL: From alessio at skye.it Wed Oct 31 16:09:51 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 31 Oct 2012 15:09:51 +0100 Subject: [Dovecot] maildir S= and W= In-Reply-To: <1351691732.8425.1.camel@heisenberg.scientia.net> References: <1351645983.24721.0.camel@fermat.scientia.net> <5090D14B.9080805@skye.it> <1351683711.7825.0.camel@heisenberg.scientia.net> <509113CB.7020402@skye.it> <1351691732.8425.1.camel@heisenberg.scientia.net> Message-ID: <5091312F.1080603@skye.it> Il 31/10/2012 14:55, Christoph Anton Mitterer ha scritto: > On Wed, 2012-10-31 at 13:04 +0100, Alessio Cecchi wrote: >> maildrop (2.4) > Ah thanks... yeah I had an old version.. > > >> add S= by >> default. >> http://www.inter7.com/courierimap/README.maildirquota.html > AFAIU that... ,W= is however not set, right? :( Yes, W= is not set by maildrop, only dovecot-lda set W=, anyway S=size is sufficient for speed up the system. If the message was stored with CR+LF linefeeds, and are the same. More info here: http://wiki2.dovecot.org/MailboxFormat/Maildir Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From jpdalbec at ysu.edu Wed Oct 31 17:15:56 2012 From: jpdalbec at ysu.edu (John Dalbec) Date: Wed, 31 Oct 2012 11:15:56 -0400 Subject: [Dovecot] IMAP proxy between Office 365 client/Sun Messaging Server Message-ID: <509140AC.80602@ysu.edu> I would like to be able to migrate messages from existing end-user accounts on Sun Messaging Server to Office 365 using an administrator login and password. Unfortunately the migration tool for Office 365 doesn't support SASL AUTHENTICATE PLAIN login. The online documentation I've found suggests that I should be able to configure dovecot as an IMAP proxy and have it log in to Sun Messaging Server with AUTHENTICATE PLAIN and encode_base64("user\0admin\0adminpw") in response to a login from the Office 365 migration tool. I'd like to configure dovecot to run only the IMAP proxy if possible. I was thinking of setting all accounts to use the same (strong) password in the proxy. Would anyone be willing to share a sample configuration? Thanks, John Dalbec ellucian Luminis system administrator Youngstown State University From rs at sys4.de Wed Oct 31 17:37:53 2012 From: rs at sys4.de (Robert Schetterer) Date: Wed, 31 Oct 2012 16:37:53 +0100 Subject: [Dovecot] IMAP proxy between Office 365 client/Sun Messaging Server In-Reply-To: <509140AC.80602@ysu.edu> References: <509140AC.80602@ysu.edu> Message-ID: <509145D1.1070504@sys4.de> Am 31.10.2012 16:15, schrieb John Dalbec: > I would like to be able to migrate messages from existing end-user > accounts on Sun Messaging Server to Office 365 using an administrator > login and password. Unfortunately the migration tool for Office 365 > doesn't support SASL AUTHENTICATE PLAIN login. > > The online documentation I've found suggests that I should be able to > configure dovecot as an IMAP proxy and have it log in to Sun Messaging > Server with AUTHENTICATE PLAIN and encode_base64("user\0admin\0adminpw") > in response to a login from the Office 365 migration tool. I'd like to > configure dovecot to run only the IMAP proxy if possible. I was > thinking of setting all accounts to use the same (strong) password in > the proxy. > > Would anyone be willing to share a sample configuration? > > Thanks, > John Dalbec > ellucian Luminis system administrator > Youngstown State University perhaps look in this http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy http://wiki.dovecot.org/HowTo/ImapProxy Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich From micah at riseup.net Wed Oct 31 21:15:57 2012 From: micah at riseup.net (Micah Anderson) Date: Wed, 31 Oct 2012 15:15:57 -0400 Subject: [Dovecot] Error: Internal quota calculation error Message-ID: <87d2zyxxjm.fsf@minnow.riseup.net> Hello, I'm using 2.1.7 with seive and mysql quotas. We had an outage the other day where the database server where quotas are stored was not available for a short period of time. In dovecot land, the following types of errors occured in that scenario: Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: Internal quota calculation error Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: sieve: msgid=<20122132765181x.ABCCE457 at example.com>: failed to store into mailbox 'Trash': Internal error occurred. Refer to server log for more information. [2012-10-26 22:19:01] Oct 26 22:19:01 grosbeak dovecot: lda(example at riseup.net): Error: sieve: script /maildir/e/example/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /maildir/e/example/.dovecot.sieve.log may reveal additional details) I expect that there would be quota calculation errors as dovecot could not reach the database server, but what worried me was the 'failed to store into mailbox' message from sieve. The 'Trash' mailbox in this particular seive script is the correct location for the message to be filed into, but the worrisome message is the 'failed with unsuccessful implicit keep'. I looked through all the message-ids that reported this error and I found that the messages were properly delivered in the end, so this might be some issue interacting between sieve, dovecot and quota and just causing a scary message that can be ignored? thanks for any information you can provide, dovecot is great! micah -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available URL: From tlx at leuxner.net Wed Oct 31 21:46:27 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 31 Oct 2012 20:46:27 +0100 Subject: [Dovecot] Out of memory/Managesieve In-Reply-To: <508EFC12.4000509@jkart.de> References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> <508EFC12.4000509@jkart.de> Message-ID: Am 29.10.2012 um 22:58 schrieb Jim Knuth : > /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. Error says it all? So not really helpful. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4364 bytes Desc: not available URL: From jk at jkart.de Wed Oct 31 21:50:11 2012 From: jk at jkart.de (Jim Knuth) Date: Wed, 31 Oct 2012 20:50:11 +0100 Subject: [Dovecot] (Solved) Out of memory/Managesieve In-Reply-To: References: <508E9F71.8050208@jkart.de> <508EF86C.5070202@jkart.de> <508EFC12.4000509@jkart.de> Message-ID: <509180F3.7020608@jkart.de> am 31.10.12 20:46 schrieb Thomas Leuxner : > Am 29.10.2012 um 22:58 schrieb Jim Knuth : > >> /var/run/dovecot/login/core: Datei oder Verzeichnis nicht gefunden. > > Error says it all? So not really helpful. > yes, I know ;) But no more problems. I dont no why *bg* -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Ich schaue mir meine Filme nie an. Sie sind mir zu brutal. (Charles Bronson)